The best learning happens one-on-one—that’s what decades of educational research shows, time and again. Benjamin Bloom, in his influential 1984 study, found that students who learned via 1:1 tutoring had significantly better educational performance—a difference of two standard deviations—than those who learned via traditional, one-to-many classroom instruction.
To hone in on this feature of effective learning— 1:1 learning support—we looked to the research around how people learn and master skills. We wanted to understand:
How could we create a learning tool, CodeSignal Learn, that provides students effective 1:1 learning support at scale?
How could we use large language models (LLMs) to facilitate this 1:1 support to promote mastery-based learning?
Creating effective 1:1 support
To start, we dug into the research on the best ways to implement 1:1 learning support and feedback. What we found was:
Across these studies, scholars agree that feedback—especially when learners seek it out themselves—makes for more effective learning.
Building an AI-powered tutor for technical skills
Given the research on how crucial 1:1 support and feedback are for effective learning, our next step was to figure out how to ensure our learning platform, CodeSignal Learn, could provide this support at scale to help people achieve mastery in a variety of technical skills. What we came up with was an AI-powered tutor that harnesses the power of multiple large language models (LLMs) to provide immediate, helpful, and context-aware feedback to learners: Cosmo.
Cosmo is an AI assistant built into the CodeSignal Learn platform that provides learners with personalized, 1:1 support. Learners can chat with Cosmo to clarify instructions for a practice exercise, look up documentation, debug code, and more.
Compared to a human tutor, an AI-powered tutor offers a range of benefits:
For employers, AI can allow organizations to scale professional development opportunities to more employees via autonomous learning. Traditional executive coaching, for instance, is highly effective at improving manager performance, but it is costly—and as a result, not accessible to most employees. AI makes this type of 1:1, personalized coaching cost-effective and scalable.
For individual learners, an AI-powered tutor provides access to 1:1 learning support at any time of the day, and for a very low cost. AI makes tutoring accessible and affordable for learners who work odd hours or who can’t afford to hire a personal tutor.
An AI–powered tutor also allows learners to ask questions freely, without the threat of being judged for asking “silly” questions in front of their peers. Asking questions allows learners to be more engaged in their learning and helps build their skills. This is particularly beneficial to women and other groups underrepresented in STEM, who are often afraid to make mistakes that could confirm stereotypes in highly competitive learning environments.
Our next step was to pilot our AI tutor to see how learners responded to it and how it impacted their learning.
Developers’ responses to an AI-powered tutor
As a part of our CodeSignal Learn Alpha program, we engaged 49 developers who expressed an interest in building skills in data structures and algorithms or data science and who had at least one year of coding experience.
Over the course of 4 months, Alpha participants took courses on CodeSignal Learn and participated in an early qualitative feedback survey, a post-course feedback survey, and in-depth, post-course 1:1 interviews.
Here’s what we learned from our post-course surveys.
Responses to Cosmo, an AI-powered tutor
In the post-course survey, we included Likert scale items to measure user experiences. We found that the reactions to the AI tutor were overwhelmingly positive again: 80-90% of Alpha participants had favorable reactions for the AI tutor questions we asked.
Some strong points were the tutor’s personality, application of context when responding, relevance of responses, and adaptation to the way the learners phrase their questions. These were all strong indicators for us that the AI tool was giving helpful and motivating feedback for learners.
Overall post-course reactions
At the end of the Alpha program:
- 100% of participants were more confident with their skills;
- 100% were satisfied with the overall experience;
- 90% would recommend CodeSignal Learn courses to others.
In both quantitative surveys and 1:1 interviews, 90% of learners said they would be glad if their employer offered CodeSignal Learn for their professional development. And, 80% of participants said they were motivated to continue learning in a format like this.
Overall, this early research left us optimistic about the potential for AI-powered tutors to open doors for developers and other learners who may not get access to this kind of feedback and practice otherwise.
Conclusion
One-on-one tutoring, once considered an impractical and costly way to teach, is now scalable and accessible thanks to recent innovations in AI. Through our Alpha program piloting a new learning platform, CodeSignal Learn, we found that learners responded positively to engaging with an AI-powered tutor and felt confident in the skills they gained from the courses they took. This research suggests that AI holds immense potential to support learning in ways that were not possible even just a few years ago: making personalized, 1:1 feedback scalable and accessible to all.
If you’d like to check out CodeSignal Learn for yourself, it is now generally available. Sign up to get started for free.
About the authors
Adam Vassar is the Director of Talent Science at CodeSignal. He holds a Master’s in Industrial-Organizational Psychology and has worked in the talent management industry for 20+ years leveraging assessments and technology for candidate selection, leadership development, team collaboration, and employee engagement.
Seterra Riggs is a Talent Scientist at CodeSignal with a PhD in Industrial-Organizational Psychology. She provides research-backed insights that drive organizational strategies and product development, focusing on Diversity, Equity, and Inclusion (DEI), talent assessment, and employee learning & development and retention. With her expertise and commitment to DEI and career development for underrepresented groups in STEM, she helps shape industry best practices.
