Here’s a cool new study on learning using AI:
The authors, Sarah Shi Hui Wong and Sophia Xuefei Qiu, ran a simple experiment with 196 university students and published the paper earlier this year in Educational Psychology Review. What struck me about the study was that the results seemed so obvious. I must be missing something. But maybe I came to that same conclusion after using various AI tools and just figured out on my own what they do best and what they don’t do very well at all. For me, I found that if I engaged the AI with something already in mind I was far more productive and the experience was more satisfying, whereas if I just randomly poked around with the AI with an empty mind then we both just ended up running around in circles and I learned nothing.
From the abstract:
“Thinking of one’s own ideas first, then collaborating with ChatGPT to improve them, promotes learning gains in independent human creativity.”
And with that, I knew I was on the right track and read the rest of the paper.
Wong and Qiu are researchers working in cognitive and educational psychology, and their study draws heavily on the memory and learning research that those fields have built up over the past several decades. The tradition also traces back to people like Robert Bjork, whose PhD is in mathematical psychology, and other researchers studying how the human brain encodes and retrieves information. These questions predate generative AI, but now they can be studied in the context of AI to explore new possibilities to use the technology. The findings of the ChatGPT study support my own experience since I’ve been using AI specifically for learning from the beginning. It’s the best teaching tutor I’ve ever had by far. The possibilities for education are endless. I wish I had these tools when I was a kid in school.
Here’s what they did with ChatGPT, but it works with any chat-based AI tool. They split students into three groups. The first group solved a creative task alone. The second group used ChatGPT however they liked. The third group followed what the authors called a “think first, ChatGPT later” method. So that third group of students had to think of their own ideas first and then iterate with the AI to improve those ideas. And then they’d submit a final answer on their own like the other groups.
On the first task, the group that used ChatGPT freely did the best. And if you’ve used AI at all, it should be obvious that the first group came out ahead initially. They seemed more productive. But they outsourced their own thinking entirely. Why does that matter? Well, from a longer term learning perspective, this is where it gets most interesting when you look more carefully.
Next, all three groups did a second and harder creative task with no AI access at all. At that point, the free-use group’s advantage vanished. Their scores dropped back to the level of students who never touched AI. But the guided group using AI was different. They hadn’t looked especially strong on the first task, but on the second task working alone they ended up beating the other groups. The researchers then read the chat transcripts and found a possible reason. Students using ChatGPT freely mostly just asked it for external ideas outright, just like I did when I was aimlessly messing around with AI. The guided group, however, mostly brought their own ideas to the chat conversation and asked ChatGPT to push on them. In other words, they iterated. That difference in human behavior, not the AI itself, predicted who actually learned something that resonated over time.
So why is this important? Use AI to help you learn! Simple. And we now have some data in a controlled study to substantiate that AI is a useful tool for learning beyond just anecdotal experience. Sure, AI can make you more productive really quickly, and I use it for that purpose all the time. But to actually leverage the tool it makes more sense to go in with a plan in mind so you can be both more productive and learn something valuable in the process. Then you can take what you’ve learned to focus your prompts to the AI so that you can move even more deeply into the subject. Again, this process is obvious if you’ve spent any time poking around with AI.
Others have also discovered this phenomenon as well. Tamara Tate published a piece called Think First: Why the First Idea Shouldn’t Come from AI in March 2026 before the ChatGPT study, and she made a similar argument about generative AI and writing.
As I read both pieces I found the concepts familiar because they reminded me of a book on learning I read years ago: Make It Stick: The Science of Successful Learning. Make it Stick is probably the best book on leaning I’ve ever read. Every page is pure gold for people who are motivated to learn on their own. The book is based on the same body of cognitive psychology research as the ChatGPT study, particularly the work of Robert and Elizabeth Bjork on what they called “desirable difficulties.” Wong and Qiu cite that same research to help explain why the AI guided group may have performed weaker initially but excelled later. The key is that the students were doing more work up front. Attempting retrieval on your own (thinking first) before instruction beats instruction alone followed by retrieval (testing) afterwards. That’s not a new idea. It’s just being tested on a chatbot now. We now have a new tool at our fingertips to implement an old principle.
Aside from going into an AI conversation with at least a rough plan up front, it also makes sense to spend some significant time iterating with the AI at length. The ChatGPT study only covered a 12 minute timeframe before answers were submitted. But try going longer. I can talk to the damn thing for hours just going back and forth asking and answering questions, brainstorming ideas, taking quizzes, working thorough tutorials, and anything else I can think of. I tend to have hand-written notes to the side, though, to track things that are important and need more probing. I learned this from an engineering manager I worked for at Sun in Solaris engineering where I observed her managing meetings of engineers. She asked one question after another until they solved the problem. I was surprised how deep she could go to get the engineers to think differently about the issue. She always had another question to ask, which led to additional questions the engineers would ask each each other as well. Those meeting sessions would also often lead to further conversations on the team’s mailing list. It works. And AI has infinite patience for this iteration. I guess Socrates was right.
Here’s the final sentence in the ChatGPT study:
“Generative AI is not to be a substitute for human creativity but, when strategically harnessed, a powerful tool for enhancing it.”
Makes sense to me. Working it every day.