Learning: the hardest problem in computer science

"How do you know when you've learned something?", my senior colleague asks me.

After a year in my job of helping people learn, I still haven't thought enough about this question — and I know it. But I hate hate hate letting on about this fact, so I think for a minute or two (a feature of our company is that we always take as much thinking time as we need) and then I say:

"I know I've learned something when I can apply it. When I can do something new that I couldn't before."

He seems unsatisfied. I know this is not his favoured answer. I know because we've had this discussion before, and because since then I've read about the difference between learning and performance of that learning. But I'm stubborn.

I explain to him, unprompted, that I've learned to be suspicious of any other answers to that question. I've been burned too many times by students who have expressed understanding, often in the form of an "Oh! I get it!" and then been unable to do anything new.

Plausibly they have learned something, I admit, but it is tempting to believe that this is a step on the road to competence and I don't believe this is a fair assumption. (This is me using a metacognitive strategy, which is a new term I have learned in the past year. It means a strategy for thinking about thinking.)

I ask him, after a little too long, how he knows when he's learned something. He thinks for about half a minute.

"I know I've learned something when I can interpret my experiences. When I can make new meaning from things that meant nothing before, or meant something else."

It makes me think of file formats. Like installing Microsoft Word so you can read .docx files. It makes sense. Or... it makes meaning.

I believe at the centre of all professional disciplines, deeply embedded, there are one or two questions like this. Hard, complicated questions with thousands of existing answers but that nonetheless must be answered personally on the way to competence, albeit uneasily.

How do you know when someone else has learned something?

How can complex systems remain easy to change?

If you choose to learn to code, it is likely that no one will explain to you how programs work. It is likely that you will not meet anyone who would be capable of telling you how programs work for many years. This knowledge is extremely rare. It is, surprisingly, not really required. It is unnecessary in the same way that a decent understanding of linguistics is unnecessary to read this passage.

All this makes me extremely sad. Please let me tell you why.

We have a problem in computing education — many people are unable to learn to code. This is not a nice thing to say, but it is true. I believe it is due to our current method of teaching beginners how to code.

Here's how it works: we give you examples of variables, methods, operations, program flow, classes, etc. We also give you some analogies and stories. We give you some tasks to do that require you to learn how these things work to some degree.

What we don't do is tell you how they work. We just try to speak slowly and in simple enough terms that something in your brain catches onto what we might be talking about. And many people can make that leap. Maybe they've had experience with spreadsheets or games or other similar enough systems to adapt their understandings. But many find this too hard.

Consider this common test for beginners:

a = 1
b = 2
a = b

Many find this challenging. They can grasp it with time, but it often viewed to be self-evident and most programming requires a fluid understanding of the conceptual model is underlying this — cell-based memory (basically absent in the real world, but present in spreadsheets). This is just one example amongst many — variable scope, program flow, method calling and returning — the learner is mostly left to figure these things out.

I imagine this first leap across this unexplained zone as an unassisted leap over a wide chasm. Some make the jump, some don't, some don't try.

Our answer to frustrated learners? Watch people jump, listen to stories about how jumping is just like long dancing (you never danced), and keep trying to jump.

And the worst part? It works pretty well for a lot of people. In fact, this is how we learn a lot of things. Natural language, basic physics — you observe, you interact, you somehow develop a working model. It would be much more challenging, perhaps impossible, to develop that understanding from the theoretical basis.

But actually the same isn't true of computers. You could learn them from their basics if you wanted to. I think you'd end up being a stellar programmer and after a fairly short time be having fun too.

But more than that — you could do it reliably. Given enough time, by going from the basics most people have (arithmetic, physical space) you could build a good understanding of computers in anyone's brain. You wouldn't have to exclude people who just so happen not to have spent much time on one of the magic activities that make it easier to pick up computers.

So why sad? Because no one wants to learn this way. Humans want to gain new powers and understand what's going on around them. Very few of us wish to build up our understanding from small abstract pieces, at least not when there is a plausible alternative. It is a long, frustrating, risky process.

And so my dream becomes economically unviable. Maybe a few people would reward this effort, but not enough to make it worthwhile. I concluded, after lengthy reflection, that this simply isn't the right universe for that idea.

I teach. It's a scary word. We don't often use the word 'teach' to apply to adults — it implies a power relationship that, ironically, is probably harmful to learning. I work at a company called Makers where we use the word 'coach'. I compromise publicly on the word 'educator', which another colleague told me she hated, but it feels like the best of a bad bunch.

Secretly I still say 'teach' in my head.

I teach. I help people learn. Sometimes with words, sometimes with tasks, sometimes with environments. It's my job to design and implement learning experiences — or, perhaps I should say, learnable experiences. As in the theatre: I can put on a show that you can find moving, but you must make the tears yourself.

I love it. I love it more than anything.

It is also, fucking terrifying.

I am going to tell you about three students. They are not actual people, but they are real. The first I will call Isaro.

I meet Isaro for the first time in the first week of my course and within a minute or two I start to feel a growing excitement in my chest. I am being challenged, I am looking foolish, uncertain, and it is fantastic. It is the euphoria of redundancy. I am not required here, she will be absolutely fine on her own. Isaro will learn at least 10x more today from her own questions than from anything I might say. My role is to help her find more daring and terrifying questions to pose to herself. I show her three different ways of doing the same thing. I show her something that makes no sense. I walk away excited to see what she does next — and to have her as a peer in a year or two.

The second I will call Emily. She talks quite easily, has known success in her life, is generally an interesting person. She asks strange questions, suggestive of confusion or misapplied models. She is distractible but determined. This journey is not always going to be easy for her, but given enough grit and time she will succeed. I spend quite a lot of time with her over the few months she is in my class, mostly requested and facilitated by her, much of which we spend on concrete practical matters like code quality and reorganising certain tricky concepts. People like her are the reason I have a job and why I am useful.

The third I will call Esme. I sit with her a lot in her first few weeks with me. She believes she is struggling. We set goals, she misses them. We try to do some basic tasks that she must have been able to do to get here, but she cannot do them in my presence. She becomes very frustrated, sad. This took me some time to get used to when I started teaching — how much affective labour there was to be done. I know from the first few interactions that something is wrong, something I have seen before but never want to believe. We try everything I've tried before, things that should work and things that shouldn't, new things, old things, things my colleagues suggest. Nothing works. She finds another life outside of my field, I hope it is for the best.

Who are these people? What are they made of? I read their notes, I ask them about their histories, I try to make sense of how it all fits together. Musicians, traders, parents, poker players, teachers themselves. They are all different and it is my task to help them all do the same thing. It is humbling to be trusted to teach all of these amazing people. It is humbling to fail, and even more so to see them succeed.

My colleague, our internal recruiter, is interviewing me to help him do his job better.

"Why do you work here?"

I think for a minute or two.

"It's not easy, but it means something. I never have to wonder why I come here every morning."

When I see the work myself and my colleagues are doing, and the totally divergent work of other teams like How to Design Programs, I start to feel like we in computing education have barely scratched the surface of what is possible. Sometimes I'm vain and ambitious enough to wonder what my place might be in the journey to come.

What do I think it ought to be? Am I okay to leave that big chasm lying there? What would a world with genuinely accessible software engineering education be like? Is it possible?

My team and I are agonising for the nth time over this question — how do you tell if someone has learned something... without negatively impacting their learning?

I think back to Esme and the tens of other students who have gone to pieces as I've sat quietly next to them, who have told me afterwards what a knock it was to their confidence to put their fingers on the keyboard and, at that moment, find nothing coming out. Most of them come back even stronger, but a few...

I think of the calm, friendly neutrality of the error messages that graced my screen as a developer, day-to-day, moment-to-moment informing me where and how I had made a mistake. Frustrating, true, but I have never seen anyone look half as defeated at a compiler error as they have at me.

There is something deep and wonderful about this problem. I never knew there would be problems like this, and that I would have the opportunity to think so hard about them.

I have a vision of myself in the darkness of the ocean, an anglerfish a hundred feet below me, tracing out a lazy path, just as mysterious, just as intriguing. I wonder what the zoologist who I helped learn to code would think of this ending.

Want to work on the hardest problem in computers? We at Makers are looking for our next coach.

Otherwise, do get in touch with me at @neoeno if you'd like to chat about learning and computers. I'd love to talk about it with you — it's how I learn!