Cognitive debt in AI coding

Parachuting into an unfamiliar codebase, where the original author is long gone, is an experience that will be familiar to a lot of developers.

People writing with AI assistants have been encountering this in a new form.

Simon Willison wrote a brief post about it recently, concluding:

I no longer have a firm mental model of what they can do and how they work, which means each additional feature becomes harder to reason about, eventually leading me to lose the ability to make confident decisions about where to go next.

— Simon Willisonsimonwillison.net
Interestingly, this has been my experience in the large codebases of every company I've ever joined. Even with reasonably good documentation, it can be difficult to get to grips with any sufficiently large codebase – say, over a million lines.

This can also be true of smaller systems. If you're new to programming, you will be learning what works best as you go, and so your codebase will change as you learn. Even an experienced developer can make plenty of design mistakes in a domain they are unfamiliar with.

Working with large, successful systems are the circumstances under which code review skills are built. Something true in one part of the system may not hold in another. Experienced developers have their own methods to build understanding, like creating or adding to test suites, extracting microservices, and cultivating the patience, persistence, and humility to read the code.

What's new, though, is experienced developers needing these skills in their own personal projects.

Systems like Claude Code now have an integrated MEMORY.md that they can use for projects in order to persist comprehension across sessions. I am using it and CLAUDE.md religiously in order to build persistent knowledge of the system into each project – but it's early days.

Coding assistants are taking people from idea to working system much faster than before. They are reaching the points at which design and testing become valuable much more rapidly. Not only that, but it's also possible to turn many ideas that might have remained pleasant daydreams into reality, and dreams brought into reality can sometimes turn out to be nonsense.

This may even tie into at least some of the sense of burnout that people have been reporting. It can be comforting and intrinsically rewarding to tinker away on a codebase, but if you're building a business, or at least software for other people, code is just part of it.

It can be quite disheartening to realise that customers won't just appear out of thin air. Building for yourself is fun, but building for nobody is pointless. Claude Code isn't talking to your customers. It's not being thoughtful about the problem. It doesn't have a strong sense of aesthetics.

It doesn't know that what you don't build can be as important as what you do build.

All sorts of people, developers included, are finding out that coding is not software engineering, and that software projects are not startups. This is not new information, but it can take a long time for it to sink in. If people are able to get to that lesson faster, then it's a good thing.


There is currently no comments system. If you'd like to share an opinion either with me or about this post, please feel free to do so with me either via email (ross@duggan.ie) on Mastodon (@duggan@mastodon.ie) or even on Hacker News.