Programs as spaces

Paul Graham's essay Holding a Program in One’s Head describes how a good programmer immersed in their code holds it in their mind: "[Mathematicians] try to understand a problem space well enough that they can walk around it the way you can walk around the memory of the house you grew up in. At its best programming is the same. You hold the whole program in your head, and you can manipulate it at will."

The article’s mainly concerned with the organizational consequences of needing to "load the program into your head” in order to do good work. But I want to focus on the spatial metaphor. Thinking through a program by walking through the spaces in your head is an image I've heard other programmers use, and it reminds me of the memory methods described by Frances Yates in The Art of Memory. (While Graham does make reference to writing and reading, I don't think this is aural memory; his references to visualization seem more fundamental.)

I wonder about the kind of cognitive access a programmer has to their program once it’s loaded. Descriptions of walking through a building imply that moment-by-moment the programmer is only dealing with a subset of the problem, although the whole thing is readily available in long-term memory. He’s thinking about the contents of a particular room and how it connects with the other rooms, not conceptualizing the entire house and all its relationships at the same instant. I imagine this is necessarily the case, since short-term memory is limited. If true, this imposes limitation on the topology of the program, since the connections between different parts are localized and factorizable – when you walk out of the bedroom you don’t immediately find yourself in the foyer. Consequently, problems that can’t be broken down (or haven’t been broken down) into pieces with local interactions of sufficiently limited scope to be contained in short term memory will not be soluble.

Graham also has a great insight on what makes programming special: "One of the defining qualities of organizations since there have been such a thing is to treat individuals as interchangeable parts. This works well for more parallelizable tasks, like fighting wars. For most of history a well-drilled army of professional soldiers could be counted on to beat an army of individual warriors, no matter how valorous. But having ideas is not very parallelizable. And that's what programs are: ideas." Not only are programming tasks not like fighting wars as Graham imagines them; they're not like manufacturing widgets either. The non-parallelizability of ideas implies their interconnections, and here we have the fundamental tension: ideas may be highly interlaced by their nature, but the nature of the brain limits the degree to which we can cope with their complexity.