Consider the Lobster
Creating software is often described using metaphors that tend to overlook that the process can be at times painful, messy, and frustrating. To put this right, I propose a new metaphor which explains where some of the pain comes from, and why it is often worth it.
A backstory
I’ve been writing a Vulkan renderer recently. Actually, it feels more like I wrote three renderers, one after another, plus half an engine, before I more or less got a handle on it.
And while this has been a good school so far, it was at times acutely painful: Twice, with great regret, I felt that I had to abandon an approach together with its codebase. I had pushed it so far that it was about to collapse under its own weight in complexity: adding anything new to it felt harder than shedding the project and starting again.
When writing software, my prime motivation is not necessarily to seek out pain. And so, naturally, I ended questioning myself: Was I doing it wrong? Was my approach to writing software fundamentally flawed? Am I the only one with these problems? Is it too late for me to abandon the field altogether and join the French Foreign Legion?
When faced with existential questions, I tend to rationalise and search for answers and solace in books; Surely, a technical discipline such as software engineering, after decades of reflection and practice must by now have a fool-proof, guaranteed-success method for dealing with complexity. If not that, then maybe at least it could provide a common language I could use at a twelve-stepper?
And so I began to amass a stalagmite of books.
The search for (metaphorical) perfection
To rest my research (and the structure beginning to tower over my bedside table) on sound foundations, I began with the heaviest book: Steve McConnell’s Code Complete, which pretty much opens with this observation:
"Metaphors help you understand the software-development process by relating it to other activities you already know about. Some metaphors are better than others."
To prove this point, McConnell criticises two apparently common metaphors, the “Software Farming” and the “Oyster Farming” metaphor, before he introduces his favourite model: Building software, he suggests, can best be described in the language of building construction: There, substantial time of architectural planning is followed by design, structural engineering and testing, until, at the very last step, code is actually written, based on detailed specification and documentation. Decision-making therefore progresses linearly from the more wide-ranging and abstract to the fine-grained and concrete.
My main concern with this metaphor is that it implies that software is ever finished, and built into a never-changing context. Software, as described in the Construction Metaphor, is something inanimate, ideally perfect, or in other words: dead and beatified .
Engineering is (probably) not what you think it is
Glenn Vanderburg articulates further reservations in his talk Real Software Engineering, where his main point is a critique of “[…] the idealised view of engineering that academic software engineering started out in pursuit of.”
He then presents an alternative definition of engineering:
"There is no kind of engineering where all we have to do is put something in one end and turn the crank. Engineering is inherently a creative, exploratory process marked by false starts and dead ends, and failures, and learning processes, no mattter what discipline you see."
And he then quotes Billy Vaughn Koen who wrote:
"The Engineering Method is the use of heuristics to cause the best change in a poorly understood or uncertain situation within the available resources"
No wonder engineering is not well described by metaphorical oysters forming layered accretions around a problem, or idealised top-down waterfalls scheming to build a house. What now?
Consider the Lobster
Perhaps let me introduce a new metaphor. I call it The Lobster Model. This has nothing to do with the lesser orange site.
While admittedly also being waterborne, the metaphor used in The Lobster Model additionally acknowledges that the process of writing software is a creative endeavour, and as such can be messy, and not without danger.
Lobsters carry their skeleton on the outside. This is great for structural support, and even better for defence. Lobsters are therefore never naked, one could think. How nice for them, they must only ever blush when cooked. But that’s not the whole truth.
During their (often surprisingly long) natural life, lobsters will want to expand just as any well-funded concern, or organism with an energy surplus. But inevitably they too will hit their limits, and quite literally so. A lobster’s shell is rigid and will not grow with them. Therefore, they must prepare for a process called “molting”, where they get rid of their old shell. This is wonderfully described in The Secret Life of Lobsters:
"Before molting the animal must diet away half the mass in its claws or risk getting stuck in its old clothes. Worse, because a lobster is an invertebrate, every anatomic feature that is rigid is part of the exoskeleton, including the teeth inside the stomach that grind food. The lobster must rip out the lining of its throat, stomach, and anus before it is free of the old shell. Some die trying."
Sounds pretty nasty to me. But it resonates with past moments where I had to rip out entire code bases, so that I could make another attempt to grow a project to the next level of complexity.
For a lobster, the molting period is when they are at their most vulnerable, and defenceless.
For software the greatest peril similarly is inbetween implementations, where the motivation driving its development exists only as naked ideas and untested learnings, which, on top of it all, may already have abandoned a previous codebase. Such a moment is of course full of promise, too. Software, to grow, sometimes needs to molt.
In the wild
Surely I’m not the only one who has seen forces of creation behave like this. In the wild, we may have seen the Lobster Model in action at medium scale when OpenGL became Vulkan. Or at large scale, when Mac OS 9 transitioned to OS X. Or at massive scale, whenever Madonna releases a new album.
How wasteful, some might say, to leave a codebase behind, even if it was outgrown. How can this ever be sustainable? Nature, or more specifically lobsters, might have an answer for that. What do they do with their abandoned shell?
They eat it. Bon appetit.
References
The following books and videos were referenced, or contributed ideas to this post:
[1] McConnell, Steve. 2004. Code Complete. 2nd Edition. Microsoft Press. See also: wiki
[2] Corson, Trevor. 2007. The Secret Life of Lobsters. HarperCollins e-books. See also: author’s website
[3] Vanderburg, Glenn. 2018. Software Art Thou: Glenn Vanderburg - Real Software Engineering [video file]. retrieved from youtube
[4] Kuhn, Thomas. 1962/2012. The Structure of Scientific Revolutions: 50th Anniversary Ed. The University of Chicago Press. See also: wiki
[5] Ousterhout, John. 2018. A Philosophy of Software Design. Yaknyam Press. See also: book review, author’s talk: youtube
Tagged:
RSS:
Find out first about new posts by subscribing to the RSS Feed