The Mind Is A Terrible Thing

Or, How one programmer works, after fifty years in the trenches

Below I list the things I have learned in the fifty years since I keypunched my first line of Fortran into a Hollerith card. While the maxims are phrased facetiously, and almost all are too abstract to apply directly, I promise you this: I consider them all the time while coding, and they drive my choices while I am coding.

The tl;dr good news?

Programming is easy if we do not think too much.

The trick is first to code a lot so we build the coding instinct, then learn to let go and listen to that instinct. I spent twenty years on the first half of that sentence, and I remember clearly the one year in the late nineties spent on the second half.

“Be the ball, Danny. Just be the ball.” — Ty, Caddy Shack

I was working on a clinical drug trial management system, and for a solid year I noticed time and again that some whacky idea, entertained only briefly, is where I ended up. And the whacky idea was downright perfect for the problem.

“Gee,” I would think. “You were right the first time.”

And for a year my next thought was, “And this keeps happening.” Then one day I just brushed off the sense of whackyness and nervously Just Coded some goofy idea.

Why am I writing this if it did not turn out well?

Quite liberating to just skip along footloose and carefree coding up the first algorithm that comes to mind. It works because they come from the problem.

How we derive algorithms from problems

We must listen to the problem while we are coding it up. The problem will talk to us with bugs, and with functionality that refuses to emerge.

We just have to listen. Most programmers do not listen, because they do not realize the problem is talking to them. They think they have to hack in another twenty or hundred lines of code to get around a bug or make some recalcitrant feature work. Maybe. Maybe not.

We need to turn off our minds. Minds make trouble.

Below we talk about seeing and listening, instead of reflexively hacking out more code when difficulties arise. The style is scattershot, bullet points without the bullets. Pardon the speed bumps.

It took those twenty years of coding before I turned my mind off and let what follows direct my coding. Hopefully this compendium of lessons learned hard will accelerate your path to Mushin, or mind without mind.

Programming is easy if we do not think too much.

I know this all sounds fanciful and fantastic. It is not. It is how this programmer deals with every line of code, every day.

Mrs. Willens

Mrs. Willens had stumped our French class with a word translation. After a long silence one of us finally remembered the correct word, started to say it, and the whole class joined in to finish the word in harmony.

“We are much better at recognizing the right word,” Mrs. Willens noted with a smile, “than we are at producing it.”

This ^^^.

Do not create. Copy.

Do not create a program. Instead, see the domain problem clearly in terms of the domain, ignoring coding, then copy what you see into code.

Or, think of it as literally translating a real world problem into code.

The problem determines the solution. This is why good programmers, given the same problem, will come up with the same solution. Corollary: my code should look like the domain problem.

Even Van Gogh took his easel out to copy a starry night. Guessing.

by archer10 (Dennis) licensed under CC BY-SA 2.0

Listen to the problem

Every problem encountered while coding is either a silly bug, trivially fixed, or a message to me from the abstract problem about my solution. I heed these.

I do not write great code. I admit it when I have written bad code.

The difference between master and disciple is how they react to bugs. The capable disciple makes the bug go away. The master asks how the bug arose, and if they should start over on their whole approach. The problem is speaking to us, and it is not happy with our design.

Maybe I have a chunk of code handling a flow of information, maintaining all the while a model shaped by this flow, and I am using a couple of flags to help me keep track of where we are. Then it breaks. Whassup…oh. Hmmm. Maybe a third flag will…and this is where I stop. Is this really a three flag problem I am solving? When I write out what I am doing in a note to Mom, does it sound this complicated? Cue Albert.

“Make everything as simple as possible, but no simpler.” — Albert Einstein

It is a great line, and says more than first jumped out at me: forget the three forces, every problem has an intrinsic quantity of simplicity, a value that exists prior to our translating them into code. We see this in the caveat but no simpler; if my solution is simpler than the problem, I better have either made some profound insight into the true nature of the problem, or — I missed something in my solution. Both happen, by the way.

I do not write great code. I know when I have written bad code. And then I change it. I converge on simple code, directed by the problem. If I listen.

Many smart programmers are so smart they do not even realize they have written bad code. When difficulties arise, they work around it by writing more code. Other programmers do realize their approach has gotten difficult, but they do not understand that they need to start over.

Start over.

Craft

If I craft individual lines of code well, and if my ERD is normalized, then I will be OK.

“Take care of the pence and the pounds will take care of themselves.” — English proverb

Three programmers were told to solve a problem. One was told to make it as fast as possible, another to use as little memory as possible, a third to make the code as clear as possible. Each programmer won on their assigned metric. Guess which one came in second in the unassigned categories.

Code as if the next developer is five years old. Eschew obfuscation. Clojurians love to flex and write obscure threading expressions using as few characters as possible. But now we cannot see what they wrote.

Craft is about doing all the little things right, because the little things can kill us. Rope drag is my favorite metaphor for what happens when we ignore craft. The Euler-Eytelwein formula will always stop us, right in the middle of a cliff. Software projects fail when rope drag is ignored.

The thing about rope drag is that no one carabiner placement locks the rope. Each unfortunate placement by itself adds just a little drag. What locks the rope is someone not understanding that unfortunate placements add up. Place each carabiner well and the rope will tend itself.

Design

Design is a knife-edge ridge. Stay on the edge and we have an easy stroll. One step to either side and we may never come back.

Consider. Our design is what emerged as we copied the problem into data structures and code. We are coding away happily, implementing that design. Functionality rises easily from our problem-shaped design. Then we discover something we had not anticipated.

Our design has a hole, maybe even a flaw. We take the next step very carefully. More I cannot say, in the abstract. Just know that one step to either side of the most elegant design path will quickly unravel.

So far we have talked about seeing the domain problem clearly, and about listening to the problem. Now we have to be able to feel when our local response to an unanticipated problem takes us off the knife-edge into oblivion. Feel? This may help: if we find ourselves writing ugly code, stop. Wrong way. Turn back. Speaking of refactoring…

Listen to the code

• When refactoring, I expect to delete more code than I write. If I start writing more code to make a refactoring work, I abandon the new approach.

git reset — hard

• A good refactoring will let you delete apologetic comments, because the refactoring will have eliminated what we had to explain.
• Some of the worst code is produced by true (no snark!) geniuses. They do not even know when they have written bad code, because they just muscle through every difficulty. No matter how smart we are, do not do that. Acknowledge the speed bumps and eliminate them, even if it means a whole new approach.
• I try to solve “the real problem”. Often we respond to a coding difficulty with new code aimed at a mere consequence of the real problem. I look at a difficulty, explain it to myself like I am five, maybe even free write the problem, then write code that addresses the real problem, or more likely, refactor the existing code to make the problem go away.

Debugging

• The first thing I do when presented with a bug is work on something else. I work on making the bug easily and conveniently reproducible. A single command or key chord should run something that fails on the bug. Next, I work on adding a dozen carefully chosen print statements. Now I am ready to debug.
• When debugging like that, I keep in mind that, sadly, I am now debugging my debugging as well as the original code. We always create false alarms and mirage successes while debugging, so we always have to look over our shoulders as we go.
• One good way to screw up is leave debugging code behind. I once had a long-running program process only every third record. Three was also the packing factor on a key step. I got so used to seeing numbers off by a factor of three, I forgot the debugging code was in there. And shipped. Now I put the comment “HHACK” wherever I am hack-hacking, and usually remember to look for those before shipping.
• When presented with a software failure manifesting itself in several ways, we should work first on what seems like the first problem, the one possibly and likely causing the others. A classic, illustrative war story.

A bug means our problem dislikes our solution.

• Do not try to figure out bugs. Print a zillion diagnostics and recognize bugs. (see Mrs. Willens).
• Do not debug interactively, use print statements to debug. Interactive is slow, and relies on our thinking to ask the right things. Just dump what is going on, you will recognize the issue.
• When tired, I stop working on a hard problem. When tired, we try to think through a problem because we are too weary to bang in ten more print statements. Take a nap, add the ten print statements, then we simply see the problem in some output where you never expected it.
• How to debug? Watch a few episodes of House. Notice how he thinks and works. He does not trust any information. The patients always lie. House sends his team to the patient’s house to see the conditions. Buddha and physicists teach that everything is connected. House looks at everything, and if a test says something he does not believe, he re-runs the test.
• “The most dangerous gun is an unloaded gun.” Discuss. When stuck, we need to learn to think as if what we know is not true. Not easy. Vulcan Mind-Meld territory.
• Similarly, beware of “we already tried that”. We may have fixed it and not known it because of a second bug, and put the first bug back in.
• On the other hand, when stuck we might make an exploratory change that introduces a new problem, but we do not notice because it is shadowed by the original bug. Now when we find the real bug and fix it…the code still will not work. Clean up failed experiments as you go, unless you are sure you fixed a problem with that experiement. This one is a judgment call. But there can be two faults at once in new code. This applies to the interior lights of a car, as well.
• Is some algorithm stubbornly failing unreasonably? If it is short enough, hide the code and rewrite from scratch. You might not make the same silly, invisible mistake again.
• New bug? We know the drill: did it ever work? Maybe this is new code, or code that has never been tested. If it did ever work, when was the last time? If recently, what has changed recently?

Maintaining Other People’s Code

• First, we kill all the comments. Writing is hard for anyone. For engineers, hopeless. Me, I use syntax highlighting and assign very light gray to comments to make them nearly invisible.
• Next, correct the names. Variables and functions, that is. See Confucius on the rectification of names. Amazing real life example from my days in tall buildings: AA, BB, CC, and DD.

Easy begets hard.

Back in the day, imperative languages had GOTO. People found it easy to program with GOTO, until they got to about 500 lines of code. After that, they were in serious, project-risking trouble.

Today, GOTO is unthinkable. Back then, developers clung to GOTO, because it seemed easier than figuring out a structured, functional solution. Just throw in another GOTO! BAM! We’re done!

In my first programming job I joined a team working on a module with just 2000 lines of COBOL. And about fifty critical GOTOs. After two years and three senior analysts, it still did not work. I myself spent several hours “pairing” with two project veterans as they flipped the print-out back and forth trying to guess what GOTO path was breaking.

Did someone say “easy”?

With the AVPs encouragement I rewrote the thing following structured programming principles in two months. Turned out the real problem, a CRUD app, was seriously easy.

I was assigned three developers to roll out the full implementation, one of whom thought I had gotten carried away with eliminating GOTOs. One day she was stuck, and asked for a consult. I was utterly stumped, one problem being this was 1981 and a compile to add print statements took an hour. No joke.

After I was out of clever debugging ideas, we bit the bullet and smothered the twenty-line bug zone with print statements.

No output appeared.

Back then our monitors were 24x80 on a good day, so we only saw so much code, but we were dead in the water, so I scroll-wandered mindlessly up screen after screen…to the GOTO 2099-END statement that was bypassing all the code we had been analyzing for three hours.

During a stint at a small shop recently, I asked why the PostgreSQL database schema gave third normal form so little respect. A senior developer was honest; he said everyone found it easier without honoring 3NF. It also meant I could not understand their schema, because they did not follow one.

A year later, the team was gone, replaced by an outsourced effort.

Politics

• The Golden Rule of IT: The only thing that matters is quality code delivered quickly. There are no unhappy winning locker rooms. Winning translates to users having good, fast software that can change easily with their business requirements.
• When things are going poorly, users and IT will start to fight. The company loses. Now see the “Golden Rule”.