Here are a few thoughts on institutional memory from a talk by Admiral Hyman G. Rickover.
When important decisions are not documented, one becomes dependent on individual memory, which is quickly lost as people leave or move to other jobs. In my work, it is important to be able to go back a number of years to determine the facts that were considered in arriving at a decision. This makes it easier to resolve new problems by putting them into proper perspective. It also minimizes the risk of repeating past mistakes.
I’ve never seen this done effectively. I’ve been part of business and non-profit organizations that kept good records, but I don’t recall anyone ever looking back through the records to review why a decision was made.
This is especially a challenge in software development where the myth of progress runs so strong. Newer is always better. The people before us were stupid, so why should we go back and read what they thought? Or maybe they weren’t stupid, but they were working with version 10.5 of FooBar. Now we’re up to version 11.2 and so everything has changed.
14 thoughts on “Why did we do this?”
Have you seen the Architecture Decision Record proposed by Michael Nygard? He’s identified the same problem and his solution appeals to me a lot. http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions.html
Github has a good solution for this, where they use pull request discussions to document any proposed change to the system, including all the architectural discussions involved, etc.
See the excellent video of Zach Holman’s talk: . Worth watching and pondering.
I’ve been trying to encourage our team to include a ‘Rationale ‘ field in our wireframe documentation. i.e. “I don’t like this solution but we didn’t have money for a Solution ABC. Next release consider Solution XYZ.” or “This is compromise between what the propositions team wanted and limitations in the data provided by System XYZ”.
It’s hard work though and the first bit of documentation to go AWOL when people are up against short deadlines.
In the context of university governance documentation of past decisions is occasionally useful. Offhand I can think of a couple instances of “how should we handle this situation? well, it looks like we did it this way when a similar issue arose 10+ years ago”, and several instances of “hey, when you made this decision you promised us that this mess we’re currently in would never end up happening”.
I understand what you mean. But is this the difference between changing a line of code and changing the seals on a nuclear reactor? Certainly I which standards developers did a better job of recording the rationale behind certain design decisions made.
This reminds me a lot of legal precedent. That’s an area where past decisions are documented along with the reasoning that gave rise to them.
I’m not sure I would say that we do it well, but in medical device design, we are obligated to keep extensive records, and I have found them very useful in device design, test method creation, and drafting regulatory submissions. You definitely don’t want to contradict your last submission!
I have a great example of this.
At one company, I had a super-urgent request from sales to fix a particular bug in a software component I was writing a replacement for at the time. It was an obvious fix, but the buggy behaviour was non-obvious and looked like it had to have been done deliberately. I had to fight with the head of development to get time to investigate what had happened. When I looked through the version control history, I discovered that the buggy behaviour had been added in response to an urgent request from sales about a year previously, and they just hadn’t anticipated that the new behaviour might not always be desirable.
Our institute doubles as consultant in the field of water management and hydraulic engineering. For certain markets, we keep logs of what’s happening with business leads: who has spoken to whom and what was discussed and decided? The main reason is that some leads may take years to mature, but in the process we’re building a nice little archive of decisions that may be consulted if and when appropriate.
This is EXACTLY where literate programming comes into its own.
If you have a literate program then you should be able to give it to
someone new, have them go away for a couple weeks (paid trip to
hawaii?), and when they return they know all about the program.
They know why it was written, they know the key design decisions,
and they know how to maintain and modify it.
Literate programming capture the most important information of
a program. We currently throw away all of this information. Done
right, you should be able to recreate the program from the text,
where the program expresses the same ideas even if the program
text (or language) is not the same as the original.
For an example of a literate program in HTML, see
I take it onto myself to write internal documents when I reach milestones in my projects.
– I can’t be told that I’m hiding anything (cover your horse)
– If the project must to be transferred to someone else for resource balancing, I don’t feel any rush to prepare anything (prepare for the future by performing work now which is harder in the future due to forgetting)
– I appear to be the most literate and contributory player on any team (differentiation)
– My contributions are documented (quantification of value)
– I have internal papers to record on my CV (prepare for the future by demonstrating past success)
Before I started tutoring, I was in data processing. We were in the same building with the programmers and equipment personnel. We logged EVERYTHING, no matter how miniscule it may have seemed. Each group had a hand off at every shift and copies of each groups’ logs were made for every other group and the second line managers and THEIR bosses. If ever a problem was discovered, the fix became part of the SOP manual, Standard Operating Procedure, which of course we all heard about as soon as possible.
Repeating past mistakes was not a part of our culture and we almost always understood why we did what we did. Sharing of information was absolutely key. It could work in other places as long as everyone has the same goals.
I’ve also been in a situation where a “bug” was a previously requested feature. In my case, it was two different groups in sales wanting different behavior.
My group does a decent job of documenting the code we write, but I’m not sure about noncoding related tasks, such as server configurations. I have documentation from when I was setting up a new server, but the next person to handle it promptly forgot about my notes, and I have no idea where/if current notes are stored.
It appears that there are at least some niches where design decisions are well-documented including the discussions and tradeoffs that lead to them. I remember that Eric Lippert (who works on the C# compiler) digs up those documents from time to time to anwer questions why C# is the way it is. Quite enlightening.