People best understand computer programs in a different order than compilers do. This is a key idea of literate programming, and one that distinguishes literate programs from heavily commented programs.
Traditional source code, no matter how heavily commented, is presented in the order dictated by the compiler. The computer is the primary audience. Literate programming is more humanistic in the sense that the primary audience is a human. The computer has to go to extra effort to arrange the code for its needs. As Donald Knuth describes it in his book on literate programming,
The practitioner of literate programming … strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that nicely reinforce each other. [emphasis added]
There are two steps in processing literate programs: weaving and tangling. You take files containing prose and code, and weave them into documentation and tangle them into source code. Tools like Sweave and Pweave focus on the weave process, as their names imply. The weave side of literate programming has gotten the most attention.
A half-hearted approach to literate programming doesn’t require much of a tangle process. A well-commented program has no tangle step at all. A *weave document that follows the order of the source code has a trivial tangle step: save the code to its own file, manually or automatically, but don’t rearrange it. But a full-fledged literate program may make the tangle program work harder, rearranging code fragments from human-friendly to compiler-friendly order.
Careful explanation vs. unit tests
The most obvious feature of literate programming is that it requires careful explanation. Here’s more from the paragraph I quoted above, filling in the part I left out.
The practitioner of literate programming can be regarded as an essayist, whose main concern is with explanation and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible …
The discipline of explaining every piece of code leads to better code. It serves a similar purpose to writing unit tests. I saw somewhere—I can’t remember where now— that Knuth hates unit testing and sees it as redundant effort. Presumably this is because unit testing and literate programming overlap. Unit tests are a kind of commentary on code, explaining how it used, exploring its limitations, etc.
Knuth understands that literate programming doesn’t replace the need for testing, just unit testing. He explained somewhere—again I forget where—that he would test TeX by spending days at a time trying fiendishly to break it.
My misunderstanding and experience
When I read Knuth’s book, I could see the value of carefully explaining code. What I didn’t appreciate was the value of presenting code in a different order than the order of source code.
I’m working on a project now where a few lines of code may require a few paragraphs of explanation. That’s what got me thinking about literate programming. My intention was to write my documentation in the same order as the code. It took a while to realize I had stumbled on an ideal application of literate programming: a complicated algorithm that needs to be explained carefully, both in order to hand over to the client and to reduce the chances of errors. The best order to understand this algorithm is definitely not top-down going through the code.
Why literate programming has not taken off
I think I understand better now why literate programming hasn’t gained much of an audience. I used to think that it was because developers hate writing prose. That’s part of it. Most programmers I’ve worked with would much rather write a hundred lines of unit tests than write one complete sentence.
But that’s not the whole story. There are quite a few programmers who are willing and able to write prose. Why don’t more of them use literate programming?
I think part of the reason is that having a non-trivial tangle process is a barrier to adoption. A programmer can decide to start writing more extensive comments, gradually working up to essay-like explanations. But it’s one thing to say “I’m going to heavily comment my C code” and quite another to say “I’m not going to write C per se any more. I’m going to write CWEB files that compile to C.” Even if a programmer wants to write CWEB in secret, just checking in the tangled C files, other programmers will edit these C files and the changes won’t be reflected in the CWEB source. Also, the output of tangle is less readable than ordinary code. The programmer secretly using CWEB as a preprocessor would appear to be writing undocumented code.
Tricky code benefits from a literate presentation, but routine code does not benefit so much. You either have to have two ways of writing code—straight source for routine code and literate programs for the tricky parts—or impose the overhead of literate programming everywhere. Most code is mundane and repetitive, challenging because of its volume rather than its cleverness. Knuth doesn’t write this kind of code. He only writes code that benefits from a literate presentation.
To write a good literate program, not only do you need to be able to program, and need to be willing and able to write good prose, on top of that you need to have a good sense for story telling, arranging the code for the benefit of other readers. If this is done poorly, the result is harder to understand than traditional programs.
I may use literate programming more now that I’m starting to understand it, at least for my own benefit and hopefully for the benefit of clients. I usually deliver algorithms or libraries, not large applications, and so it wouldn’t be too much work to create two versions of my results. I could create a literate program, then weave a report, and manually edit the tangled code into a traditional form convenient for the client.
40 thoughts on “Literate programming: presenting code in human order”
I think the problem with Literate Programming is that assumes there is a single best order of presentation of the explanation. I agree that the order imposed by the compiler is not always best, but different readers have different purposes. You don’t read documentation like a novel, cover to cover. You read the parts that you need for the task(s) you want to do now. What would be ideal is a tool to help construct such paths for each reader, just-in-time; not a tool that makes the author choose a single pth for all readers.
I agree. I think most literate programmers are lone rangers, like Knuth. They may have a near-optimal presentation order, the order that best fits how they think of their work. Or maybe the order they think is optimal for presentation to an audience. I’m more in the latter camp, handing code over to a client.
Leo (http://leoeditor.com/) has a feature to clone nodes (chunks) and re-org them into different trees so that the same literate program can have multiple presentation orders. I’ve never seen this in any other LP tool so far.
My experience matches the description made by Peter Norvig. Indeed, I saw the same limitation with _my_ projects: literate prose and structure tends to get in the way when you are trying to, say, add a new feature or fix a bug.
I still think it’s great tool to think clearly, especially when you have somethings like Emacs Org + Babel at your disposal, but LP it’s not a great fit for regular projects.
I’ve started embedding YAML in my comments. Initially, the purpose was to store requirements & traceability information alongside the source code in a machine-readable format — but I’m thinking that it could also contain documentation metadata — such as information required to weave the documentation in various orders. The source code is marginally less “literate” — but we gain additional control over how documentation can be rendered. It might be fun to try an experiment along the lines suggested by Peter Norvig above.
I read Knuth’s book some years ago, and have also over here “TeX: The Program”, which is a masterpiece.
I flirted with literate programming by then, but had ot give up. My head was going to explode trying to remember the relations between chunks, figuring out what needs to be modified to refactor something, variable scopes, …. it stressed me.
One of the problems I see is that you are using flat files to represent a tree. I used Emacs noweb-outline.el and Leo to no avail. You are still writing a tree structure, requires a lot of mental overhead.
How on earth could Knuth figure all this out, write the tools, and write TeX and METAFONT? What a brain man.
The source on Donald Knuth’s opinion on unit tests:
I went in for literate programming (using CWEB) in a pretty big way circa ’96 or so, and gave up on it completely by late ’98.
It was absolutely terrific for documenting / commenting on complex mathematical operations in code. TeX does math right, and it really helped make those bits clear. But even in the fairly math-heavy work I do, that represents less than 10% of the code.
On the other hand, it’s pretty much just a nuisance for the sort of boiler-plate code which is fairly mandatory in C++ programming. Looking at my implementation of a NURBS curve class, for instance, I see fourteen member functions which are shortish boilerplate; so simple that expecting to explain them literately is just asking you to come up with ways of blathering about nothing. Maybe six functions where a better explanation would really be useful. And the really tricky math bits are all abstracted away somewhere else so they can also be used for the NURBS surface class. :) (I also found two functions that were obsolete, and I have just deleted them!)
The other thing is that my code tends to use a lot of autogenerated C++ code. I guess my code to do the autogeneration could be done literately, though since it’s Perl it would probably need a different tool than CWEB. However, it’s not at all clear to me that I wouldn’t get more benefit from rewriting the code in nice clean Perl 6. (The big problem with the code as it exists is that the embedded C++ heredocs have to be outdented, totally ruining the visual flow of the Perl code. When I first heard about Perl 6, being able to indent heredocs properly was the feature I was most excited about…)
To Peter Norvig: Of all the possible alternative orders of presentation, I think there will be just a few that will be helpful (the order of access is a different matter.) In non-code documentation, we have variations on two themes: the tutorial and the reference, and the reference is not an explanation; it assumes a certain level of understanding.
When it comes to making an explanation, there are some fundamental rules, which are mostly about not introducing ideas until you have reached a point where they make sense. A large subject may have many topics that can be treated as subjects in their own right, but they generally fall into a fairly strict hierarchy of preconditions. There is often a core idea (which may be simpler than reality) that can be elaborated upon once it is understood.
While we have the technology to create non-linear narratives, a linear approach seems to work best, most of the time, when the goal is to understand something.
Actually Literate Programming did take off and failed! Many authors tried it, but, and this is my hypothesis, the readers didn’t like it! I certainly didn’t! I’ve had the misfortune to read some books written in that style and I hated them. Certainly I didn’t hate those books because of their content, which was very good. I just hated the presentation, that is exactly the literate programming style. I think that one of the main drawbacks of this style is the fact that the code is intermingled with the text and that makes the code more confusing. The code is never complete and you always have to jump back and forward to see the other parts of the algorithm and this just makes it harder to understand how the algorithm actually works.
Here’s my approach to literate programming that a) doesn’t require writing a lot of prose, b) integrates well with tests, c) relies heavily on controlling order for presentation, but d) explicitly tries to support non-linear reading: http://akkartik.name/post/wart-layers
The key property is that you can build and test layer 1 in isolation, layers 1+2, 1+2+3, and so on. This allows you to start reading a much simpler version than what currently runs in production, understand the core skeleton and then gradually add features back in at your own pace.
After writing 20kLoC with this (https://github.com/akkartik/mu), I find that the organization into layers is so powerful that I don’t usually need to write comments at greater length. That keeps the reader’s experience non-linear. The emphasis isn’t on end-to-end readability but on non-linear workflows of *tactile interactivity*:
a) Running simpler or more complex versions than the current one, adding or removing features.
b) Getting useful line numbers in debuggers.
c) Jumping down to look at the tangled sources (which remain readable, unlike with Knuth’s original tools)
The Axiom computer algebra system has been using literate
programming for years. See
Literate programming is a LONG TERM investment.
If your company depends on a program you need LP. You will have people (probably the whole team eventually) leave the company, taking with them the deep understanding of the code.
I started using literate programming when I got my old code back after 10 years. I knew what the code did. I could tell you what bytes the compiler would generate. I knew HOW it worked. I just didn’t remember WHY I wrote it. Literate programming is about capturing the WHY and explaining it to humans.
If you want to see a real literate program done well, buy the book “Physically Based Rendering” (it won an Academy Award).
Once you’ve see the book, imagine hiring a new person to work on that program. You could give them the book, send them on a 2 week expense-paid Hawaii vacation (I call it the Hawaii test, and, yes, REAL programmers would read the book in Hawaii). When they return they could maintain, modify, and extend the program. Code they add to the repo would be accompanied by a paragraph and go into a new version of the book. Imagine how much easier it would be to hire, how much less screaming there would be for a ground-up rewrite, and how well people could understand the program.
Literate programs aren’t just for you. They aren’t just for now. They are there to capture the “WHY” for the poor souls who have to maintain your magic code piles.
If your code doesn’t look like the Physically Based Rendering book and your company depends on that code to live, you’re doing it wrong. Hire an English major as Editor-in-Chief. Insist at the code review that the paragraphs explaining the code make sense. Code must be checked in by the Editor-in-Chief who maintains the “Golden copy” of the resulting book(s).
Your criticism of literate programming comes from your own point of view as the author. Authors hate their own books. But you’re NOT the audience. Future programmers are your audience.
@Tim Daily – When developing safety critical systems that will be fielded for decades after the original development team has disbanded — an institutional focus on documentation and traceability is rather essential. (Anything else would be irresponsible). The issue (for me) is that these things are too hard to achieve in practice — interrupting precious concentration and destroying flow. My objective is to find ways that we can improve documentation and traceability — even if only by a little bit — at the absolute bare minimum of cost in terms of disruption and distraction. Hence my desire to put the “user interface” for the documentation and traceability infrastructure as close to the code as possible — minimizing the number and severity of context switches.
@payne We’re building “safety critical systems” that will be “fielded for decades”? Let’s assume something like a nuclear reactor control program as an example with a plan to run for 50 years. This code will almost certainly need to be maintained “after the original development team has disbanded”.
We want to achieve the goal of software that can be maintained, modified, and extended safely, implying that the developer(s) deeply understand the implications of their changes. Yet some of the original team members are dead (as has happened with Axiom) and the software was written for “user interface” devices that are no longer manufactured (as has happened with the TRACON air traffic control hardware).
To maintain a large code base we need to have a good understanding of it, a “coherent whole”. We need to know that a change is going to affect distance parts (e.g. the database). Which implies that we have a way to understand parts we don’t maintain.
In the 1970s I worked on a machine with 8K of memory. If my editor file was 1 byte over 4k it overwrote the operating system and I lost everything. So we invented “.h” files, subdirectories with “meaningful names” like “doc” and built these “piles of sand” code repositories. Oddly enough we STILL write code like it was 1970 despite 16G of “core memory”!
The “pile of sand” (POS) development model makes it impossible to get a wide and deep overview, especially on a large project. Axiom has 1.2 million lines of code. A POS development model, after the original developers leave, is a disaster waiting to happen.
Suppose, instead, we introduce a literate programming model incrementally. Designate an Editor-in-Chief (not a developer, an English major). Require that EVERY code review has a 2-paragraph (WHY, not HOW) natural language explanation that ALSO gets reviewed, AND approved by the Editor, AND checked in, AND maintained.
The Editor is responsible for arranging ALL the code into book(s), indexing, cross-referencing, creating sections and chapters, working with the architect to write overviews, and all the other Editor-like tasks. The book is “published” at every system build and failures ARE treated as code failures.
Developers still write code, even in POS style if you must. They just have to explain WHY they wrote it (e.g. “We need to keep core temps in limit so we don’t overheat cause that would be bad.”). There are no context switches and nothing changes except that their words ALSO get code-reviewed. Don’t allow check in of good code with bad words.
After a while, introduce the idea that the actual, executable code is AUTOMATICALLY extracted from the BOOK so that the code you read is the code you run.
In the long term you have a book that you can use for the Hawaii test. The new developer will understand that this code will alarm (throw some random exception to the alarm handler with some magic number) and know what the alarm and magic number mean.
Literate programming is not difficult. The tools are there. We just have to stop coding in POS style based on 1970s hardware.
Changing developer mindset is hard so sneak the change in one paragraph and one code review at a time.
New developers, writing code to throw a safety exception, can turn to chapter 4, section 4.3, and know what each magic number MEANS, what it will DO, and WHY it will do it. Methinks that might make the nuclear power plant a bit safer.
Think LONG TERM and have pity on the next guy.
Code is generally hierarchical, so if you are looking at it as if on a print-out, there is an immediate impedance mismatch (your use of top-down above is in the latter sense.) Any half-decent IDE should make it easy to navigate the hierarchy, and in general, I think this is the sort of problem that is best addressed by tools rather than language features (Alan Kay suggested that languages should come with tools (beyond a compiler) to the people developing Ada.)
One thing that I have not seen, but which may well exist in some IDE, is the ability to bring up a short description of a function inline at its point of use (assuming that description has been written), instead of having to jump to the function and back again to see if it has useful comments. IDEs could have extensive support for hyperlinking between documentation and code, and if they do not, it is probably because of the typical programmer’s abhorrence of writing expository text – without the will, having a way does not help.
“Code is generally hierarchical”… until we decided to add examples to comments… which affected the browser layout… and decided that we could use the examples as unit tests… which affected the compiler and the build system… and affected the interpreter to show examples on request… and affected the database that holds comments. The code may be hierarchical but the product is a gestalt.
I’m something of a curmudgeon about build tools. The IDE, like the Makefile, is a “programmers response” to complexity. I’m deep in a POS and I need to find a function (as in, check the Index in the book). I need to find out who calls it (as in, check the cross reference in the book). We understand every book and know where to look for answers. That’s not true of POS code.
“IDEs could have extensive support for hyperlinking between documentation and code”… but “it is probably because of the typical programmer’s abhorrence of writing expository text”.
Two comments. First, hyperlinking in an electronic book is trivial but there is less need because the “documentation” (I hate that word, it should be “explanation” or “the-why”) and the code are on the same page.
Second, “writing expository text” is a key difference between a professional and an amateur. If I commission an office building I expect to see blueprints. If I build a shed I fling nails at wood or points at user stories. (See https://www.quora.com/In-a-nutshell-why-do-a-lot-of-developers-dislike-Agile/answer/Miles-English). When I’m paid to work on a project that a company will depend on for 50 years I expect to deliver professional quality results. To me that means a program that can be maintained, modified, expanded, and understood.
“…without the will, having a way does not help”. Some companies require that code be “in company standard format” (I believe Google does) before it can be checked in. Requiring a 2 paragraph “explanation” before code can be checked in “provides the will”, at least for the programmer. It is up to the Editor-in-Chief to insist that it is written to fit into the book.
I’m going WAY out on a limb here but I’m going to make the claim that it is the mark of a PROFESSIONAL programmer that they provide human readable documentation with their product that allows the next generation of the team to maintain, modify, extend, and understand the product.
I believe that, after many POS projects (like the Railroad Retirement disaster https://fcw.com/Blogs/The-Spec/2015/05/rrb-legacy-systems.aspx) companies will stop this nonsense about Google whiteboard complexity interviews and look for professionals who can deliver a well written, easily maintained, and properly explained project.
If I could change one thing at places like Google, Microsoft, Amazon, etc. it would be to embed the meme that “Professional Programmers write Literate Programs”. Imagine the flow of interesting books that would create (e.g. “The Google TensorFlow book”, the “Microsoft Excel book”, the “Amazon AWS Container” book), all of which could be READ. You could “judge a programmer by their literate programs”.
It’s the late ’90s, not the early 70’s. Its time to be professional.
@Cook “Weave”, as you know, is Knuth’s name for extracting the document. Axiom does not needs a “weave” step. The “source code” IS a latex document so there is no need to “weave”.
“Tangle” is Knuth’s name for extracting code. The Axiom tangle program is a trivial C program (172 lines) that extracts named blocks from the document. Using it, an IDE can easily extract any function from any book. Or you could use a text editor and a command line.
So changing to a literate development style required a 172 line C program. Oh, and a change of mindset.
Of course, you can write literate programs in HTML if you want. See http://axiom-developer.org/axiom-website/litprog.html
It’s not the tool. It’s the Long-Term mindset. It is rumored that 80% of programming is maintenance. Yet we don’t act that way when we develop. The Pentagon still uses IBM Series/1 computers with 8 inch floppies, probably because they no longer know how to upgrade the programs. See http://www.cnn.com/2016/05/26/us/pentagon-floppy-disks-nuclear/
I had a friend who worked at **** bank. On her first day they showed her the Wang MAINFRAME, a custom 6-million dollar machine (Wang doesn’t make mainframes, I thought). They built it to run the Fructurization algorithm which was originally run on a Wang mini. But they lost the source code and nobody understood it so… (It turned out that years later she met Fructur and found he had the printout of the source code on his shelf. He still worked there. Nobody thought to ask him.)
As an aside, I fear Neural Net programs. Few people “sort-of” know how they work and have access to the training data. Deploying NNs to drive autonomous cars is an unfolding disaster. I suspect they are not trained on baby carriages crossing intersections. We ARE about to find out though. I don’t know how you maintain, modify, extend, or understand such a program.
Think LONG TERM and have pity on the next guy.
Thanks for letting me know about Physically Based Rendering. I hadn’t heard of that book. I’ve ordered a copy and look forward to reading it.
@John I consider PBR the gold standard of literate programming. Please let me know what you think of it. My other gold standard book is Lisp in Small Pieces (https://www.amazon.com/Lisp-Small-Pieces-Christian-Queinnec/dp/0521545668). I’m doing everything I can to try to write such “Hawaii-Class” programs.
I have to say that if I had a company, I’d hire any of those people in a heartbeat. The quality of their programming skills is obvious.
The only things lacking are electronic. We ought to be able to “drag and drop” code from the book. We ought to be able to watch embedded graphs, gifs, video, etc. I can ALMOST do these things in my Axiom PDFs. And the IDE should edit the book as a book. At the moment I edit the latex, then kick off a ‘make’ that (1) rebuilds the PDF, (2) extracts the code and builds it, and (3) runs all the tests, (4) pops up the new PDF. So I have no real distinction between “edit the code” and “edit the book”.
The larger goal (for me) is “Reproducible Research”. We need to include the ACTUAL source code in published papers. If you publish a paper (or give a talk at a conference) I should be able to drag the code from your paper and run it while you give the talk. (http://www.jstatsoft.org/v46/i03/paper).
Thank you for writing about Literate Programming. So few people have even heard of it. I spent years at CMU and the grad students in Comp. Sci. hadn’t even heard of Knuth, let alone LP. No wonder Google wastes time on whiteboarding.
@Tim – I’m still a baby in these matters – relatively speaking – but I agree most fervently the need for documentation that explains *why* specific design decisions were made in the way that they were. I also believe that literate programming (or something like it) may well be part of the solution.
I’m also pretty sure that improving standard practice will require both tools and marketing, and that presentation and polish will be critical both to adoption and to utility.
I’m slowly putting together my own tools for this (and other reasons) – do you have any pointers or references? Ideally, I’d like to find a literate programming solution written in Python that I can easily adopt and extend.
It’s not the tools, it’s the combination of human-oriented text for understanding with code, where the text is the focus.
That said, since the idea is to arrange the code so it can be explained in logical order your chosen solution needs four features.
(0) it needs a documentation format
(1) It needs to be able to name code.
(2) It needs to be able to extract a block of named code.
(3) It needs to be able to order those blocks for the compiler.
A documentation format (0) could be a .txt file, .html file, .org file, .md markdown files or any documentation format you like. I use latex because it gives me what I consider “professional quality” but you can use anything that allows you to prioritize the focus on text over the code.
To name code (1), suppose you have a simple tag that surrounds code with a name. I use latex so I wrote a tag to name a chunk of code:
code goes here
but you could do this in any language. In my litprog.html link above I use html tags on pre
To extract code (2), you need a tool to extract those blocks (by convention, blocks with the same name get concatenated). So you need a python program that reads your chosen text (.txt, .html, .tex, etc), puts every named chunk into a hash table, and dumps any block you name to standard output. That’s probably 50 lines of python (or you can use my C program).
To arrange output (3), you need a way to order the chunks for the compiler. I use another tag I call ‘getchunk’. During the hash table extraction I replace every ‘getchunk’ with its hash table entry so the getchunks disappear on output. For HTML it looks like:
This allows code to be “clipped out of code” for detailed explanation. But it is also the trick that allows code to be arranged for the compiler.
To arrange code for the compiler, make a chunk for that. For example,
and then use your extraction program to extract “compiler”… which, by the above logic will expand each getchunk… which will output the full program. (See the litprog.html example above).
All you need to start doing Literate Programming is a text format, a coding convention, a trivial extraction tool (50 lines of code), and a change of focus on the long term, professional quality work.
It works in any language. I’m writing a book on Clojure (http://daly.axiom-developer.org/clojure.pdf) which is written in Java. Java has the annoying POS convention that file system structure MUST conform to the function name.
In the chapter on “Building Clojure” I include the C extraction program. Use it to extract the Makefile (also included). Running that Makefile rebuilds the Java source tree, runs the Java compiler to build Clojure, runs the test suite, and rebuilds the PDF so you always have the latest version. So the cycle is
edit the text/code
Writing a Literate Program is trivial. Writing a beautiful, professional-quality book is hard.
To quote Ralph Gomory again, each programmer costs 1/3 to 1/2 million dollars a year (including all overhead like office space and health plans). So a team of 9 programmers for 3 years costs 9 million dollars. If it costs 9 million dollars to build a building would you expect them to fling nails at wood and call it “done”? Or would you expect blueprints, engineering, safety inspection, permits, … in short, a Professional job? Why would you expect less for your 9 million dollar program?
Professional programmers generate quality documentation.
Literate programs are the gold standard.
Sigh. The comment tool used here ate my HTML examples. You’ll have to look at the litprog.html link above to see them.
Tim, I notice your loop above doesn’t include reading the rendered pdf. Is that happening in a separate window? I’ve been following with interest your comments here and in other threads on the subject. However, I’ve been frustrated in the past by having to simultaneously manage pdf and text editor, and that experience has led me to deemphasize rendering in my own practice of LP, and keep the literate sources in a markdown-like form that is directly readable without an extra rendering step: http://akkartik.name/post/literate-programming (which you’ve commented on)
Ideally the explanation and the code are always up-to-date, in-sync, and tested. Your process may differ but here’s my attempts at better quality. Ideas for improving quality development are welcome.
I work in Emacs on Linux editing a Latex file which also contains the code. I display the PDF is a separate window using evince, which has the property that if the PDF changes then evince will redisplay it automatically.
After I edit the document (text or code), I swap to a shell buffer and type ‘make’. That rebuilds the PDF so it causes evince to redisplay. (It also rebuilds the program and runs the tests).
Every change I make soon gets checked with a complete system rebuild. Any document failures or program failures show up immediately. When writing code I run ‘make’ every 10-15 lines or so. When writing words I run ‘make’ every few paragraphs. That way I know that whatever I broke was likely something local. I tend to ‘git commit’ the code frequently. Since 2003 I have averaged over 4 commits per day on Axiom.
At the end of the day I know that the program will build and test properly and the explanations are up to date. (Not bug-free, of course. I’m building a proof-checker into the process but that’s a whole other subject we’ll ignore.)
That said, I’m a down-to-the-bits person. I choose tools that allow me to know exactly what bits go where and why. I don’t like programs that do things for me (e.g. IDE refactor) or mangle my code. In short, I’m a curmudgeon, so my process will probably not work for everyone.
Find a process that works for you. Ponder why a bug got past your process and try to fix the root cause. Don’t just fix the bug, fix the process that let it occur. Focus on anything that improves quality. Keep “raising your game”. Be a professional.
I gave a talk about this at the “Write the Docs” conference (where, unfortunately my machine hard-crashed so I had to give the talk without slides). It was given to people who write documentation for companies:
I like the idea of literate programming very much, yet every time I see a literate programm, I find, that it is not working. Potentially it works for small programs of a few pages, outlines of algorithms – of the length of a journal article. For anything more complicated I never found it useful.
That goes for Knuth’s own weave and tangle. Those resulting pdfs that I have seen have hardly been better than what a text-only description of the programm accompagnied with a sparsely-commented, but well-named source code could have down.
It is frustrating, because the idea of literate programming is so convincing, but in practice I find it neither beneficial as a program-writer, nor as a program-reader.
And I am not only talking about tangle and weave. The last piece of literate haskell that I read, I stripped away all text and converted it into the bare source file. I learned much better from that source-only file with occasional jumps to the original text that I did when I tried scrolling over the text-and-code mixture. Python projects often make extensive use of docstrings, docstrings align with the structure of the code (so no extra step to generate code) and encourage documenting code while writing ist. In practice, I often do not find these docstrings very viable, the API-documentations that are created with them are often very poor.
However, this is only a paradox if you think that maintaining a bigger code base and explaining that code is completely aligned. In fact it can be orthogonal (and even diametral). Each aspect should get the attention it deserves. So I am definitely argueing for good documentation, however, I maintain my view that code should speak for itsself (good naming, some comments at central places to help the reader, unit tests communicating assertions on code & functionality) and documentation (both docs for developers/maintainers, and for users) should be well-written text.
Holger: Yes I agree, most literate programs out there are not very useful. I linked above to my rant about that. However, I wouldn’t conclude that the entire approach has been refuted. Tim’s comments here have been quite eloquent, and so it might be worth checking out his project. And then there’s my project; it’s 20kLoC (substantial, but not as large as Tim’s Axiom) that a couple of people have tried to read and said they enjoyed. If you tried to read it, I’d be very interested to hear your thoughts on what was easy and what was hard: https://github.com/akkartik/mu
I can handle a 10k line program. I can’t handle a 10M line program (e.g. Windows). I find that real programs, meaning large applications (think Firefox, Gimp, Veterans Admin, Railroad Retirement, Nuclear Missiles, IRS taxes) are too large to “put into my head”. (ref: Dijkstra “The Humble Programmer)
If the problem is a simple one (e.g. a phone game app), if your problem isn’t one the company cares about, if your problem fits in 10k lines, then you might find that Literate programming is overkill. Don’t use it.
But if you own a company that depends on a program written by a team that will be gone next year, you really don’t want to have new people reverse engineering the code. That’s why LP is a Long-Term thinking process.
If the problem is complex and requires a lot of background (like the Physically Based Rendering task), then reverse-engineering a piece of code without the background knowledge is hopeless. I have some code on Lie-algebras that could use a comment or two. It is only a few dozen lines but it is pretty opaque.
If the programming style is obscure you might find it difficult to reverse-engineer. Look at the Alice-bot python code. The control flow depends on the fact that dictionary lookup throws an exception. There are 3 different data structures combined into a single dictionary so the semantics of the exception depends on why you were asking. It took weeks to reverse engineer. Oh, I wished for LP.
LP isn’t a silver bullet. Don’t use it for everything (although I admit that I do). But if you have to reverse-engineer the IRS admin code I’ll bet you’d wish someone thought to use it, because who remembers that special case tax cut for sled dogs?
if (sldDgExcpt == 3) …
Clearly I’m selling a viewpoint here and ignoring the exceptions.
But I’m hoping that in the future the exceptions will be FizzBuzz and not the Air Traffic Control code. Nobody expects blueprints for dog houses because flinging nails at wood is just as good. But if I’m paying 9 million dollars for a program that my company really needs then I expect professional quality work. I don’t want my auto mechanic reverse-engineering my autonomous parking neural network.
Life is going to get even worse. I program Altera FPGAs. Intel bought Altera last year and will release a CPU/FPGA combination shortly (I think big companies like Google can have them now). When I write my special “universal number” instructions (see Gustafson’s “The End of Error”) in Verilog so I can do infinite numerics, somebody somewhere is going to hope I wrote it in a literate programming style.
I agree with you that “code should speak for itself” and people should write “dirt-simple” code. But no amount of code clarity will reveal “WHY” the code was written. Enter LP, stage left.
Aligning the code and the explanation is the Editor-in-Chief’s job. That’s why the Editor should be an English major, not a developer. If the Editor can’t understand WHY you wrote it, then you can’t check it in.
Recently I’ve been writing programs in a style inspired by literate programming. However, instead of interspersing code and writing and dealing with the tangle and weave problem, I’ve been writing “literary commits”. All of my Git commit messages are very thorough; chronicling the reasoning and impetus behind every change.
One of the interesting advantages of this approach addresses Peter Norvig’s comment about constructing reading paths to suite each reader’s needs. By writing “literate commits”, we can use tools like `git blame` to construct narratives about how and why a particular piece of code came to be.
What’s really cool about this kind of writing/programming is that it also presents how a program evolved. Rather than being presented with a final, eloquently explained architecture, readers are given an entire historical text explaining in glorious detail all of the trials and tribulations a piece of software went through to arrive in its present state.
I’ve only been writing small practice programs with this approach, but it’s proven to be fairly valuable to me already. I wrote up a bit more about the idea here:
It’s a pretty simple idea (TL,DR: write better commit messages), but hopefully it’s interesting to some people.
Pete: yes, my approach was also inspired by version control. However, the issue that I ran into was that the version history of a project is immutable. If a project undergoes a massive reorganization for version 3, all the value of the literate commits for versions 1 and 2 is lost. I elaborate on this idea in the final section of the post I’ve been linking to in this thread:
@Corey It’s an excellent idea. I mentioned something similar (although not git logs) in response to a question during my “Read the Docs” talk. There are three problems, though.
The first is that commit messages are not actually associated with the code seen by every other tool. Who would know to look for them? I suppose there could be a “git weave” that matched them to the code.
Second, if I collected all of the git log messages it would be like taking several versions of a book, ripping out every page, and putting the whole pile in “historical order”. The git log messages don’t have versioning so I end up with either one old message or a couple dozen “historically updated” messages.
The third goes directly to Norvig’s criticism. Books don’t necessarily dictate the order. Indeed a lot of the books I read have a preface that shows several logical paths through the text.
In the Literate Program book “Physically Based Rendering” (buy it… and no, I don’t get a cut). the section on “Indexing and Cross-Referencing” tells the tale. Quote:
“The following features are designed to make the text easier to navigate. Indices in the page margins give page numbers where the functions, variables, and methods used on that page are defined (i.e. hyperlinks). Indices at the end of the book collect all of these identifiers so that it’s possible to find definitions by name. Appendix C, “Index of Fragments,” lists the pages where each fragment is defined and the pages where it is used. Within the text, a defined fragment name is followed by a list of page numbers on which that fragment is used.”
So the paper text has hyperlinks, cross-references, who-calls, and who-called links. (Of course, a PDF can do it all with hyperlinks). PBR did an amazing job overcoming the limits of physical books. They did an even better job with their explanations. What a great Hawaii book it is.
Indeed, some of the key advantages of an electronic version are that it HAS hyperlinks. It is also always up-to-date.
Ultimately I think Norvig’s criticism has no basis. At most, it is valid to say that the author’s choice of order might not be best for your needs but that’s true of any choice.
Another unmentioned but fundamental change found through literate programming is handling complexity. To quote Knuth:
“Yet to me, literate programming is certainly the most important thing that came out of the TeX project. Not only has it enabled me to write and maintain programs faster and more reliably than ever before, and been one of my greatest sources of joy since the 1980s — it has actually been indispensable at times. Some of my major programs, such as the MMIX meta-simulator, could not have been written with any other methodology that I’ve ever heard of. The complexity was simply too daunting for my limited brain to handle; without literate programming, the whole enterprise would have flopped miserably.”
When you’re tasked with a sub-problem (e.g. throttle control in a self-driving car) you need to know a lot about other subsystems (e.g. don’t use “insane mode” in a traffic circle). You might need to know under what circumstances the “brake control” might be asserted (“WHY, not “HOW”).
One major effect from years of literate programming is that while thinking in order to write the text sometimes that “think-time” helps discover missed cases or bad code.
This is especially true during code review. Other eyes look at your explanation, look at your code, and then ask, politely, why they seem to be unrelated :-)
I’m finding that literate programming makes it easier to pick back up where I left off when I’m interrupted.
John: interestingly enough, I said the same thing about tests many years ago: http://akkartik.name/post/2009-01-24-19-46-26_002-soc. I’m totally willing to believe that LP helps with context-switching as well. I was already being productive in sessions as short as 10 minutes before I started using it, so perhaps I didn’t notice.
@Corey in my experience that approach doesn’t work very well.
When I am reading source code, I need to see the “why”s right there. First because that’s the context where you need them, and second because Git archeology is a PITA when the source code evolves, files renamed, refactors, ….
To me, a Git message documents the patch, and a code comment documents the program. Sometimes the division line is fine, and sometimes my Git messages say “see the rationale in the code comment in this patch”, for example as in
Knuth just gave a talk on literate programming
Notebooks (e.g. Jupyter notebooks) are a way to move closer to literate programming IMHO.
John Kitchin does something similar at CMU:
I attended one of his classes. His presentation was done using Emacs org-mode which allows inline code, text, and execution output. Everything he did in class, including updates to his presentation, were captured immediately. The final document was available to the students. They did not have to take notes. Everything he did was exactly as presented.
Knuth claims to write 5 literate programs a week (from the above talk).
Knuth writes in “stream of conscious” mode, developing the program as he thinks about it. This is probably ideal for tools like Jupyter or Kitchin’s org-mode.
I’m focused on “book-based LP”. I feel that “we know how to use a book” so there is a good foundation for communication to people we’ll never meet. Pharr and Humphrey (Physically Based Rendering) are quoted as saying that this is challenging because authors can usually “hand wave” about some material and programmers never explain… but LP authors can’t “hand wave” and must explain.
Ultimately, though, Jupyter notebooks are just a tool. Literate programming is a mindset. In other words, hammers versus architecture.
@Tim Daly: One of the really neat features of literate programming with org-mode is that you are able to edit the woven source have the changes propogated back to the literate document. In my opinion this is a highly under-rated features that lessens the barriers to adoption and collaboration.
I think the defence of LP in this regard is along these lines:
i) LP Programs are like written in “teacher mode” (Knuth’s talk at the ACM 50th anniversary), and code- being composed of a FORMAL language like mathematical formulae- appears in the document like the formulae in mathematics literature. Only this can be tested and compiled.
Knuth is clearly interested in this- he gave a course on Mathematical Writing, and uses Mathematica to check his logic while writing “Art of”- is it too much to suggest he uses the Mathematica notebooks?
ii) The prettyprinting in the original idea is a big part of it. Again Knuth’s interest in this is apparent from his lecture series “Can computers help us create beautiful books?”.
I made a small tool on my own system to enable LP, but is a generic file editing tool, and the thing that I loved best about the outputs was the prettyprinting. It’s like reading a piece of mathematical literature, along with a working program. I can’t afford to make too many mistakes while composing formal literature that someone else might consume- and I take the task of writing code more seriously, with every sign, symbol and variable discussed, dissected and analysed, as it is done in a physics or mathematics textbook.
iii) We reuse the principles from 2,500 years of mathematical literature- people find mistakes in formulae- they inform the writer; the stability of TeX is a great proof of this; if people don’t want to read the whole thing, they take a look at the index and the contents- this is what you were talking about, I think.
iv) Programming while keeping in mind the distinction between what is newly invented and what has already been tested and re-tested produces powerfully accurate software- and follows in the tradition of naming theorems and formulae in mathematics. We still call it Dijkstra’s algorithm for a good reason- and comment lines are not good places to hold this kind of info.
Another full-length-book example of a literate program:
“Designed as a self-study guide, the book describes the real-world tradeoffs encountered in building a production-quality, platform-retargetable compiler. The authors examine the implementation of lcc, a production-quality, research-oriented retargetable compiler, designed at AT&T Bell Laboratories for the ANSI C programming language. The authors’ innovative approach-a “literate program” that intermingles the text with the source code-uses a line-by-line explanation of the code to demonstrate how lcc is built.”