Computational survivalist

Some programmers and systems engineers try to do everything they can with basic command line tools on the grounds that someday they may be in an environment where that’s all they have. I think of this as a sort of computational survivalism.

I’m not much of a computational survivalist, but I’ve come to appreciate such a perspective. It’s an efficiency/robustness trade-off, and in general I’ve come to appreciate the robustness side of such trade-offs more over time. It especially makes sense for consultants who find themselves working on someone else’s computer with no ability to install software. I’m rarely in that position, but that’s kinda where I am on one project.

Example

I’m working on a project where all my work has to be done on the client’s laptop, and the laptop is locked down for security. I can’t install anything. I can request to have software installed, but it takes a long time to get approval. It’s a Windows box, and I requested a set of ports of basic Unix utilities at the beginning of the project, not knowing what I might need them for. That has turned out to be a fortunate choice on several occasions.

For example, today I needed to count how many times certain characters appear in a large text file. My first instinct was to write a Python script, but I don’t have Python. My next idea was to use grep -c, but that would count the number of lines containing a given character, not the number of occurrences of the character per se.

I did a quick search and found a Stack Overflow question “How can I use the UNIX shell to count the number of times a letter appears in a text file?” On the nose! The top answer said to use grep -o and pipe it to wc -l.

The -o option tells grep to output the regex matches, one per line. So counting the number of lines with wc -l gives the number of matches.

Computational minimalism

Computational minimalism is a variation on computational survivalism. Computational minimalists limit themselves to a small set of tools, maybe the same set of tools as computational survivalist, but for different reasons.

I’m more sympathetic to minimalism than survivalism. You can be more productive by learning to use a small set of tools well than by hacking away with a large set of tools you hardly know how to use. I use a lot of different applications, but not as many as I once used.

Related posts

10 thoughts on “Computational survivalist

  1. I didn’t know about the -o option but `grep … | sort | uniq -c | sort -nr` is very firmly embedded into my muscle memory.

  2. I didn’t know about `-o`. That’s handy.

    I think I told you about Gow years ago, and you weren’t interested. I’m glad you’ve come around. :)

  3. That is the bioinformatician way! We stick to the unix basics because they are reliable, efficient (handy when working with huge files), work on text files and they are available on most unix machines.

  4. Severin Pappadeux

    Just checked and in powershell core command

    > (Get-Content “.\pareto.R” | Select-String -Pattern “a” -AllMatches ).Matches.Count

    produced exactly the same count (26) on both Windows 10 and Linux (ubuntu 18.04) powershell installations.

Leave a Reply

Your email address will not be published. Required fields are marked *