Small-scale automation


Saving keystrokes is overrated, but maintaining concentration is underrated.

This post is going to look at automating small tasks in order to maintain concentration, not to save time.

If a script lets you easily carry out some ancillary task without taking your concentration off your main task, that’s a big win. Maybe the script only saves you five seconds, but it could save you from losing a train of thought.

If your goal in writing a script is to preserve concentration, that script has to be effortless to run. It’s worth taking a few minutes to search for a script that is going to save you an hour. But if the purpose of a script is to preserve your state of flow, having to search for it defeats the purpose.

Remembering what you’ve written

I’ve often said to myself “I’ve had to do this task several times. I should automate it!” Good idea, except I couldn’t quickly find the code later when I needed it.

I’ve heard many people say you should automate repetitive tasks; I’ve never heard anyone discuss the problem of remembering what you’ve automated and how to invoke it. Maybe there’s less discussion of the memory problem because the solution is personal. It depends, for instance, on what tools you use and what you find memorable.

One suggestion would be to periodically review what you’ve written, say once a month [1]. Maybe you’ve put useful aliases in your shell configuration file. Skimming that config file occasionally could help you remember what’s there. If you have a directory where you keep custom scripts, it could help to browse that directory once in a while. It helps if there aren’t too many places you need to look, which leads to the next section.

Tool priorities

It would also help to minimize the number of tools you use, or at least the number of tools you customize.

And even with a very minimal tool set, it helps to have a primary emphasis on one of those tools. For example, maybe your work environment consists mostly of a shell, a programming language, and an editor. When it’s not obvious which tool to pick, are you going to write a shell script, a program, or an editor extension? By picking one tool as your default, you get better at that tool, accumulate more sample code for that tool, and have fewer contexts to explore when you’re looking for something you’ve written.


[1] A long time ago I heard someone say he reads documentation ever Friday afternoon. I did that for a while and recommend it. Maybe set aside a few minutes each Friday afternoon to review and tweak config files. If you don’t get through everything, pick up next week where you left off.

6 thoughts on “Small-scale automation

  1. “I’ve heard many people say you should automate repetitive tasks; I’ve never heard anyone discuss the problem of remembering what you’ve automated and how to invoke it.”

    I’ve discussed it many times and at length, but everyone just ignores it: package everything into OS packages, from configuration to tools you write, and you will never face this problem again, because it’s not really a problem since it has been solved by the fathers of UNIX a long time ago already.

    There is an industry standard, and that standard says that one’s software should go under company’s stock symbol; for individuals, that won’t be the case, so for you, let’s pick “jdc”, for “John D. Cook”:

    /opt/jdc/bin/ should contain your tools as executables; no extensions, and they should look and feel no different than the ones in /usr/bin/;

    /opt/jdc/sbin/ should contain executables which require elevated privileges, be they root or some technical user, like named, pdns, or oracle; if it’s meant to run as a daemon or not meant to be run by regular users, sbin/ directory it is;

    /opt/jdc/lib/ is where 32-bit and platform independent libraries and back-end files or executables go;

    /opt/jdc/lib64 on GNU/Linux or /opt/jdc/lib/64 if you are on Solaris / illumos / SVR4 UNIX is where the 64-bit shared object libraries of your application would go;

    /opt/jdc/share/man/man1/ is where the manual pages would go for applications in bin/;

    /opt/jdc/share/man/man1m/ if you’re on a SVR4 UNIX or /opt/jdc/share/man/man8/ if you’re on a BSD or GNU/Linux is where manual pages for tools that you put in /opt/jdc/sbin/ would go;

    /opt/jdc/share/man/man5/ on GNU/Linux or /opt/jdc/share/man/man4/ on SVR4 / illumos / Solaris is where the file format manual page(s) of your configuration files would go;

    /opt/jdc/share[/application]/ is where read-only data for your tool would go, such as for example templates or ancillary files which your tool would need;

    /etc/opt/jdc[/application]/ is where the configuration for your programs would go; /etc/opt/jdc if it’s a single configuration file, and /etc/opt/jdc[/application]/ if there is more than one; that way, you have a catalogue of all your configuration for all your applications in one, industry standard place;

    /var/opt/jdc[/application]/ is where the read-write data, such as logs, input and output of your automation tools would go, but bear in mind that logs should really be streamed to syslogd via the logger utility, so that their location and logging configuration can be maintained by the OS in one place (or forwarded to a central logging system if you configured it that way, perhaps for monitoring or dashboards).

    This is all based off of the AT&T System V Release 4.0 filesystem specification, and documented here for developers:

    the Linux Standards Base, Filesystem Hierarchy Standard is in turn derived from the above specification by AT&T:

    …and if you then make a package which adds /opt/jdc/bin and /opt/jdc/sbin to the system-wide PATH, and deliver manual pages with your tools and run catman -w in the package postinstall, you’ll never again have to worry about what you did and didn’t automate. Once you make that package, say JDCsetpath, you could then formally declare in your JDCdevtools package that it requires JDCsetpath, and the OS would enforce that and ensure that it is installed in the correct order. This, by the way, are the building blocks of information technology architecture.

    Let’s say you remember that you wrote some tool, but don’t remember what it was called, however you remember that you delivered it as part of your JDCdevtools package; you could then query the OS to list the files in that package and tell you what you delivered and even tell you what the manual page(s) is(are) called.

    Granted, you could now make the counterargument that by the time you got all this done, your concentration would be completely shot by going off on these tangents, but long term, one ends up building a library of more and more tools and have less and less diversions, and you would never again have to remember what you already automated or how to find it. I’m writing from decades of experience to draw upon here.

  2. I have code in my shell (zsh) setup that saves all of the commands (but not the arguments) to a local file.

    I use this file and a script that analyzes it to do the following regularly:
    – check which aliases I have not used recently
    – see the statistics on which commands/aliases I’m using the most

    In particular, checking which aliases I have not used recently leads to me either starting to re-use the alias, deleting the alias, or creating a new one.

    Given that my .alias_zsh file is 1100+ lines, it’s become a necessity.

  3. I write a script, use it a few times, then forget about it. A year later I can’t remember what the script was called so I end up rewriting it. When I go to save it I give it what I think is the best name for it and it turns out that’s what I named it as last year!

  4. >Saving keystrokes is overrated

    I try to save keystrokes when using the microwave.
    If I want about a minute and a half, why type 130 when I can type 90 – one less button press!
    If I want one minute forty seconds, why type 140 when I can type 99 – which is only one second less than I wanted, but even better than the previous example, I’m pressing the same button twice, not moving between buttons, that’s got to be a saving, right?!

    I’ve spent more time thinking about this than I have saved using those tricks in my entire life!

  5. I’ve certainly thought about this, but I have not talked about it.

    The most common tasks that I want to automate are manipulating data to the format I need as input to my statistical analysis software. I have a collection of scripts that I use for that purpose, but what I *don’t* do is go through them on a regular basis, and so I have often spent more time than I needed to looking for the one I have, in the place I know, but which one?

    I like the idea of setting aside some time weekly to curate my own scripts, and I will adopt it.

Comments are closed.