Learn PowerShell as a shell first

Posted on 22 June 2012 by John

When I was learning PowerShell, I thought of it as a scripting language that has a shell. But the right way to think of it may be the opposite, a shell that has a scripting language. Apparently others have followed this same change of perspective.

Don Jones and Jeffrey Hicks open their new book Learn Windows PowerShell 3 in a Month of Lunches by explaining that they first saw PowerShell as a VBScript replacement and taught it that way.

Since late 2009, however, a shift has occurred. More and more administrators who don’t have prior VBScript experience have started trying to learn the shell. All of a sudden, our old teaching patterns didn’t work very well, because we focused on scripting and programming. That’s when we realized that PowerShell isn’t really a scripting language. It’s really a command-line shell where you run command-line utilities. Like all good shells, it has scripting capabilities, but you don’t have to use them, and you certainly don’t have to start with them.

[Don Jones wrote the first edition of the book, which I haven’t read. This quote comes from the second edition, coauthored with Jeffrey Hicks.]

Other PowerShell books I’ve read felt more like a programming language book than a book on a shell, more like, for example, The C Programming Language than The Linux Command Line. The latter seems to be a better analog for the new PowerShell 3 book: emphasis on interactive use first, programming second. At least that’s my first impression. The book isn’t finished yet—it’s part of Manning’s Early Release program—and I haven’t yet read all of the part that has been released.

If you’d like a chance to win a copy of Learn Windows PowerShell 3 in a Month of Lunches, please leave a comment below. I’ll draw a winner and the publisher will send you a copy of the book.

Update: I’ll draw a winner on Friday, June 29. Comments need to be posted by midnight Thursday in my time zone.

Review: The Linux Command Line

Posted on 25 January 2012 by John

No Starch Press recently released The Linux Command Line: A Complete Introduction (ISBN 1593273894) by William E. Shotts, Jr.

True to its name, the book is about using Linux from command line. It’s not an encyclopedia of Linux. It doesn’t explain how to install Linux, doesn’t go into system APIs, and says little about how to administer Linux. At the same time, the book is broader than just a book on bash. It’s about how to “live” at the command line.

The introduction explains the intended audience.

This book is for Linux users who have migrated from other platforms. Most likely you are a “power user” of some version of Microsoft Windows.

The book has a conversational style, explaining the motivation behind ways of working as well as providing technical detail. It includes small but very useful suggestions along the way, the kinds of tips you’d pick up from a friend but might not find in a book.

The book has four parts

Learning the shell
Configuration and the environment
Common tasks and essential tools
Writing shell scripts

The book could have just included the first three sections; the forth part is a bit more specialized than the others. If you’d prefer, think of the book has having three parts, plus a lengthy appendix on shell scripting.

The Linux Command Line is pleasant to read. It has a light tone, while also getting down to business.

Perverse hipster desire for retro-computing

Posted on 6 August 2011 by John

Here’s my favorite line from an article Life on the Command Line by Stephen Ramsay:

I also don’t do this [work from the command line] out of some perverse hipster desire for retro-computing. I have work to do. If my system didn’t work, I’d abandon it tomorrow.

That’s refreshing. Some of the more ardent command line advocates give the impression that they use the command line out of pride rather than out of a desire to get work done. Ramsay is recommending his way of working, not bragging about what he’s able to do. He has some interesting ideas, especially in his follow-up article The Mythical Man-Finger.

By the way, I’m no command line wizard; I’m a fairly typical computer user. On the other hand, my use of the command line and Emacs has been increasing.

Comparing the Unix and PowerShell pipelines

Posted on 9 June 2009 by John

This is a blog post I’ve intended to write for some time now. I intended to come up with a great example, but I’ve decided to go ahead and publish it and let you come up with your own examples. Please share your examples in the comments.

One of the great strengths of Unix is the shell pipeline. Unix has thousands of little utilities that can be strung together via a pipeline. The output of one program can be the input to another. But in practice, things don’t go quite so smoothly. Suppose the conceptual pattern is

A | B | C

meaning the output of A goes to B, and the output of B goes to C. This is actually implemented as

A | <grubby text munging> | B | <grubby text munging> | C

because B doesn’t really take the output of A. There’s some manipulation going on to prepare the output of A as the input of B. Strip these characters from these columns, replace this pattern with this other pattern, etc. The key point is the Unix commands spit out text. Maybe at a high level you care about programs A, B, and C, but in between are calls to utilities like grep, sed, or awk to bridge the gaps between output formats and input formats.

The PowerShell pipeline is different because PowerShell commands spit out objects. For example, if the output of a PowerShell command is a date, then the command returns a .NET object representing a date, not a text string. The command may display a string on the command line, but that string is just a human-readable representation. But the string representation of an object is not the object. If the output is piped to another command, the latter command receives a .NET date object, not a string. This is the big idea behind PowerShell. Commands pass around objects, not strings. The grubby, error-prone text munging between commands goes away.

Not all problems go away just because commands pass around objects. For example, maybe one command outputs a COM object and another takes in a .NET object. This is where more PowerShell magic comes in. PowerShell does a lot of work behind the scenes to implicitly convert output types to input types when possible. This sort of magic makes me nervous when I’m programming. I like to know exactly what’s going on, especially when debugging. But when using a shell, magic can be awfully convenient.

Manipulating the clipboard with PowerShell

Posted on 17 October 2008 by John

The PowerShell Community Extensions contain a couple handy cmdlets for working with the Windows clipboard: Get-Clipboard and Out-Clipboard. One way to use these cmdlets is to copy some text to the clipboard, munge it, and paste it somewhere else. This lets you avoid creating a temporary file just to run a script on it.

Update: Looks like

For example, occasionally I need to copy some C++ source code and paste it into HTML in a <pre> block. While <pre> turns off normal HTML formatting, special characters still need to be escaped: < and > need to be turned into < and > etc. I can copy the code from Visual Studio, run a script html.ps1 from PowerShell, and paste the code into my HTML editor. (I like to use Expression Web.)

The script html.ps1 looks like this.

    $a = get-clipboard;
    $a = $a -replace "&", "&amp;";
    $a = $a -replace "<", "&lt;";
    $a = $a -replace ">", "&gt;";
    $a = $a -replace '"', "&quot;"
    $a = $a -replace "'", "&#39;"
    out-clipboard $a

So this C++ code

    double& x = y;
    char c = 'k';
    string foo = "hello";
    if (p < q) ...

turns into this HTML code

    double&amp; x = y;
    char c = &#39;k&#39;;
    string foo = &quot;hello&quot;;
    if (p &lt; q) ...

Of course the PSCX clipboard cmdlets are useful for more than HTML encoding. For example, I wrote a post a few months ago about using them for a similar text manipulation problem.

If you’re going to do much text manipulation, you may want to look at these notes on regular expressions in PowerShell.

The only problem I’ve had with the PSCX clipboard cmdlets is copying formatted text. The cmdlets work as expected when copying plain text. But here’s what I got when I copied the word “snippets” from the CodeProject home page and ran Get-Clipboard:

    Version:0.9
    StartHTML:00000136
    EndHTML:00000214
    StartFragment:00000170
    EndFragment:00000178
    SourceURL:https://www.codeproject.com/
    <html><body>
    <!--StartFragment-->snippets<!--EndFragment-->
    </body>
    </html>

The Get-Clipboard cmdlet has a -Text option that you might think would copy content as text, but as far as I can tell the option does nothing. This may be addressed in a future release of PSCX.

Improved PowerShell prompt

Posted on 17 August 2008 by John

A while back I wrote a post on how to customize your PowerShell prompt. Last week Tomas Restrepo posted an article on a PowerShell prompt that adds color and shortens the path in a more subtle way. I haven’t tried it out yet, but his prompt looks much better than what I’ve been using.

If you’re a long-time Windows user you might be worried that all this PowerShell stuff is starting to look a lot like Unix. Well, it is. Some of the folks on the PowerShell team have a Unix background and they’re bringing some of the best of Unix to Windows. The Unix world has more experience operating from the command line and so it’s wise to learn from them.

On the other hand, PowerShell is emphatically not bash for Windows. PowerShell is thoroughly object oriented and in that respect unlike any Unix shell. Also, PowerShell is strongly tied to Microsoft libraries, particularly .NET but also COM and WMI.

PowerShell one-liner to filter a sitemap

Posted on 1 August 2008 by John

Suppose you have an XML sitemap and you want to extract a flat list of URLs. This PowerShell code will do the trick.

        ([ xml ] (gc sitemap.xml)).urlset.url | % {$_.loc}

This code calls Get-Content, using the shortcut gc, to read the file sitemap.xml and casts the file to an XML document object. It then makes an array of all blocks of XML inside a <url> tag. It then pipes the array to the foreach command, using the shortcut %, and selects the content of the <loc> tag which is the actual URL.

Now if you want to filter the list further, say to pull out all the PDF files, you can pipe the previous output to a Where-Object filter.

        ([ xml ] (gc sitemap.xml)).urlset.url | % {$_.loc} |
        ? {$_ -like *.pdf}

This code uses the ? shortcut for the Where-Object command. The -like filter uses command line style matching. You could use -match to filter on a regular expression.

Comparing PowerShell and Bash

Posted on 8 July 2008 by John

On the Windows PowerShell blog, Jeffrey Snover links to a article in Linux Magazine by Narcus Nasarek comparing Windows PowerShell and Linux’s bash shell.

The article’s sequence is unexpected. Not until near the end of the article does Nasarek get to the main difference between PowerShell and bash: PowerShell pipes objects, not text. Nasarek says regarding PowerShell’s object pipeline “Bash cannot compete here.” He says that the disadvantage of bash in this regard is that “it relies on the abilities of external programs to handle data structures.” That is an understatement. The disadvantage of bash is that it requires fragile, ad hoc text manipulation to pluck data out of the pipeline.

Nasarek is being fair to PowerShell, but he was limited by space. He had only two pages for his article, and only about half of those two pages were devoted to text.

Customizing the PowerShell command prompt

Posted on 12 May 2008 by John

By default, the PowerShell command prompt does not echo the current working directory. To customize the command prompt, simply create a function named prompt. If you want this customization to persist, add it to your profile.

For example, adding the following line to your profile will cause the working directory to be displayed much like it is in cmd.exe.

    function prompt { "$pwd>" }

However, the prompt function can contain any code at all. Here’s a prompt function that will display the right-most part of the working directory. This keeps long working directory names from taking up most of the space at the command line.

    function prompt
    {
        $m = 30 # maximum prompt length
        $str = $pwd.Path
        if ($str.length -ge $m)
        {
            # The prompt will begin with "...",
            # end with ">", and in between contain
            # as many of the path characters as will fit,
            # reading from the end of the path.
            $str = "..." + $str.substring($str.length - $m + 4)
        }
       "$str> "
    }

For example, if

    C:Documents and SettingsAdministratorMy DocumentsMy Music

is the current directory, the prompt would be

    ...atorMy DocumentsMy Music>

Update: See the next post for an update.

Integrating the clipboard and the command line

Posted on 30 April 2008 by John

Two of my favorite cmdlets from the PowerShell Community Extensions are get-clipboard and out-clipboard. These cmdlets let you read from and write to the Windows clipboard from PowerShell. For example, the following code will grab the contents of the clipboard, replace every block of white-space with a comma, and paste the result back to the clipboard.

(get-clipboard) -replace 's+(?!$)', ',' | out-clipboard

I saved this to a file comma.ps1 in my path and run it when I get a list of numbers from one program delimited by newlines or tabs and need to make it the input to another program expecting comma-delimited values. For example, turning a column of numbers into an array for R. I copy one format, run comma.ps1, and paste in the new format.

In case you’re curious about the mysterious characters in the script, s+(?!$) is a regular expression describing where I want to substitute a comma. The s refers to white-space characters (tabs, spaces, newlines) and the +says this is repeated one or more times. So match one or more consecutive white-space characters. That would be enough by itself, but it would replace trailing white-space with a comma too, so I might get an unwanted comma at the end. The sequence (?!$) fixes that. The $ matches the end of line. The (?! before and the ) after form a negative look ahead, meaning “except when the thing inside matches.” So taken all together, the regular expression matches chunks of white-space except at the end of the input.

Update: See Manipulating the clipboard with PowerShell

Command line