PowerShell output redirection: Unicode or ASCII?

What does the redirection operator > in PowerShell do to text: leave it as Unicode or convert it to ASCII? The answer depends on whether the thing to the right of the > operator is a file or a program.

Strings inside PowerShell are 16-bit Unicode, instances of .NET’s System.String class. When you redirect the output to a file, the file receives Unicode text. As Bruce Payette says in his book Windows PowerShell in Action,

myScript > file.txt is just syntactic sugar for myScript | out-file -path file.txt

and out-file defaults to Unicode. The advantage of explicitly using out-file is that you can then specify the output format using the -encoding parameter. Possible encoding values include Unicode, UTF8, ASCII, and others.

If the thing on the right side of the redirection operator is a program rather than a file, the encoding is determined by the variable $OutputEncoding. This variable defaults to ASCII encoding because most existing applications do not handle Unicode correctly. However, you can set this variable so PowerShell sends applications Unicode. See Jeffrey Snover’s blog post OuputEncoding to the rescue for details.

Of course if you’re passing strings between pieces of PowerShell code, everything says in Unicode.

Thanks to J_Tom_Moon_79 for suggesting a blog post on this topic.

7 thoughts on “PowerShell output redirection: Unicode or ASCII?

  1. Thanks for posting this info. It happened to provide a succinct answer to a problem I had been working on for a few hours. Thanks for your straight forward delivery of the topic.

  2. I guess it leaves the encoding for an email object in unicode. This is causing me all kinds of problems. Any chance you know how to force powershell to send email in ascii instead of unicode?

  3. Very helpful thanks.
    Got caught out recently trying to process some files from Mapinfo without losing the little Copyright symbol and while still allowing the Application to read them. Tried UT8 first, as ASCII seemed to lose the symbol.
    Now trying Unicode.
    It might turn out that the default was the one I needed!

  4. Take note that powershell cmdlet out-file with utf8 encoding includes the byte order mark and there is no option for turning that off with this cmdlet. I had to use [System.IO.File]::WriteAllBytes(string, string) which, by default, writes utf8 without byte order mark.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>