What does the redirection operator > in PowerShell do to text: leave it as Unicode or convert it to ASCII? The answer depends on whether the thing to the right of the >
operator is a file or a program.
Strings inside PowerShell are 16-bit Unicode, instances of .NET’s System.String
class. When you redirect the output to a file, the file receives Unicode text. As Bruce Payette says in his book Windows PowerShell in Action,
myScript > file.txt
is just syntactic sugar formyScript | out-file -path file.txt
and out-file
defaults to Unicode. The advantage of explicitly using out-file
is that you can then specify the output format using the -encoding
parameter. Possible encoding values include Unicode, UTF8, ASCII, and others.
If the thing on the right side of the redirection operator is a program rather than a file, the encoding is determined by the variable $OutputEncoding
. This variable defaults to ASCII encoding because most existing applications do not handle Unicode correctly. However, you can set this variable so PowerShell sends applications Unicode. See Jeffrey Snover’s blog post OuputEncoding to the rescue for details.
Of course if you’re passing strings between pieces of PowerShell code, everything says in Unicode.
Thanks to J_Tom_Moon_79 for suggesting a blog post on this topic.
Thanks for posting this info. It happened to provide a succinct answer to a problem I had been working on for a few hours. Thanks for your straight forward delivery of the topic.
I guess it leaves the encoding for an email object in unicode. This is causing me all kinds of problems. Any chance you know how to force powershell to send email in ascii instead of unicode?
for ASCII, use out-file -filePath file.txt -encoding ASCII
Very helpful thanks.
Got caught out recently trying to process some files from Mapinfo without losing the little Copyright symbol and while still allowing the Application to read them. Tried UT8 first, as ASCII seemed to lose the symbol.
Now trying Unicode.
It might turn out that the default was the one I needed!
Thanks a lot. This was really helpful for me.
Take note that powershell cmdlet out-file with utf8 encoding includes the byte order mark and there is no option for turning that off with this cmdlet. I had to use [System.IO.File]::WriteAllBytes(string, string) which, by default, writes utf8 without byte order mark.
The correct parameter is -FilePath not -path.
Please correct.
Out-File : A parameter cannot be found that matches parameter name ‘path’.