Remembering COM

In the late ’90s I thought COM (Microsoft’s Component Object Model) was the way of the future. The whole architecture starting with the IUnknown interface was very elegant. And to hear Don Box explain it, COM was almost inevitable.

I was reminded of COM when I saw the slides for Kevlin Henney’s presentation Worse is better, for better or for worse. Henney quotes William Cook’s paper that has this to say about COM:

To me, the prohibition of inspecting the representation of other objects is one of the defining characteristics of object oriented programming.  …

One of the most pure object-oriented programming models yet defined is the Component Object Model (COM). It enforces all of these principles rigorously. Programming in COM is very flexible and powerful as a result.

And yet programming COM was painful. I wondered at the time why something so elegant in theory was so difficult in practice.  I have some ideas on this, but I haven’t thought through them enough to write them down.

Just as I was starting to grok COM, someone from Microsoft told me “COM is dead” and hinted at a whole new approach to software development that would be coming out of Microsoft, what we now call .NET.

COM had some good ideas, and some of these ideas have been reborn in WinRT. I’ve toyed with the idea of a blog post like “Lessons from COM,” but I doubt I’ll ever write that. This post is probably as close as I’ll get.

By the way, this post speaks of COM in the past tense as is conventional: we speak of technologies in the past tense, not when they disappear, but when they become unfashionable. Countless COM objects are running on countless machines, including on Mac and Unix systems, but COM has definitely fallen out of fashion.

Related post: Technologies never die

Humbled by a debugging book

I started developing software for Windows in 1995, but I hardly know anything about Windows. I feel like my understanding of Windows peaked around the turn of the millennium and has declined since. I was reminded of the depth of my ignorance by a new book I received recently, Inside Windows Debugging.

This book goes way beyond stepping through code with the Visual Studio debugger. It dives into the architecture of Windows and the details of how different kinds of debugging work: user-mode, kernel-mode, native, managed, remote, etc. It has a little to say about Visual Studio, but focuses on more advanced tools such as WinDbg.

Obviously Inside Windows Debugging is a book about debugging. But it’s also a good book for someone wanting to know more about Windows internals.

WinRT, Projections, and COM

Martyn Lovell gave a great talk on the new Windows Runtime at Lang.NEXT this week. You know it’s going to be a good talk when the speaker uses the word “soporific” four seconds into the presentation.

One of the ideas I found interesting was language projections, providing multiple programming languages idiomatic access to the lowest level system APIs. What’s new is that these projections are not hand-crafted interfaces but software to create interfaces. The operating system calls are described in metadata that the projections use to create language-specific interfaces. This prepares the way for future growth.

In years gone by, Microsoft would first make an API available to C++, then make it available to VB in a later release. But starting with Windows 8, all languages will have immediate access to new features as they are released. Microsoft is writing projections for .NET, JavaScript, and native code. Third parties will write projections for other languages. If an obscure language has a projection, it will gain access to new runtime features at the exact same time that C# does.

Another point I found interesting is that WinRT is reusing some ideas from COM. I thought the basic ideas of COM were brilliant, though in practice COM development was painful. Near the end of the video (42:17), Martyn Lowell explains how WinRT relates to COM.

Philosophically, COM was about taking a world of different languages and using interfaces to abstract them. And we’re about taking a world of different languages [and abstracting them] so in that sense the philosophy is similar. But if you worked with COM at the time, as I did quite a lot, there were a lot of pain points. … The piece of COM that is largely unchanged in Windows Runtime is IUnknown and the marshalling and proxy system. Beyond that, almost everything you can see about the Windows Runtime is about re-imagining an interface-based system …

If COM brings back bad memories, understand that Microsoft is not saying you need to go back to writing COM, only that they’re using ideas from COM to provide you higher-level interfaces.

You can find the talk in various video and audio formats on Channel 9.

Clipboard history

The Windows clipboard only remembers the most recent thing you copied [1]. This can be very disappointing. Maybe you cut a large block of text intending to paste it somewhere, but without thinking you cut something else, and then realize you’ve just wiped out your first cut. Here are a few possible alternatives for Windows, MS Office, and Emacs.

A few years ago I tried ClipX, a free clipboard history manager for Windows, based on Jeff Atwood’s recommendation. It worked OK at first, but then I had problems with it and stopped using it. It looks like the software hasn’t been updated since 2008. If you know of a better alternative, please let me know. Also, please let me know if you have suggestions for Ubuntu.

(Update: See the comments for numerous suggestions of ClipX-like products. Also, see PHenry’s comment about clipboard history in Visual Studio.)

MS Office has its own clipboard that will store 24 entries. It stores things that have been copied either inside or outside of Office applications, but you can only paste from the Office clipboard using an Office application.

Emacs has a “kill ring” which is essentially a clipboard. It also has “registers” which are like named clipboard items.

Yesterday Emacs Knight posted a handy little bit of code for viewing your kill ring.

(global-set-key "C-cy" '(lambda ()
    (interactive) (popup-menu 'yank-menu)))

If you add this to your .emacs file, you can type ctrl-c y to pull up a little GUI menu displaying the contents of your kill ring.

[1] I’ve heard that Windows actually keeps more clipboard history, but it doesn’t provide a user interface for accessing that history. I don’t know whether this is true. But if it is, you could access the history programmatically. Maybe Office isn’t maintaining its own clipboard but just exposing more of the system clipboard.

Software knowledge shelf life

In my experience, software knowledge has a longer useful shelf life in the Unix world than in the Microsoft world. (In this post Unix is a shorthand for Unix and Linux.)

A pro-Microsoft explanation would say that Microsoft is more progressive, always improving their APIs and tools, and that Unix is stagnant.

A pro-Unix explanation would say that Unix got a lot of things right the first time, that it is more stable, and that Microsoft’s technology turn-over is more churn than progress.

Pick your explanation. But for better or worse, change comes slower on the Unix side. And when it comes, it’s less disruptive.

At least that’s how it seems to me. Although I’ve used Windows and Unix, I’ve done different kinds of work on the two platforms. Maybe the pace of change relates more to the task than the operating system. Also, I have more experience with Windows and so perhaps I’m more aware of the changes there. But nearly everything I knew about Unix 20 years ago is still useful, and much of what I knew about Windows 10 years ago is not.

Related posts:

Two-finger scrolling on Windows

One of my favorite features of Mac laptops is two-finger scrolling. This lets you scroll down a long document similar to the way the middle wheel does on a Windows mouse.

I mentioned on Twitter this evening that it would be nice if Windows had this feature. Apparently many newer Windows laptops come with this capability.  Todd Brooks replied that my laptop might be able to the two-finger scroll if I upgraded my touchpad driver. He was right. I went to the Microsoft Update site and saw that one of the optional updates was exactly what Todd suggested I install.

So far it seems that the two-finger scrolling on my ThinkPad isn’t as smooth as it is on a MacBook Pro, but it’s still nice to have.

Deleting the Windows recycle bin desktop icon

I’ve never kept many icons on my desktop, and tonight I decided to reduce the number to zero. Deleting the recycle bin icon took a little research.

Windows Vista will let you simply delete the recycle bin but other versions of Windows will not.

On Windows 7 you can right-click on the desktop, select Personalize -> Change desktop icons, and uncheck the box for the recycle bin.

On Windows XP, you can edit the registry as described here. The registry changes will take effect next time you log in. [Update: unfortunately the link is no longer available.]

If you don’t want to edit your XP registry, you can right-click on the desktop, select the Arrange Icons By menu, and uncheck the Show Desktop Icons menu. However, this will hide all icons, not just the recycle bin, and will not let you see anything you drag to the desktop until you re-check Show Desktop Icons.

If you miss the recycle bin icon, it’s still in the file explorer on the left side.

Related post: Using Windows without a mouse

Using the Windows file explorer without a mouse

The Windows File Explorer has a number of keyboard shortcuts that do not apply to Windows programs in general.

First of all, you can type Windows key-E to open the File Explorer. You can close it by typing Alt-F4.

Alt-D highlights address box. (Alt-D also highlights the address box of web browsers: IE, Firefox, Safari, etc.) F4 opens a drop-down list of folders in the address bar.

There are several numeric keypad shortcuts for expanding and collapsing folders.

  • * expands everything under the current selection
  • + expands the current selection
  • - collapses the current selection.

Note that the above keys must be on the numeric keypad; the - on the top of the main part of the keyboard, for example, has no effect on the File Explorer.

You can use the up and down arrow keys to move between files and folders.

The right arrow key expands the current selection. If the current selection is already expanded, the key takes you to the first child.

The left arrow key collapses the current selection. If the current selection is already collapsed, the key takes you to the parent folder.

F2 lets you rename an object. (For the rest of this post, “object” means “file or folder.”)

Shift-F10 opens the context menu of an object, as if you had right-clicked on the object. (There’s also a special key for this; the key has a picture of a mouse selecting something from a list.) Once you bring up the context menu, you can use the up and down arrow keys to  highlight a menu item and the enter key to click it.

Alt-Enter opens the Properties dialog for an object, as if you had right-clicked and selected its Properties from the context menu.

F6 lets you cycle between the panes of the File Explorer.

***

For daily tips on using Windows without a mouse, you can follow @SansMouse on Twitter or subscribe to its RSS feed. This post will be split into bite size pieces and added to SansMouse a few weeks from now.

Related posts:

Using Windows without a mouse

Why would you not want to use your mouse? Some tasks are most efficiently done with a mouse, but others can be done more efficiently with the keyboard.The problem isn’t so much using a mouse versus using a keyboard but rather the time it takes to switch between the two modes. Particularly when using a laptop with a touchpad, it’s faster to use the keyboard.

Why does it even matter? So what if you save a few seconds here and there? It’s a matter of keeping up with your thoughts. Suppose some series of tasks takes 20 seconds with a mouse but you can accomplish the same tasks in 12 seconds using the keyboard. The big deal isn’t that you’ve saved 8 seconds; the big deal is that you’re more likely to finish your tasks before you lose the thought that motivated them.

The same could be said for learning to type more quickly. Typing 20% faster doesn’t directly make you 20% more productive unless you’re a professional typist. The benefit is that your fingers can come closer to keeping up with your brain.

If you’d like to get in the habit of using your keyboard more and your mouse less, you may find this helpful. I’ve created a Twitter account for posting one tip per day on using Windows without a mouse. If you’d like to follow using Twitter, it’s @SansMouse. If you don’t use Twitter, you could subscribe via RSS. I’ve written a few dozen tips so far and they’re in a queue to be dribbled one per day. You could practice one simple tip per day until it is natural to use your mouse much less.

I use my mouse fairly often, though I’m trying to get into the habit of using it less. I’ve recently become persuaded that it’s worthwhile to use the keyboard more and that it doesn’t take that much effort.

Related post: Four patterns in Windows keyboard shortcuts

Four patterns in Windows keyboard shortcuts

Here are four patterns for organizing the most common keyboard shortcuts for Windows. First I’ll list the patterns, then I’ll give some qualifications and elaborate on the patterns.

  1. Keyboard shortcuts involving letters are all of the form Control-<letter> or Windows-<letter>.
  2. The letters used in Control shortcuts and Windows shortcuts don’t overlap.
  3. Control in combination with navigation keys moves the cursor. Shift in combination with navigation keys makes a selection.
  4. The Tab key cycles through things. What the key cycles through depends on what it is paired with.

When I say Control-<letter> I refer to shortcuts such as Control-C, holding down the Control key and pressing C in order to copy something. When I say Windows-<letter> I refer to holding down the Windows logo key in and pressing some letter.

My goal here is to stick to the most common shortcuts, ones that work across several versions of Windows and with many applications. Also, I’m not including any accessibility sequences such as sticky keys etc.

Control key with letters

Here are the common Windows keyboard shortcuts of the form Control key followed by a letter.

A Select all
B Toggle bold
C Copy
F Find
G Go to
H Find and replace
I Toggle italics
N New
O Open
P Print
S Save
U Toggle underlining
V Paste
W Close document
X Cut
Y Redo
Z Undo

Windows key with letters

Here are the common shortcuts using the Windows key with a letter.

D Show desktop
E Open file explorer
F Find
L Lock computer
M Minimize all windows
R Run command

Exceptions

There is one common shortcut that uses a letter and more than just the Control key or Windows key: the combination Windows-Shift-M maximizes all minimized windows. But there are no common shortcuts of the form Alt-<letter> or Control-Shift-<letter> etc.

F is the only letter used with both the Control key and the Windows key. In both cases the command finds something. Control-F finds text within a file and Windows-F searches across directories.

Navigation keys

All the navigation key shortcuts come in pairs.

Control-Home moves the cursor to the top of a document; Control-End moves the cursor to the end.

Control-Left Arrow moves the cursor to the left one word; Control-Right Arrow moves the cursor to the right one word.

Control-Up Arrow moves the cursor up a paragraph; Control-Down Arrow moves the cursor down a paragraph.

Control-Shift-Home selects from the top of the document to the cursor location; Control-Shift-End selects from the current location to the bottom of the document.

Control-Shift-Left Arrow selects one word to the left; Control-Shift-Right Arrow selects one word to the right.

Shift-Left-Arrow expands the selection one character to the left; Shift-Right-Arrow expands the selection one character to the right.

Shift-Up-Arrow selects one line up; Shift-Down-Arrow selects one line down.

Tabbing

The Tab key alone moves the focus in a window, cycling through the controls in the order specified by the application.

Control-Tab cycles through tabs or through windows in an application with multiple windows.

Alt-Tab cycles through running applications.

Windows-Tab cycles through the Task Bar.

Adding the Shift key to any of the above key reverses the cycle order. For example, Alt-Shift-Tab cycles through applications in the opposite order of Alt-Tab.

Miscellaneous shortcuts

Most common Windows keyboard shortcuts are listed above. However, there are several shortcuts that are commonly used but do not fall into a regular pattern. Some of these shortcuts are listed below.

  • Shift-F10 brings up a properties dialog, just like right-clicking.
  • Shift-Delete permanently deletes a file, bypassing the recycle bin.
  • Alt-F4 closes the active window or opens the shutdown dialog if there is no active window.
  • Alt-Down Arrow opens a drop down list box.
  • Alt-Print Screen grabs an image of the active window rather than the entire screen.
  • Alt-Space opens the current window’s system menu.
  • Windows-Pause brings up the System Properties dialog.

Most of the function keys are not used often. The most commonly used function keys are

  • F1 to bring up help,
  • F5 to refresh, and
  • F10 to activate an application’s menu bar.

Related posts:

PowerShell 2.0 for Windows XP etc.

PowerShell version 2.0 shipped with Windows 7 and with Windows Server 2008 R2, but it only recently became available for other versions of Windows.

The release of PowerShell 2.0 has been more like a leak than a product launch. The announcement page hardly reads like an announcement. The title reads “Description of the Windows Management Framework on Windows XP, Windows Server 2003, Windows Vista, and Windows Server 2008.” What’s this “Windows Management Framework”? I’ve never heard of that. I just want the new PowerShell. The first time I saw this page was when someone sent me a link saying PowerShell 2.0 was available for XP. I thought they’d sent me the wrong link by mistake because I didn’t see anything about PowerShell at first. Only if you scroll down to the middle of a long page can you see links to download PowerShell.

I expected something more like the following.

PowerShell 2.0 Released

Download for your platform:

Related links:

Simple backup software

I was asking about backup software for Windows the other day and a couple people recommended Cobian Backup. It’s simple to use, but also very configurable. And it’s free.

You can have the software simply copy files or you can have it zip the output (.zip or .7z format). In either case, you don’t need the backup software in order to restore your files.

The software has all the features you’d expect. You can perform full, incremental, or differential backups. You can run backups manually or as scheduled tasks. Etc.

Adding fonts to the PowerShell and cmd.exe consoles

The default font options for the PowerShell console are limited: raster fonts and Lucida Console. Raster fonts are the default, though Lucida Console is an improvement. In my opinion, Consolas is even better, but it’s not on the list of options.

Mastering PowerShell by Tobias Weltner explains how to expand the list of font options for the PowerShell console. The same trick increases the list of font options in the Windows command prompt cmd.exe as well. The book is free for download. See page 16 for details. However, I have two comments about the instructions it gives.

First, the book says “The name must be exactly the same as the official font name, just the way it’s stated under [registry key].” However, the Consolas font is listed in the registry as “Consolas (True Type)”. You should enter “Consolas” and leave out the parenthetical description.

Second, the book says “the new font will work only after you either log off at least once or restart your computer.” When I tried it, logging off was not sufficient; I had to reboot my computer before the font change would work.

Update: In order to make this post self-contained, I’ve added below the necessary information from Mastering PowerShell.

Run regedit.exe and navigate to HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindows NTCurrentVersionConsoleTrueTypeFont.

Right-click in the panel on the right side and create a new string value. Name that value “0” or “00” or however many zeros you need to create a new key. That string’s value is the name of the font to add.

Update: See Necessary criteria for fonts to be available in a command window

Related posts:

Improved PowerShell prompt
A couple thoughts on typography
Better R console fonts

Numerical computing in IronPython with Ironclad

In a previous post, I discuss my difficulties calling some Python modules from IronPython. In particular I wanted to call SciPy from IronPython and couldn’t. The discussion following that post brought up Ironclad as a possible solution. I wanted to learn more about Ironclad, and so I invited William Reade to write a guest post about the project. I want to thank William for responding to my request with a very helpful article. — John


Hi! My name’s William Reade, and I’ve spent the last year or so working on Ironclad, an open-source project which helps IronPython to inter-operate better with CPython. Michael Foord recently introduced  me to our host John, who kindly offered me the opportunity to write a bit about  my work and, er, how well it works. So, here I am.

To give you a little bit of context, I’ve been working at Resolver Systems for several years now; our main product, Resolver  One, is a spreadsheet with very tight IronPython integration. We like to describe  it as a “Pythonic spreadsheet”, and that’s clearly a concept that people like.  However, when people think of a “Pythonic spreadsheet”, they apparently expect it  to work with popular Python libraries — such as NumPy and SciPy — and we found that IronPython’s incompatibility put us at a serious disadvantage. And, for some reason, nobody seemed very keen to  solve the problem for us, so we had to do it ourselves.

The purpose of Ironclad is to allow you to use Python C extensions (of which there are many) from inside IronPython without recompiling anything. The secret purpose  has always been to get NumPy working in Resolver One, and in release 1.4 we finally  achieved this goal. Although the integration is still alpha level, you can import  and use NumPy inside the spreadsheet grid and user code: you can see a screencast  about the integration here.

However, while Resolver One is a great tool, you aren’t required to use it to get the benefits: Ironclad has been developed completely separately, has no external  dependencies, and is available under an open source license. If you consider  yourself adequately teased, keep reading for a discussion of what Ironclad actually  does, what it enables you to do, and where it’s headed.

As you may know, Python is written in C and IronPython is written in C#. While IronPython is an excellent implementation of Python, it works very differently  under the hood, and it certainly doesn’t have anything resembling Python’s API for  writing C extensions. However, Ironclad can work around this problem by loading a  stub DLL into an IronPython process which impersonates the real python25.dll, and  hence allows us to us intercept the CPython API calls. We can then ensure that the  appropriate things happen in response to those calls … except that we use  IronPython objects instead of CPython ones.

So long as we wrap IronPython objects for consumption by CPython, and vice versa, the two systems can coexist and inter-operate quite happily. Of course, the mix of  deterministic and non-deterministic garbage collection makes it a little tricky [1] to  ensure that unreferenced objects — and only unreferenced objects — die in a  timely manner, and there are a number of other dark corners, but I’ve done enough  work to confidently state that the problem is “just” complex and fiddly. While it’s  not the sort of project that will ever be finished, it hopefully is the sort that  can be useful without being perfect.

The upshot of my recent work is that you can now download Ironclad, type ‘import ironclad; import scipy‘ in an IronPython console, and it will Just Work [2]. I am  programmer, hear me roar!

Hundreds of tests now pass in both NumPy and SciPy, and I hope that some of you will be inspired to test it against your own requirements. For example, the Gaussian error function has been mentioned a few times on this blog (and, crucially, I have a  vague idea of what it actually is), and I can demonstrate that scipy.special.erf works perfectly under Ironclad:

C:devironclad-headbuild>ipy
IronPython 2.0 (2.0.0.0) on .NET 2.0.50727.3053
Type "help", "copyright", "credits" or "license" for more information.
>>> import ironclad
>>> from scipy.special import erf
Detected scipy import
  faking out numpy._import_tools.PackageLoader
Detected numpy import
  faking out modules: mmap, nosetester, parser
>>> erf(0)
0.0
>>> erf(0.1)
0.1124629160182849
>>> erf(1)
0.84270079294971478
>>> erf(10)
1.0

Numerical integration also seems to work pretty well, even for tricky cases (note that the quad function returns a tuple of (result, error)):

>>> from scipy.integrate import quad
>>> quad(erf, 0, 1)
(0.48606495811225592, 5.3964050795968879e-015)
>>> quad(erf, -1, 1)
(0.0, 1.0746071094349994e-014)
>>> from scipy import inf
>>> quad(erf, -inf, inf)
(0.0, 0.0)
>>> quad(erf, 0, inf) # ok, this one is probably more of a 'stupid' case
Warning: The integral is probably divergent, or slowly convergent.
(-1.564189583542768, 3.2898350710297564e-010)

And, while this exposes a little import-order wart, we can re-implement erf in terms of the normal CDF, and see that we get pretty similar results:

>>> from scipy import misc # shouldn't really be necessary - sorry :)
>>> from scipy.stats.distributions import norm
>>> import numpy as np
>>> def my_erf(x):
...   y = norm.cdf(x * np.sqrt(2))
...   return (2 * y) - 1
...
>>> my_erf(0.1)
0.11246291601828484
>>> my_erf(1)
0.84270079294971501
>>> quad(my_erf, 0, 1)
(0.48606495811225597, 5.3964050795968887e-015)
>>> quad(my_erf, -inf, inf)
(2.8756927650058737e-016, 6.1925307417506635e-016)

I also know that it’s possible to run through the whole Tentative NumPy Tutorial [3] with identical output on  CPython and IronPython [4], and the SciPy tutorial appears to work equally well in both  environments [5]. In short, if you’re trying to do scientific computing with  IronPython, Ironclad is now probably mature enough to let you get significant value  out of SciPy/NumPy.

However, I can’t claim that everything is rosy: Ironclad has a number of flaws which may impact you.

  • It won’t currently work outside Windows, and it won’t work in 64-bit processes.  However, NumPy itself doesn’t yet take advantage of 64-bit Windows. I’ll start work  on this  as soon as it’s practical; for now, it should be possible to run in 32-bit  mode without problems.
  • Performance is generally poor compared to CPython. In many places it’s only a matter of a few errant microseconds — and we’ve seen NumPy integration deliver some great performance benefits for Resolver One — but in pathological cases it’s  worse by many orders of magnitude. This is another area where I would really like  to hear back from users with examples of what needs to be faster.
  • Unicode data doesn’t work, and I don’t plan to work on this problem because it’ll disappear when IronPython catches up to Python 3000. At that point both systems will have Unicode strings only, instead of the current situation where I would have to map one string type on one side to two string types on the other.
  • NumPy’s distutils and f2py subpackages don’t currently work at all, and nor do memory-mapped files.
  • Plenty of other CPython extensions work, to a greater or lesser extent, but lots won’t even import.

However, just about every problem with Ironclad is fixable, at least in theory: if you need it to do something that it can’t, please talk to me about it (or even send me a patch!).

Footnotes

[1] CPython uses reference counting to track objects’ states, and deletes them deterministically the moment they become unreferenced, while .NET uses a more advanced garbage collection strategy which unfortunately leads to non-deterministic finalization.

[2] Assuming you have the directories containing the ironclad, numpy and scipy packages already on your sys.path, at any rate. I personally just install  everything for Python 2.5, and have added the CPython install’s ‘Dlls‘ and ‘lib/site-packages‘ subdirectories to my IRONPYTHONPATH.

[3] Apart from the matplotlib/pylab bit, but even that should be workable with a little extra setup if you don’t mind using a non-interactive back-end.

[4] Modulo PRNG output, of course.

[5] That is to say, not very well at all, but at least they go wrong in similar ways.