Rstats

R in Action

Posted on by John

No Starch Press sent me a copy of The Art of R Programming last Fall and I wrote a review of it here. Then a couple weeks ago, Manning sent me a copy of R in Action (ISBN 1935182390). Here I’ll give a quick comparison of the two books, then focus specifically on R in Action.

Comparing R books

Norman Matloff, author of The Art of R Programming, is a statistician-turned-computer scientist. As the title may imply, Matloff’s book has more of a programmer’s perspective on R as a language.

Robert Kabacoff, author of R in Action, is a psychology professor-turned-statistical consultant. And as its title may imply, Kabacoff’s book is more about using R to analyze data. That is, the book is organized by analytical task rather than by language feature.

Many R books are organized like a statistical text. In fact, many are statistics texts, organized according to the progression of statistical theory with R code sprinkled in. R in Action is organized roughly in the order of steps one would take to analyze data, starting with importing data and ending with producing reports.

In short, The Art of R Programming is for programmers, R in Action is for data analysts, and most other R books I’ve seen are for statisticians. Of course a typical R user is to some extent a programmer, an analyst, and a statistician. But this comparison gives you some idea which book you might want to reach for depending on which hat you’re wearing at the moment. For example, I’d pick up The Art of R Programming if I had a question about interfacing R and C, but I’d pick up R in Action if I wanted to read about importing SAS data or using the ggplot2 graphics package.

R in Action

Kabacoff begins his book off with two appropriate quotes.

What is the use of a book, without pictures or conversations? — Alice, Alice in Wonderland

It’s wondrous, with treasures to satiate desires both subtle and gross; but it’s not for the timid. — Q, “Q Who?” Star Trek: The Next Generation

R in Action is filled with pictures and conversations. It is also a treasure chest of practical information.

The first third of the book concerns basic data management and graphics. This much of the book would be accessible to someone with no background in statistics. The middle third of the book is devoted to basic statistics: correlation, linear regression, etc. The final third of the book contains more advanced statistics and graphics. (I was pleased to see the book has an appendix on using Sweave and odfWeave to produce reports.)

R in Action includes practical details that I have not seen in other books on R. Perhaps this is because the book is focused on analyzing and graphing data rather than exploring the dark corners of R or rounding out statistical theory.

Kabacoff says that he wrote the book that he wishes he’d had years ago. I also wish I’d had his book years ago.

Related links

The Art of R Programming

Posted on by John

Here are my first impressions of The Art of R Programming (ISBN 1593273843). I haven’t had time to read it thoroughly, and I doubt I will any time soon. Rather than sitting on it, I wanted to get something out quickly. I may say more about the book later.

The book’s author, Norman Matloff, began his career as a statistics professor and later moved into computer science. That may explain why his book seems to be more programmer-friendly than other books I’ve seen on R.

My impression is that few people actually sit down and learn R the way they’d learn, say, Java. Most learn R in the context of learning statistics. Here’s a statistical chore, and here’s a snippet of R to carry it out. Books on R tend to follow that pattern, organized more by statistical task than by language feature. That serves statisticians well, but it’s daunting to outsiders.

Matloff’s book is organized more like a typical programming book and may be more accessible to a programmer needing to learn R. He explains some things that might require no explanation if you were learning R in the context of a statistics class.

The last four chapters would be interesting even for an experienced R programmer:

  • Debugging
  • Performance enhancement: memory and speed
  • Interfacing R to other languages
  • Parallel R

No one would be surprised to see the same chapters in a Java textbook if you replaced “R” with “Java” in the titles. But these topics are not typical in a book on R. They wouldn’t come up in a statistics class because they don’t provide any statistical functionality per se. As long as you don’t make mistakes, don’t care how long your code takes to run, and don’t need to interact with anything else, these chapters are unnecessary. But of course these chapters are quite necessary in practice.

As I mentioned up front, I haven’t read the book carefully. So I’m going out on a limb a little here, but I think this may be the book I’d recommend for someone wanting to learn R, especially for someone with more experience in programming than statistics.

Related post: R: The Good Parts

RLangTip changing hands

Posted on by John

I’ve decided to hand my Twitter account RLangTip over to the folks at Revolution Analytics starting next week. I thought it would be better to give the account to someone who is more enthusiastic about R than I am, and so I offered it to David Smith. If you’ve enjoyed RLangTip so far, I expect you’ll like it even better under new ownership.

If you’d like to continue to hear from me on Twitter, you can follow one of my 10 other daily tip accounts or my personal account.

Descriptions of these accounts are available here.

Calling C++ from R

Posted on by John

This post relates my experience with calling C++ from R by writing an R module from scratch and by the inline module.

The most reliable way to speed up R code is to take it out of R. I’ve looked at various tricks for speeding up R code and none have improved the performance of code more than a few percent. Rewriting R code in C++, however, can speed it up by a couple orders of magnitude.

Until recently, my preferred approach to speeding up an R program has been to rewrite it entirely in C++. Now I’m looking at ways to rewrite only key parts of the code because I’m working with collaborators who want to work in R as much as they can. I’ve tried two approaches. The first I’ll call the bare-knuckle approach, writing an R package starting from C++ sample code from a colleague. The second approach is using the R module inline to embed C++ code in R.

I’ve tested everything on six computers: two running Windows 7, and one each running Windows XP, Ubuntu 10.04, Ubuntu 11.04, and Mac OS X 10.6.

Working with R and C++ on Windows requires installing Rtools and adding it to your path. I have not been able to get Rtools to work on my Windows XP machine. It apparently installs successfully, but everything that requires it fails. On my Windows 7 machines I’ve been able to install R packages containing C++ source but I’ve not been able to build them.

(In R terminology, “building” a package means gathering the source into a format that R wants. This does not “build” in the sense of compiling the C++ source; the C++ compilation happens on install. So, in theory, one could build an R package on one operating system and then install it on another.)

I’ve been able to build R packages on Ubuntu and Mac OS X and install them on Windows 7, albeit with warnings. The install process complains bitterly, but apparently everything works. At least once I got an error message saying that R couldn’t find the compiler, but apparently it did since the code did in fact compile. (Rtools on Windows installs a Unix-like build system, including the gcc compiler.)

Building and installing R packages on Ubuntu has worked smoothly. I’ve been able to build R packages on OS X and install them on Ubuntu, but not always the other way around. Sometimes R packages built on Ubuntu will install on OS X and sometimes not. My Ubuntu 10.04 box runs a newer version of gcc but an older version of R compared to the Mac, so the portability problems may come from tool version differences rather than operating system differences.

The bare-knuckle approach to writing R packages involves a great deal of tedious work. The R package Rcpp eliminates much of this tedium. The R package inline makes the process even simpler. Using inline you can include C++ source code in your R file and have the code compile automatically when the R code runs. This is the simplest way to call C++ from R, when it works. You can read far more about Rcpp and inline on Dirk Eddelbuettel’s website.

I find the inline approach promising. It could be a convenient way for me to give C++ code to collaborators who may not be able to write C++ but who could read it. It would also eliminate the portability problems that could arise from building a package on one platform and installing it on another.

The inline module worked smoothly on my Ubuntu 11.04 machine, but not at all on Ubuntu 10.04. I was unable to install Rcpp or inline on Ubuntu 10.04 from R itself, though I was able to install these packages through the Synaptic Package Manager. The packages load from R, but fail when I try to use them.

I was able to get inline to work from Windows 7 and from OS X, though in both cases there are numerous warnings.

It was not immediately clear to me how to use inline to call a C++ function that depended on another C++ function. If you want to call a single C++ function f() from R, you could put its source code in an R string and pass that string as an argument to cxxfunction. But what if the function f() uses a function g()?

My first thought was to define g() inside f(). C++ doesn’t let you define a function inside a function, but it does let you define a struct inside a function, so you could make g() a method on a struct to get around this language limitation.

It turns out there’s a more direct method. In addition to the body argument for source code, the function cxxfunction also has an argument includes for header files. You could put the source of g() in a header file and pass a string with a #include statement for that header as the value of includes. However, I don’t know where inline looks for headers. Apparently it does not look in the working directory of the R process that calls it. However, you could pass the content of the header file as the value of includes rather than a #include statement. After all, the first thing the C++ compiler will do with a header file reference is replace it with the contents of the file. So you could put the source for g() or any other dependencies in a string passed as the includes argument.

Calling C++ from R has been painful, but I expect will be easier in the future now that I’ve learned a few things the hard way.

Word frequencies in human and computer languages

Posted on by John

This is one of my favorite quotes from Starbucks’ coffee cups:

When I was young I was mislead by flash cards into believing that xylophones and zebras were much more common.

Alphabet books treat every letter as equally important even though letters like X and Z are far less common than letters like E and T. Children need to learn the entire alphabet eventually, and there are only 26 letters, so teaching all the letters at once is not bad. But uniform emphasis doesn’t scale well. Learning a foreign language, or a computer language, by learning words without regard to frequency is absurd. The most common words are far more common than the less common words, and so it makes sense to learn the most common words first.

John Miles White has applied this idea to learning R. He did a keyword frequency analysis for R and showed that the frequency of the keywords follows Zipf’s law or something similar. I’d like to see someone do a similar study for other programming languages.

It would be interesting to write a programming language tutorial that introduces the keywords in the approximately the order of their frequency. Such a book might be quite unorthodox, and quite useful.

White points out that when teaching human languages in a classroom, “the usefulness of a word tends to be confounded with its respectability.” I imagine something similar happens with programming languages. Programs that produce lists of Fibonacci numbers or prime numbers are the xylophones and zebras of the software world.

Related posts

R Q&A

Posted on by John

There is an organized effort to promote the StackOverflow site for questions and answers around the R programming language. It’s working: the amount of R activity on StackOverflow has greatly increased lately.

If you’re familiar with StackOverflow but not R, you might want to take a look at the R Project website and these notes about the R language.

If you’re familiar with R but not StackOverflow, allow me to introduce you. StackOverflow is a website for questions and answers related to programming. The site is open to all programming languages and environments, but it’s pretty strict about sticking to programming questions. (StackOverflow has two sister sites for other computing questions: ServerFault for system administration and IT issues, and Superuser for almost anything else related to computing.)

I’d like to see the R community take advantage of StackOverflow’s platform. According to Metcalfe’s law, the value of a network is proportional to the square of the number of users in the network. As more people go to StackOverflow for R Q&A, everyone gets better information and faster responses.

Related posts

Programming book I wish someone would write: R, The Good Parts

Posted on by John

Douglas Crockford’s book JavaScript: The Good Parts is terrific. Crockford is both a critic of and advocate for JavaScript. He’s quite frank about the language’s faults. His book is the clearest exposition of the pitfalls of JavaScript that I’ve seen. But he also believes there’s a great language at the heart of JavaScript. He doesn’t just complain about the bad parts; he explains how to avoid them. He has identified his recommended subset of the language. He has written programming style guide intended to increase the chances that JavaScript code does what the programmer intends. And he has written a tool, JSLint, to warn of potential problems. (Crockford reminds me of Luke Skywalker, convinced that there is good in Darth Vader and determined to rescue him from the dark side of the force.)

I wish someone would write a book for R analogous to the one Crockford wrote for JavaScript.

The R language has a lot in common with JavaScript. Both are Lisp-like languages at their core with C-like syntax. Both are dominant languages in their respective niches: R in academic statistics and JavaScript in web browsers. (R doesn’t have the monopoly in statistics that JavaScript has in the browser, but it’s still pervasive.) Both languages are powerful but maddening to debug. JavaScript has an undeserved reputation for being ugly because it is typically used to program the browser DOM; it’s the DOM that’s buggy and non-standard, not JavaScript. Similarly, R’s reputation may suffer from the numerous poorly written modules implemented in R.

More R posts