Why SQL failed

SQL (Structured Query Language) has been enormously successful as a tool for programmers. However, it completely failed at its intended purpose: a way for non-programmers to ask questions of a database.

I don’t think any other syntax would have been more successful. SQL syntax can be a little awkward, but that’s not the reason the general public doesn’t write SQL queries. The barrier to general use is that few people can form a moderately complex query precisely. Doing so requires thinking in terms of sets and logic. The difficulty is intrinsic to the problem; it’s not a problem of syntax.

I think about SQL every time I hear someone talking about how programmers are going to go away and that users will write their own software. And learning SQL is a pretty small task compared to learning everything else you need to know to develop software.

I also think about SQL when I hear someone talk about writing the ultimate report generator. Such a generator would be equivalent to SQL, and so it would take a programmer to use it. A good report generator is useful precisely because it is specialized to some context and therefore easier to use.

Update:  I don’t want to imply that you need to be a professional programmer to understand SQL or that you need years of training. But you do need an aptitude for programming. Someone with an aptitude for programming might catch on to SQL quickly. But someone without an aptitude for programming is going to require a combination of incentive, patience, and training before they’ll ever use SQL. If Google’s home page required users to enter Boolean expressions, they would have gone bankrupt long ago.

Related post: Why there will always be programmers

12 thoughts on “Why SQL failed

  1. When I first saw your title my first thought was long the lines of “probably the connection to the database”.

  2. I think it depends on what your definition of programmer is. I find that my friends who took a quarter or two of introductory computer science courses but who are by no means “programmers” are able to pick up basic SQL without too much trouble. Once they get over the initial fear of “programming,” they’re surprisingly capable of applying an Excel-like analogy to understand joins and grouping.

    I’m probably biased, but I think these days a computer science course or two should be a college requirement just like calculus is. If that took hold, I could see a day when almost anybody could use SQL.

  3. Danny, Good point. I updated my post in response to your comment. You don’t need to be a programmer, but you need to have an aptitude for programming.

  4. Most people do not need to query data in such a completely structured way often enough to justify learning SQL. If I use my family as a measure, the database gets a work out once or twice per year, updating addresses for Christmas cards.

    The largest database in our house is Quicken’s check register, and that has such intuitive search capability that SQL cannot come close to its ease of use.

    To John Venier: I laughed so hard at your response I almost cried.

  5. SQL fails because there is too much inertia around databases to address the any of the myriad issues they have.

    There are significant needs that can be met with simple tools. We don’t know how far non-programmers go in satisfying their own needs. Your comment is too generous to current query tools, SQL, and SQL databases.

    You imply that the intrinsic difficult of the problem prevents a solution. I claim that since we have not tried to solve the problem in a meaningful way, we have no idea whether it is technically possible to store and retrieve data without requiring programming skill.

    The challenge of non-programmer access to databases is:

    – How do we reduce the inertia in database technology

  6. I’ve seen nonprogrammers build pretty good reports using Access and InfoMaker. If a report isn’t too complicated, there are graphical tools that do a pretty good job of hiding the code.
    On the other hand, a friend of mine in college did a survey for a Psychology class where the people in the study had to solve a simple logic problem. For most people, the solution was either immediately obvious, or they had a really hard time understanding the solution, even when it was explained to them. So, I think that there is something to having an aptitude for programming.

  7. I’m probably biased, but I think these days a computer science course or two should be a college requirement just like calculus is. If that took hold, I could see a day when almost anybody could use SQL.

    I only wish!

    I’ve long thought some knowledge of calculus ought to be a requirement for graduation from college in any major. It is, after all, a major intellectual innovation and tool, right up there with any of the best.

    While I agree with there will always be programmers in the same manner I believe there will always be an England, I disagree with small parts of the article. In particular, we’ll never know what SQL might have become if its original intent were followed and realized rather than having been spoilt with pragmatic pseudo-declarations, and then frozen at birth by premature standardization. Surely, the SQL of today isn’t suitable for popular use except for the smallest of datasets, and the pathetic way some of the open source engines realize it. It is rapidly being marginalized by other techniques, such as large storage models in coding vehicles like R and easily accessible parallelism.

    By today’s standards SQL is terribly limited, particularly in its inability to extend the meaning of its basic operations. Sure, there can be stored procedures which wrap exogenous code, but lambda calculus it sure ain’t.

  8. … Should have mentioned that, as a practioner, the biggest problem I have with SQL is the fragility of the entire structure. In particular, when database schemas need to change, it is very difficult knowing what queries need to be updated, not in a sense of datatyping, but in the sense of having semantics which are now incorrect.

  9. These two statements strike me as contradictory:

    I don’t think any other syntax would have been more successful.

    If Google’s home page required users to enter Boolean expressions, they would have gone bankrupt long ago.

    The first implies that there is no syntax that is capable of extracting information from a database in a way that ordinary people can use, while the second describes an example of a company successfully doing just that. Most people know to use keywords to express their queries (which is in itself a DSL) and more advanced users (but not necessarily programmers or pseudo-programmers) use advanced tags like +, -, etc.

    Why can’t we build a plain-text or keywords based search engine for relational databases and call it SQL 2.0?

  10. The reason why SQL fails has nothing to do with SQL but with the dinosaur that is the relational database. Relational databases are designed to optimize input, which has the effect of choking output. Database admins are only concerned about how to most efficiently load data but in fact they should spend their time thinking about how to empower users to efficiently query data.

Leave a Reply

Your email address will not be published. Required fields are marked *