Comments on: ACID versus BASE for database transactions http://www.johndcook.com/blog/2009/07/06/brewer-cap-theorem-base/ The blog of John D. Cook Tue, 16 Mar 2010 04:10:08 -0400 http://wordpress.org/?v=2.8.4 hourly 1 By: Mike Swaim http://www.johndcook.com/blog/2009/07/06/brewer-cap-theorem-base/comment-page-1/#comment-20734 Mike Swaim Tue, 07 Jul 2009 12:59:40 +0000 http://www.johndcook.com/blog/?p=2607#comment-20734 I'd like to point out that this problem was solved 30+ years ago. Airlines need ACID, and run really high volume systems. The big difference is that they're not using SQL databases. AFAIK, attempts to duplicate the SABER system using a SQL database failed. (Besides SQL databases and object databases, there are heirarchical databases. MUMPS, which is widely used in the healthcare industry uses a heirarchical database in its default implementation, I believe.) I’d like to point out that this problem was solved 30+ years ago. Airlines need ACID, and run really high volume systems. The big difference is that they’re not using SQL databases. AFAIK, attempts to duplicate the SABER system using a SQL database failed.
(Besides SQL databases and object databases, there are heirarchical databases. MUMPS, which is widely used in the healthcare industry uses a heirarchical database in its default implementation, I believe.)

]]> By: Wedge http://www.johndcook.com/blog/2009/07/06/brewer-cap-theorem-base/comment-page-1/#comment-20729 Wedge Tue, 07 Jul 2009 08:51:08 +0000 http://www.johndcook.com/blog/?p=2607#comment-20729 It's interesting how today there is a lot of debate about the comparative advantages of functional programming vs. mutable-state systems (especially in distributed or multi-threaded environments), and there is also this long standing object vs. relational impedance mismatch problem. Yet it seems like not a lot of ideas have permeated the mostly arbitrary boundary between "programming" and "databases" (I suppose this is a testament to the success of the relational database model up to this point). We don't even have very good terminology to talk about non-relational databases, and there doesn't seem to be much discussion about functional programming ideas (especially immutable data) applied to databases. I think we're on the right track with versioned records and some of the distributed source control systems out there. CouchDB, big table, git, mercurial, etc, are all good experiments that I think are steps in the right direction. But you can see how limited and immature these ideas are compared to the richness and maturity of the relational db ecosystem. Ultimately I think the peak of the relational database era is on the horizon (in the next 5-10 years) and we'll see "databases" founded on different models rise and become dominant (due to the fundamental scalability problems of relational data more than anything). Nevertheless, I don't think anyone today has yet found the right model/featureset that will form the core of the SQL killer data systems of the future. It’s interesting how today there is a lot of debate about the comparative advantages of functional programming vs. mutable-state systems (especially in distributed or multi-threaded environments), and there is also this long standing object vs. relational impedance mismatch problem. Yet it seems like not a lot of ideas have permeated the mostly arbitrary boundary between “programming” and “databases” (I suppose this is a testament to the success of the relational database model up to this point). We don’t even have very good terminology to talk about non-relational databases, and there doesn’t seem to be much discussion about functional programming ideas (especially immutable data) applied to databases. I think we’re on the right track with versioned records and some of the distributed source control systems out there. CouchDB, big table, git, mercurial, etc, are all good experiments that I think are steps in the right direction. But you can see how limited and immature these ideas are compared to the richness and maturity of the relational db ecosystem.

Ultimately I think the peak of the relational database era is on the horizon (in the next 5-10 years) and we’ll see “databases” founded on different models rise and become dominant (due to the fundamental scalability problems of relational data more than anything). Nevertheless, I don’t think anyone today has yet found the right model/featureset that will form the core of the SQL killer data systems of the future.

]]> By: Daniel Lemire http://www.johndcook.com/blog/2009/07/06/brewer-cap-theorem-base/comment-page-1/#comment-20697 Daniel Lemire Mon, 06 Jul 2009 16:51:44 +0000 http://www.johndcook.com/blog/?p=2607#comment-20697 One clean way to deal with some of these problems is through Resource Versioning (ETags): http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning The trick is this: the data you get might be stale, but because it is tagged with a version number, it is not inconsistent with anybody's else version. If each of us got some data about the number of remaining books, and our numbers disagree, then whoever has the latest version number knows he has the right version. In this universe, data is immutable. That is, you get values as... "inventory of book x at revision number y was z" This is a fact. All servers will agree on it, no matter what. Compare with the simpler "inventory of book x is z" This can be inconsistent from server to server. I think that trying to build scalable systems without etags is crazy. Of course, one central authority must grant these version numbers. Someone, somewhere, must have the definitive say on the number of books remaining. But with a physical resource like books, that's unavoidable, isn't it? Moreover, you can partition the authority. For example, one server is in charge of all of the books starting with the letter A. (Better yet, using consistent hashing... http://michaelnielsen.org/blog/?p=613) BTW I don't think it happens often that Amazon sells the same book twice. If these types of errors happen due to database errors, I'd like to know about it. One clean way to deal with some of these problems is through Resource Versioning (ETags):

http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning

The trick is this: the data you get might be stale, but because it is tagged with a version number, it is not inconsistent with anybody’s else version. If each of us got some data about the number of remaining books, and our numbers disagree, then whoever has the latest version number knows he has the right version.

In this universe, data is immutable. That is, you get values as…

“inventory of book x at revision number y was z”

This is a fact. All servers will agree on it, no matter what. Compare with the simpler

“inventory of book x is z”

This can be inconsistent from server to server.

I think that trying to build scalable systems without etags is crazy.

Of course, one central authority must grant these version numbers. Someone, somewhere, must have the definitive say on the number of books remaining. But with a physical resource like books, that’s unavoidable, isn’t it?

Moreover, you can partition the authority. For example, one server is in charge of all of the books starting with the letter A. (Better yet, using consistent hashing… http://michaelnielsen.org/blog/?p=613)

BTW I don’t think it happens often that Amazon sells the same book twice. If these types of errors happen due to database errors, I’d like to know about it.

]]>