Did I mention #MongoDb is fast?!?! Way to go @mongodb

So now that I've got a bunch of data in the database, I figured might as well see what queries looked like. Let me just say WOW. The details speak pretty much for themselves, but here's a little background on the tests.

I setup MySQL, MongoDb, and CouchDb on one virtual machine. I ran a script to insert 100,000 documents I pulled down from wikipedia. Mysql had a single table with 4 columns. MongoDb and CouchDb were schema-less, but their docs had the exact same four properties. For MongoDb and Mysql I did a query for the individual document id's. For CouchDb I directly selected the document via their CouchDb id. In both mysql and MongoDb I indexed the id columns. Then it was time to do 100,000 queries.

+-------------------------------------------------------------------------+
|                         100000 Indexed Queries                          |
+-------------------------------------------------------------------------+
|           | Average   | Median    | Deviation | Lap Total  | Total      |
+-------------------------------------------------------------------------+
| MongoDb   | 0.00025   | 0.00021   | 0.00019   | 24.99955   | 291.38463  |
| CouchDb   | 0.01098   | 0.01067   | 0.00972   | 1097.85109 | 1101.09003 |
| MysqlDb   | 0.00199   | 0.00032   | 0.00518   | 199.32546  | 201.88513  |
+-------------------------------------------------------------------------+

Let me just say WOW!!!! MongoDb is ridiculously fast. I mean really, come on already, that's crazy. When looking at the totals, the "Lap Total" will be the one you want to look at. That is the total of just the individual queries. The "Total" time spent includes the time doing other things inside the code. In the example above during the Mongo test i was also writing to memcache for the next test.

Now if you are really ready to be impressed, take a look at this. I dropped the indexes in both MongoDb and Mysql. I expected these to be a good bit slower, so I did a much smaller result set. I really don't know what to say to these numbers.

+-----------------------------------------------------------------------+
|                      100 Non-Indexed Queries                          |
+-----------------------------------------------------------------------+
|           | Average   | Median    | Deviation | Lap Total | Total     |
+-----------------------------------------------------------------------+
| MongoDb   | 0.05662   | 0.03704   | 0.19267   | 5.66164   | 5.95255   |
| CouchDb   | 0.04619   | 0.03035   | 0.04323   | 4.61948   | 4.62373   |
| MysqlDb   | 1.99975   | 1.68468   | 1.99266   | 199.97469 | 199.97834 |
+-----------------------------------------------------------------------+

Last but not least, I did one more test. This time I pre-populated memcache with the records, so I could compare really fast speeds with a known really fast storage medium. At this point I'm going to stop typing because I can only say "fast" so many times.

+-----------------------------------------------------------------------+
|                             1000 Queries                              |
+-----------------------------------------------------------------------+
|           | Average   | Median    | Deviation | Lap Total | Total     |
+-----------------------------------------------------------------------+
| MongoDb   | 0.00035   | 0.00032   | 0.00022   | 0.35469   | 2.77333   |
| CouchDb   | 0.0164    | 0.01495   | 0.02076   | 16.39814  | 16.42793  |
| MysqlDb   | 0.00772   | 0.00826   | 0.00557   | 7.71744   | 7.73784   |
| Memcached | 0.00015   | 8.0E-5    | 0.00026   | 0.15356   | 1.9688    |
+-----------------------------------------------------------------------+

Loading mentions Retweet

Jan 23, 2010

metagoto said...

Interesting. Which client or driver did you use for each db ?

Jan 24, 2010

Ben Brown said...

All the tests were done using php and its associated drivers.

---
My spelling is perfect, my thumbs are not. Sent from my blackberry.

Feb 07, 2010

johnwyles said...

Love the benchmarks (results closely match my own). Would love to hear what else you find along the way evaluating #MongoDB

rb2k said...

Thanks for doing the comparison! I always wondered how fast it actually was compared to "regular" mysql.
Did you play with query speeds so far? e.g. a regular "select * from users where name like 'bla' " compared to a mongodb query. would be really interested in those results

Feb 08, 2010

Yes I have done some experimenting with like vs regex. Although I don't have specific metrics, I will say mongo is once again fast. All of our tests have been outstanding.

jchris said...

The problem with benchmarks like this is that they are extremely unrealistic. Most of the time when a database fails in production it is due to concurrent load (lots of simultaneous users). Single-threaded single-client performance is easy to benchmark but doesn't tell you much about real-world reliability.

These tests aren't trying to measure anything other than speed. I have run these tests across concurrent connections and the results are pretty similar.

If you are doing a full evaluation you should setup master slaves shards replicas etc. This is simply meant to be comparison of speeds between a few storage mediums. You shouldn't base your selection of production software in this one blg post.

Feb 23, 2010

Josh VanderLinden said...

Very nice. Thanks for your benchmarks. I've been curious about the performance between CouchDB, MongoDB, and MySQL for quite some time, and I'm finally getting into "research" mode.

Mar 31, 2010

scottgranneman said...

It appears your total for MongoDB in the table for 100000 Indexed Queries is incorrect. Is it?

Yep, I mention that in the actual right up.

'When looking at the totals, the "Lap Total" will be the one you want to look at. That is the total of just the individual queries. The "Total" time spent includes the time doing other things inside the code. In the example above during the Mongo test i was also writing to memcache for the next test.'

May 03, 2010

ha17 said...

I thought couchdb did do indexing, just not the way you'd do it in mysql. You sure you gave them a fair shake? I don't use couchdb, but I read about it and their indexing is a bit different, but it would probably improve their numbers.

Jul 11, 2010