A few years ago, we decided to try out some "NoSQL" databases. Up until this point, we had mostly used Oracle, along with a little bit of mysql. Our experience with Oracle was that it mostly worked fine, however we needed to buy bigger and more expensive servers as we built out our website and our traffic increased. Mysql was tried as a low (or essentially no) cost alternative to Oracle. We found that it definitely worked, but wasn't quite as fast or reliable as Oracle (this was more than five years ago).
So-called "NoSQL" databases were becoming popular and it seemed like something we should try. The most popular one at the time was MongoDB, so we made this our first choice.
We always like to try a "proof of concept" implementation before we commit to a technology. In this case, the application was a sort of "auto complete" for our website that, given a few keystrokes, suggested a stock symbol or mutual fund from a database of about 15,000 North American stocks or about 18,000 Canadian mutual funds. To this day, I am not really sure that this was the best application for MongoDB. However, in our initial tests, it appeared to be fine. I'm going to dig up the actual performance figures that we got and add them to this blog entry once I find them
Unfortunately, we had a lot of issues when we moved it into production on a two-node cluster. We seemed to lose data on some of our updates, our performance was very poor under real loads and the application appeared to be unstable. We did some research and found out that Mongo wasn't really durable, in that writes were acknowledged before they were actually written to disk. In retrospect, I think we also failed to do proper error checking. At this point, we started researching our problems and discovered a lot of disconcerting articles (a modern, although somewhat biased example of these articles would be this blog entry).
We were faced with a production application that was failing badly, which is a nightmare in which you generally look for the fastest possible solution. In this case, it turned out that our developer team had a plan B -- an implementation of our auto-complete feature using lucene. We did some quick performance tests and found the lucene version to be much faster and more reliable. So, we put it into production as fast as our promotion procedures allowed, and shut down the mongo version.
I don't want this blog entry to descend into MongoDB bashing. There is already a lot of that out there. To this day, I am not convinced that we chose an appropriate proof of concept application for MongoDB. However, we did choose an excellent one for lucene :-). What concerned us most about Mongo was that it did not seem to be durable, and we knew that global locking (a limitation which has apparently been fixed) would be an issue for many of our applications. We came to the collective conclusion that MongoDB simply wasn't for us. To be fair, a number of organizations have reached the exact opposite conclusion, and, if you are considering MongoDB, you should probably go to the 10gen website and read some of the success stories. If you are interested in a more cynical (but very funny) take, try this video.
We continue to use the lucene solution to this day and it has worked well for us. In fact, we are now extensively using solr (which is based on lucene but adds a convenient clustering and web services api around it). It turns out that solr is sort of a "NoSQL" solution in its own right, which is something I will cover in future blog entries.
Having decided to move on from MongoDB, we started looking at other "NoSQL" solutions. More on that in future blog entries.
No comments:
Post a Comment