blitz.io: Deploying CouchDB in the cloud, literally

March 31, 2011 by kowsik

Ever since we launched blitz.io a couple of weeks back, we’ve been thinking. Yeah, we are using CouchDB with multi-master replication to handle geo-latencies and we got five regions around the world with AWS that we can generate serious load from. Frankly though, we are not happy with our deployment. We really need to be right next to the data centers where these apps are deployed so we can instantly generate massive loads with very low latency.

Geo-spatial deployments

So blitz.io is a Heroku app (deployed in Seattle) and we got CouchDB clusters running in Virginia and California. But while regional deployments are awesome, the speed of light is a serious problem. Until we build out Tachyon networks, 2,000+miles is a long way for those packets to travel while we are trying to help our users with load and performance testing their apps. As we wrestled with this problem, we came up with a very interesting solution.

blitz.io balloons

We purpose-built a weather balloon that has both our scale engine as well as CouchDB on a ultra high-density miniature SSD. Each balloon is coated with spray-on solar cells that supply us with all-day power. Based on extensive A/B testing, we find that most people run load and performance tests during the day or on a full-moon night. So regular night times are not really a problem. The Erlang core from CouchDB along with the super high performance Mu Dynamics scale engine keeps the power consumption really low on these balloons.

The big conflict we have though, is with pesky birds that want to be part of the continuous replication. In order to prevent the birds from messing around with our in-the-cloud database, each balloon is also equipped with an bird-repellant spray. We used the document validation functions in CouchDB to automatically revert documents to the proper state in case we get the birdie.

We keep the balloon ego in check with a combination of self-inflation pumps and auto-release valves that trigger compaction as and when necessary. This way once the balloons are deployed, you don’t have to worry about them at all.

Database Security

One big problem with the blitz.io balloons is security. It’s visible to everyone, even if there’s no one around remote data centers. We take security pretty seriously and we found that the simplest solution was to secure these balloons to the top of the data-center with a high bandwidth super-conducting string. This tremendously cuts down on the no-SQL-injection problems that CouchDB doesn’t have. The dedicated connection from the data center to the balloons also means that our scale engines have very low latency to your cloud apps. This easily allows us to generate very massive load and performance tests against your app, all from the cloud.

Virtual Private Clouds

Our customers have observed that in-the-cloud blitz.io balloons sometimes tend to fly away. Tethering to the top of the data center helps, but we need to ensure that the balloons stay on top of the data center at all times and be highly available. As an add-on, we are happy to note that you can deploy our virtual private cloud, made from silica-nanite particles with magnetic shielding, on top of the balloons. This looks like a cloud and feels like a cloud, but is private and virtual. This private cloud continuously tracks and monitors the balloons and uses the telemetry data to keep the balloons from floating away. We find the most of our blitz.io customers end up upgrading to the virtual private cloud right after they deploy a few balloons.

What’s next?

We are super excited with the possibility of deploying our solution on the cloud and we are continuing to innovate very rapidly. For a limited time, we are including a Helium tank when you deploy more than 10 blitz.io balloons for the purposes of load and performance testing. For our really high-end customers (with data centers on top of mountains), we are actively working on blitz.io blimps that can go where no balloons can go. Stay tuned.

So yeah, check it out: http://blitz.io

Categories: Announcements, Blitz, Cloud, CouchDB, Performance
5,830 Comments
Bookmark and Share