In the past few weeks of blitz.io, we’ve had our users blitz their location-aware iPhone apps, PHP sites, couch and node.js apps, Heroku deployments, JSON/XML RESTful-ness and just having fun scaling out their app in a very agile, iterative way. Still, there’s an element of having to type the cURL-ish line on the blitz bar. While this dramatically cuts down on the cost of test creation, we thought we’ll simplify this one more step and bring load and performance testing right into your app!
Archive for Testing
blitz.io: The [long] night before an iPhone app launch
We’d been helping out a blitz.io customer late into the night and here’s what we were up against. A spanking new iPhone app was going live the next day. It was expected to support 100,000 concurrent users with cool new location-aware awesomeness (can’t really talk about it, sorry!). The back-end architecture had a load balancer, multiple web servers, Fast-CGI scripts, replicated MySQL databases, memcached and yet we just couldn’t get past a few thousand concurrent users. This blog is about insights into performance tuning and bottlenecks uncovered by using blitz.io to load test their application.
Testing for Today’s Most Popular Apps…The Holy Grail of Testing?
I’ve been in the testing business for many years now and I’ve come across a lot of grandiose claims by test tool vendors with regards to features and capabilities that just sound too good to be true. And in many cases, they are.
When it comes to today’s world of smartphones and tablets and the explosive growth of web and mobile applications, it’s mind-blowing to see the sheer quantity of apps out there. If you look at the number of apps available today on just three of the leading app stores (Apple, Android and Facebook), there’s over a million applications, with tens of thousands of new ones every single month. Full Post »
Scaling it out – Building high performance mobile apps
We’ve been using blitz.io to optimize the backend server of a location-aware iPhone app (customer) that’s going to be released soon and we learned some interesting things that I thought I’ll pass on. The first key observation really is that performance optimization is a continuous, iterative and agile process. This means that the tools you use to validate performance have to be painless and friendly so you spend more time on tuning your code instead of figuring out how to run the tests. RESTful backends for iPhone and Android apps are exploding. I think we are past the era of simple utility apps like Sudoku and Chess, but instead the social element is now connecting these apps to backend RESTful servers that are expected to scale to the extreme, given the sheer number of mobile devices there are.
Attacks @ Scale
Last month we added a new space on TestCloud with thousands of known attacks as .msl templates. Some of you may recall that the secret behind the speed of test creation with Mu is in the way we can take in a wide variety of formats including .pcap, .har, curl to name a few and convert it into MuSL. So what we did was we took another look at our known attack templates and made that available as .msl templates.
blitz.io: Benchmarking view performance in CouchDB
If you are running CouchDB on an EC2 node with an EBS volume (or otherwise), it’s kinda hard to predict how the view performance will be affected by disk seeks and other criteria. Add complex reduce functions to this mix and you are flying blind. Just in my last blog, we talked about adding parameterization to blitz.io so you can easily test location-aware iPhone and Android apps. What if we could use CouchDB itself to generate fully parameterized tests so you can blitz it?
blitz.io: Load testing location-aware iPhone, Android apps
Location awareness is the hottest thing in town and most mobile apps on the iPhone and Android are adding this capability to find friends, restaurants, movies and other things within a certain proximity of where you are. If you are rolling out a new location-aware iPhone or Android app or you want to load test your existing location-aware app, this blog is for you. With the launch of blitz.io, we are making location-aware load testing as easy and fun as it gets.
blitz.io – CouchDB in production
We launched blitz.io a couple of days ago, fully powered by CouchDB. In the last couple of days, we’ve generated over 1,000,000 hits (through our load and performance testing) against various servers and transfered over 2GBytes of data in and out EC2. Unlike massive terrabyte read-only CouchDB clusters, ours is inherently write-heavy. When each user either through the UI or through the API runs a sprint or a rush, we have multiple writes to CouchDB, constantly.
Announcing blitz.io – and we are hiring!
We just launched our public beta of blitz.io. We wanted to bring the excitement and simplicity of git push for cloud deployments to load and performance testing. As a matter of fact, we think performance testing ought to be a social sport! Most load testing solutions on the cloud today are built heavy (remnants of the J2EE world), require major investment in time and resources, not to mention you have to talk to an expert to accomplish anything meaningful. By adapting common dev tools and dramatically simplifying this, we hope to make cloud performance testing incredibly easy.
The importance of controls
When doing troubleshooting, it is really handy to have an example of system working correctly, a known good. Then you can compare failed situation with the working situation and figure out what is different. Sometimes you can figure out the failure just based on theoretic principles, a theoretical “known good”, but actual data from a working case really speeds up analysis.
