Ruby on Rails: Scaling Your App for Rapid Growth

The Web Development Series is supported by Rackspace, the better way to do hosting. Learn more about Rackspace’s hosting solutions here.

Gowalla CTO and co-founder Scott Raymond knows a thing or two about Ruby — and about Cocoa, scaling, handling huge amounts of location data, mobile apps and NoSQL.

Gowalla is a Ruby shop, and as the captain of that particular ship, Raymond has had quite a bit of experience building a popular web app from the ground up using Ruby on Rails.

In a recent e-mail interview, we found out what Raymond had to say about those experiences and what he has learned from them over the past three and a half years.

Much has been made of Ruby/Rails’ perceived flaws when it comes to scaling. In your experience, what are the unique challenges of working with this language and framework and trying to scale an app for a nation-wide consumer audience?

Most of the “Rails can’t scale” noise is outdated or was misguided in the first place. In general, the question of scalability applies at the level of architecture and systems, and not really at the level of languages and frameworks.

That said, languages and frameworks do definitely have performance and efficiency characteristics that need to be considered. Ruby’s standard interpreter doesn’t have a great reputation for being fast, but as part of a larger, well-architected system, it is very rarely the bottleneck.

So, as a developer for a popular service, the challenge becomes trying to foresee which actions will be the most frequently requested, which data types will be the fastest growing, and which actions are the most performance-sensitive. In my experience, it’s harder to predict these things than you would think. I have spent many hours trying to “pre-scale” parts of the app that never became a problem, which often leads to maintenance headaches.

You guys collect and store a massive amount of location data for your app. How do you use Ruby and Rails for this type of data in these amounts?

We use a variety of storage systems for different purposes. Our biggest database is PostgreSQL — it currently stores the canonical records for most of our data, including users, spots, and checkins.

We also rely heavily on Redis for all kinds of things, like friendships, counters and queues. We use lots of memcached for ephemeral things, Varnish for HTTP caches and Solr to keep fuzzy spot searches fast.

Increasingly, we’re using Cassandra to store a lot of stream-like data — things like activity feeds and audit logs. I expect that our usage of Cassandra will grow a lot this year, and that we’ll also start relying on Hadoop to help us understand our data better.

Here is a simplified example of how we store and retrieve checkins, using a custom-build timeline service, called Chronologic, which is backed by Cassandra:

When you’re not coding in Ruby, what other languages/tools do you use? Or, if you had to choose another language for Gowalla, what would it be?

When I’m coding at work these days, it’s usually on our iOS client — so Cocoa/Objective-C is where I’m spending most of my time.

But for server-side work, I see us gradually moving toward a more services-oriented architecture. We are always looking at our application today and trying to identify pieces of functionality that can be isolated into services that have their own data storage and deployment patterns. So far, all of those services are written in Ruby, but I won’t be surprised if some of them end up being written in Scala, or with Node.js, or something else. But when it comes to the central web application that powers gowalla.com, I’m very happy with it being Ruby/Rails.

Here is a section of one of the view controllers in the Gowalla iPhone application. This code handles authenticating with a third-party service.

From a fan via Twitter: Was the traditional ActiveRecord modeling enough or did you have to use an NoSQL alternative?

Every Rails app that I’ve ever worked on has had to break away from the “ActiveRecord way” at some point, for some part of the app. But it’s not an all-or-nothing question. Most of the time, the standard relational/ActiveRecord approach works perfectly well, and the convenience of following the Rails golden path is completely worthwhile. But most interesting apps will run into at least a few points where the standard tools break down, and you need to access your data differently.

Fortunately, it’s a wonderful time for alternative data stores. Just look through the Heroku add-ons page — you’ve got Redis, MySQL, CouchDB, Memcache, MongoDB, Solr and more — all available as hosted services. It’s incredibly freeing to be able to tinker with these tools without worrying about up-front installation and configuration.

The example from the second question is a prime example of this. Scaling activity streams with a relational database in fairly real time is tricky — it often breaks down when one user has millions of followers, or when one user follows millions of others. To make it work, we created a service called Chronologic that manages any kind of timeline, and exposes a relatively simple interface to the Rails application. Under the covers, Chronologic uses Cassandra for most of its storage.

What kind of gem testing do you use, if any?

We use Shoulda, Factory Girl and FlexMock for testing our Ruby/Rails code. After each commit, a local continuous integration server runs the test suite and notifies the developers via Campfire of any failures.

What side project(s) are you working on right now?

Most of my “side” projects these days are tangentially work-related — small experiments to learn about a new database or library, to try out a new web service or as a proof-of-concept for a new technique. My work directly is littered with dozens of tiny re-writes of our iPhone client, each exploring some new UI idea, networking optimization, etc.

Series supported by Rackspace

The Web Development Series is supported by Rackspace, the better way to do hosting. No more worrying about web hosting uptime. No more spending your time, energy and resources trying to stay on top of things like patching, updating, monitoring, backing up data and the like. Learn why.

More Dev & Design Resources from Mashable:

– HOW TO: Transfer Your Blog From WordPress.com to WordPress.org [VIDEO]
– A Beginner’s Guide to Integrated Development Environments
– 10 Chrome Web Apps to Check Out
– HOW TO: Make Your WordPress Blog More Like Tumblr
– 10 Tools for Getting Web Design Feedback

Image courtesy of Flickr, Robert Scoble.

More About: gowalla, node, programming, RoR, ruby, Ruby on Rails, Web Development, web development series

For more Dev & Design coverage: