The Blog: Engineering

Oh right, it’s 2012. Better late than never. In addition to the numerous contributions we make to existing open source projects, we also find the time to release libraries or projects that we extract from our applications. In 2011, those 22 projects were:

• leaderboard – https://github.com/agoragames/leaderboard - Leaderboards backed by Redis in Ruby – Primary author: David Czarnecki (@CzarneckiD)
• java-leaderboard – https://github.com/agoragames/java-leaderboard - Leaderboards backed by Redis in Java – Primary author: David Czarnecki (@CzarneckiD)
• scala-leaderboard – https://github.com/agoragames/scala-leaderboard – Leaderboards backed by Redis in Scala – Primary author: David Czarnecki (@CzarneckiD)
• php-leaderboard – https://github.com/agoragames/php-leaderboard – Leaderboards backed by Redis in PHP – Primary author: David Czarnecki (@CzarneckiD)
• python-leaderboard – https://github.com/agoragames/python-leaderboard - Leaderboards backed by Redis in Python – Primary author: Ola Mork (@olamork)
• clojure-leaderboard – https://github.com/agoragames/clojure-leaderboard – Leaderboards backed by Redis in Clojure – Primary author: Andrew Andkjar (@andkjar)
• erlang-leaderboard – https://github.com/agoragames/erlang-leaderboard – Leaderboards backed by Redis in Erlang – Primary author: Armando DiCianno (@greymouser)
• factory-worker – https://github.com/agoragames/factory-worker – Factories for NodeJS – Primary author: Andrew Nordman (@cadwallion)
• stache – https://github.com/agoragames/stache – A Rails 3.x (yes, even Rails 3.1) compatible Mustache Template Handler, with support for partials and a couple extra niceties to make sharing the raw templates with client-side javascript a little easier – Primary author: Matthew Wilson (@hypomodern)
• py-event – https://github.com/agoragames/pyevent – Python extension module for Niels Provos’ libevent – Primary author: Aaron Westendorf (@WashUffize)
• chai – https://github.com/agoragames/chai – Chai provides a very easy to use api for mocking/stubbing your python objects, patterned after the Mocha library for Ruby – Primary author: Aaron Westendorf (@WashUffize)
• activity_feed – https://github.com/agoragames/activity_feed – Activity feeds backed by Redis – Primary author: David Czarnecki (@CzarneckiD)
• improved_logging – https://github.com/agoragames/improved_logging – Adds improved logging capabilities to the ActiveSupport::BufferedLogger class – Primary author: David Czarnecki (@CzarneckiD)
• haigha – https://github.com/agoragames/haigha – AMQP libevent Python client – Primary author: Aaron Westendorf (@WashUffize)
• py-eventsocket – https://github.com/agoragames/py-eventsocket – Socket wrapper for libevent TCP applications – Primary author: Aaron Westendorf (@WashUffize)
• errship – https://github.com/agoragames/errship – Errship is a Rails 3.1 engine for rendering error pages inside your layout. It supports i18n, custom exceptions, and Airbrake (Hoptoad) error tracking – Primary author: Logan Koester (@logankoester)
• bettertabs – https://github.com/agoragames/bettertabs – Bettertabs is a simple Rails 3.1 Engine that includes a helper and a jquery plugin to render the needed markup and javascript for a section with tabs in a easy and declarative way – Primary author: Mario Izquierdo (@tothemario)
• seedlings – https://github.com/agoragames/seedlings – Make seeding data easier to deal with across multiple ORMs – Primary author: Matthew Wilson (@hypomodern)
• tassadar – https://github.com/agoragames/tassadar – Starcraft 2 replay parser written in pure Ruby – Primary author: Matt Pruitt (@guitsaru)
• mm_sortable_item – https://github.com/agoragames/mm_sortable_item – MongoMapper plugin that provides some basic acts-as-list style functionality on Mongo documents – Primary author: Matthew Wilson (@hypomodern)
• soonatra – https://github.com/agoragames/soonatra – Sinatra application to show a “Coming Soon” page and collect emails – Primary author: Logan Koester (@logankoester)
• javascripto – https://github.com/agoragames/javascripto – Javascripto helps organize and deliver client-side javascript – Primary author: Blake Taylor (@blakefrost)

We hope you’ve found or will find these libraries useful in your own projects.

When optimizing code, you profile it, find out where it’s spending most of it’s time, and figure out how to make it spend less time there. As a sysadmin, I tend to wear a lot of hats, but the one I wear the most by far is the DBA hat–I spend far more time hacking on MySQL optimizations than any particular other thing I do. So, in order to make myself more efficient, if I can find a way to streamline the time I spend with MySQL, I can reap the biggest benefits.

It turns out doing so isn’t so hard. Here are some tricks I’ve learned.

(more…)

GameBattles is one of our most popular sites, with an active user base in the millions. While we’ve dealt with sites in that scale before, this particular case has been no walk in the park. While we’re a Rails shop, the fact that the site is written in PHP actually has nothing to due with the difficulties we’ve encountered. The real problems are that the codebase is almost a decade old, has been developed by dozens of developers over that time, has grown organically with a clear roadmap defined only recently, and has almost never been refactored.

So, at various intervals over the last year, I (and others) have been digging into the site and trying to clean up what we can. It seemed that the process of what weve been doing is interesting, but talking about it at any length has been something I’ve hesitated to do, as I don’t like talking about something without quantifiable statistics. But, the fact of the matter is, programming is an art, not a science, and anyone who says different is selling something. And the real trick with refactoring is that it doesn’t meet any immediate business goals: it’s purpose is entirely human, as it entirely exists to make a clean and sane working environment. All of the tangible business gains (security, performance, developer velocity, etc.) are all secondary, and so unless your managers “get it,” it can be a hard thing to argue for.

In our case, it’s no secret that GameBattles’ stability was going downhill, and so we needed to do something. And the only way for us to be able to audit for problems was to make the codebase manageable.

(more…)

Here at MLG, we use a product from Opscode called Chef to manage our server environments. Chef allows us to reliably manage our infrastructure by providing us with the ability to write code that describes how a server should be configured. While not perfect, it has served us well.

(more…)

If you’ve ever wanted to include some data from an RSS/Atom feed on a page from Ruby, you’ve probably run into this problem:

It’s completely unreasonable to hammer the feed provider with a request every time the page is rendered, but it’s no more reasonable to use a big relational database like MySQL just to cache some stupid tweets or blog posts, especially if your application wouldn’t otherwise need one.

But apparently this layer of persistance has been deemed out-of-scope by the creators of popular RSS consumer libraries such as Feedzirra.

So I wrote a little gem called feedzirra-redis to take care of this as transparently as possible. You can use it just like you would Feedzirra normally, but your feeds and entries do not disappear at the end of the script, so you can do the dirty work from a Rakefile via cron instead of in the web request/response cycle while your users impatiently twiddle their thumbs.

Installation

$ gem install feedzirra-redis

See the README and test suite for example usage.

Enjoy!

Here at Agora we make use of dedicated hardware and virtual machines running on our providers’ respective clouds. In recent months, we’ve moved our RabbitMQ hosts onto hardware because we found that we could far exceed the CPU capacity of our virtual machines and it was far cheaper to run a small cluster of hardware hosts than a giant cluster of VMs. We used an existing, underutilized host for our primary traffic while awaiting delivery and installation of a new pair servers. Expecting a simple plug-and-play swap, I set out to test the new hardware before we made the transition. What follows is a harrowing tale into the deepest depths of modern hardware architecture.

(more…)

We’re pleased to announce the official release of haigha, our Python AMQP client library.

(more…)

My colleague @logankoester posed the following question in our team chat room: “Can the Github bot notify HipChat on wiki updates as well? I mean, wikis are just git repos, right? I am equally interested in documentation changes as in software changes.”

It is possible to do this in a little bit of a roundabout way. Read on to see how I did this with Hudson, our Continuous Integration server.

(more…)

Do you use points or do you use pixels? I’ve fallen in love with points. If you know me, you might already know this because I talk about them all the time. So what is it that I like about points, or perhaps, what is it that I dislike about pixels?

(more…)