The Blog: Python

Oh right, it’s 2012. Better late than never. In addition to the numerous contributions we make to existing open source projects, we also find the time to release libraries or projects that we extract from our applications. In 2011, those 22 projects were:

• leaderboard – https://github.com/agoragames/leaderboard - Leaderboards backed by Redis in Ruby – Primary author: David Czarnecki (@CzarneckiD)
• java-leaderboard – https://github.com/agoragames/java-leaderboard - Leaderboards backed by Redis in Java – Primary author: David Czarnecki (@CzarneckiD)
• scala-leaderboard – https://github.com/agoragames/scala-leaderboard – Leaderboards backed by Redis in Scala – Primary author: David Czarnecki (@CzarneckiD)
• php-leaderboard – https://github.com/agoragames/php-leaderboard – Leaderboards backed by Redis in PHP – Primary author: David Czarnecki (@CzarneckiD)
• python-leaderboard – https://github.com/agoragames/python-leaderboard - Leaderboards backed by Redis in Python – Primary author: Ola Mork (@olamork)
• clojure-leaderboard – https://github.com/agoragames/clojure-leaderboard – Leaderboards backed by Redis in Clojure – Primary author: Andrew Andkjar (@andkjar)
• erlang-leaderboard – https://github.com/agoragames/erlang-leaderboard – Leaderboards backed by Redis in Erlang – Primary author: Armando DiCianno (@greymouser)
• factory-worker – https://github.com/agoragames/factory-worker – Factories for NodeJS – Primary author: Andrew Nordman (@cadwallion)
• stache – https://github.com/agoragames/stache – A Rails 3.x (yes, even Rails 3.1) compatible Mustache Template Handler, with support for partials and a couple extra niceties to make sharing the raw templates with client-side javascript a little easier – Primary author: Matthew Wilson (@hypomodern)
• py-event – https://github.com/agoragames/pyevent – Python extension module for Niels Provos’ libevent – Primary author: Aaron Westendorf (@WashUffize)
• chai – https://github.com/agoragames/chai – Chai provides a very easy to use api for mocking/stubbing your python objects, patterned after the Mocha library for Ruby – Primary author: Aaron Westendorf (@WashUffize)
• activity_feed – https://github.com/agoragames/activity_feed – Activity feeds backed by Redis – Primary author: David Czarnecki (@CzarneckiD)
• improved_logging – https://github.com/agoragames/improved_logging – Adds improved logging capabilities to the ActiveSupport::BufferedLogger class – Primary author: David Czarnecki (@CzarneckiD)
• haigha – https://github.com/agoragames/haigha – AMQP libevent Python client – Primary author: Aaron Westendorf (@WashUffize)
• py-eventsocket – https://github.com/agoragames/py-eventsocket – Socket wrapper for libevent TCP applications – Primary author: Aaron Westendorf (@WashUffize)
• errship – https://github.com/agoragames/errship – Errship is a Rails 3.1 engine for rendering error pages inside your layout. It supports i18n, custom exceptions, and Airbrake (Hoptoad) error tracking – Primary author: Logan Koester (@logankoester)
• bettertabs – https://github.com/agoragames/bettertabs – Bettertabs is a simple Rails 3.1 Engine that includes a helper and a jquery plugin to render the needed markup and javascript for a section with tabs in a easy and declarative way – Primary author: Mario Izquierdo (@tothemario)
• seedlings – https://github.com/agoragames/seedlings – Make seeding data easier to deal with across multiple ORMs – Primary author: Matthew Wilson (@hypomodern)
• tassadar – https://github.com/agoragames/tassadar – Starcraft 2 replay parser written in pure Ruby – Primary author: Matt Pruitt (@guitsaru)
• mm_sortable_item – https://github.com/agoragames/mm_sortable_item – MongoMapper plugin that provides some basic acts-as-list style functionality on Mongo documents – Primary author: Matthew Wilson (@hypomodern)
• soonatra – https://github.com/agoragames/soonatra – Sinatra application to show a “Coming Soon” page and collect emails – Primary author: Logan Koester (@logankoester)
• javascripto – https://github.com/agoragames/javascripto – Javascripto helps organize and deliver client-side javascript – Primary author: Blake Taylor (@blakefrost)

We hope you’ve found or will find these libraries useful in your own projects.

Here at Agora we make use of dedicated hardware and virtual machines running on our providers’ respective clouds. In recent months, we’ve moved our RabbitMQ hosts onto hardware because we found that we could far exceed the CPU capacity of our virtual machines and it was far cheaper to run a small cluster of hardware hosts than a giant cluster of VMs. We used an existing, underutilized host for our primary traffic while awaiting delivery and installation of a new pair servers. Expecting a simple plug-and-play swap, I set out to test the new hardware before we made the transition. What follows is a harrowing tale into the deepest depths of modern hardware architecture.

(more…)

We’re pleased to announce the official release of haigha, our Python AMQP client library.

(more…)

Over the past year we’ve been evaluating and migrating many of our game services over to mongoDB. Its feature set gives us complete freedom to iterate with developers during game production, its performance is sufficient to power our infrastructure during peak load, our operations team appreciate its powerful administration tools, and 10gen has so far proven itself a reliable technology partner and steward of the Mongo roadmap. One of the very important features of PyMongo that we have made use of is the MasterSlaveConnection, but it’s not without its caveats, one of those being that it does not guarantee you are balancing your reads across a replica set.

(more…)

Here at Agora we take testing seriously, insisting that full test coverage always be part of a deliverable and adjusting our schedules accordingly. We are historically a Rails shop, but for a few years we have been developing an extensive Python code base and infrastructure to power our in-game offerings.

We’ve been using Mox as our mock testing solution since 2009, and though it has met all of our functional requirements, we’ve longed for the simplicity and power of Ruby’s Mocha framework. This past Friday we held our 3rd Hack-A-Thon, and Vitaly Babiy and I developed Chai, a mocking framework patterned after Mocha.

(more…)

At the core of our Hydra platform is AMQP, currently we are using the py-amqplib driver. It has worked well up to this point. We recently had a requirement to send large messages through AMQP, this is where the driver failed us. The following are couple benchmarks and fix to get around the issue.

(more…)

(more…)

I recently started integrating with some web services in Python and wanted to be able to test all the requests and responses with valid and invalid data. However, I need to be able to do this without actually hitting the web service when testing. And of course I don’t want to change my web service code to conditionally do things if running in a test environment. Enter mocking and Mox.

(more…)

One of our engineers has been doing some work with RRDTool (Round Robin Database); and more specifically PYRRD.  He’s using it to combine real time usage stats with a prediction algorithm so that we can pre-emptively allocate resources to different parts of the system.  Its real spiffy.

His additions to the PYRRD library just got commited to the trunk build, and he’s credited in the readme.

http://code.google.com/p/pyrrd/source/browse/trunk/ChangeLog
http://code.google.com/p/pyrrd/source/browse/trunk/README

Overview

Much of Agora’s work over the last few years has been done in Ruby. Recently though, we’re migrating major components to Python in order to enjoy the benefits of cross platform compatibility with Windows. The purpose of this post is to discuss testing in Python; specifically mocking.
(more…)