Tuesday, December 16, 2014

Configuring colbertix on EC2

With Stephen Colbert scheduled to take over for David Letterman as the host of The Late Show soon and The Colbert Report winding down to its conclusion in a few days, I was not surprised when someone contacted me regarding colbertix, my program to automatically get tickets to The Colbert Report.

The user was attempting to configure and run the program, but having some problems. After a few rounds of bouncing emails back and forth it became evident that we were not converging on a solution quickly. Because the user sent me their configuration file and because it is the only input the program requires, I offered to configure an instance for them on Amazon EC2. I was successful and as as result the user received two tickets to one of the show's final recordings. The following describes the steps I used to run the program on EC2, which uses Chromium and Selenium with a virtual X server on Ubuntu 14.04 LTS. The micro instances are free of charge, as is the colbertix program so this allows you to get tickets to one of the final Colbert Report shows for free.

Configuring the instance


To start, I created a free-tier para-virtualized micro instance based on Ubuntu 14.04 LTS:


Then, I chose a micro instance:


I left the instance configuration at the default settings:


I also left the storage settings at the default:


I then named my instance with a memorable name:


I then configured a custom security group that would only allow SSH connections from my network address:


The final step was then just to launch the instance:


Connecting to the instance over SSH


In order to complete the installation and run the program, I needed to connect to the instance. To do this, I first needed to generate an SSH key pair. To do this, I clicked the Connect button after selecting my newly created instance:


This shows a popup that asks to use an existing or create a new key pair. Since I do not have an existing key pair, I entered a name for a new one and pushed the Download Key Pair button:


Once this is done, another popup shows that describes how to use the downloaded key pair to connect to the instance over SSH. 


After performing the steps described, I was able to connect to the instance and now could proceed with the steps for installing and running colbertix.

Installing required packages


Before attempting the colbertix installation, I needed to install a few requisite packages. The packages were:
  • python-pip - the python package installer
  • git - required to clone the colbertix project from github
  • chromium-browser - the open source web browser, required for interacting with the website
  • chromium-chromedriver - the driver required to programatically control the browser
  • xvfb - the X virtual framebuffer, required to run graphical applications without a display
To do this, I issued the following two commands at the instance's command prompt:


Fixing Chromium's Chromedriver


After the required packages were installed, I had to fix the installation of chromium-chromedriver. The shared libraries for chromium need to be made available to the chromedriver, or it fails with an error. To do this, you simply need to perform the following:


Install and verify colbertix installation


Before configuring and running colbertix, I ran the automated test suite to ensure that there weren't any problems with the environment.


Configure and run colbertix


After seeing that all tests passed, I knew I could configure and run the program. To configure the program, I simply copied the example config file, config.ini-example, to config.ini in the same directory.

You should configure all values, even repeating the phone number if necessary, to ensure the program will be able to register successfully. If you don't have blackout dates, you can set the bad_dates: option to be empty, but do not remove the option name. When you are finished editing, config.ini should look something like the following:


Once you have a valid config file, you can start the colbertix bot. You will want to be able to disconnect from your instance and leave it running. To do this, I used screen like so:

Once you have done this, the colbertix program will attempt to register for the first available set of tickets that matches your configuration. If it succeeds, you will receive an email from the Colbert Report registration system asking you to confirm. If for some reason the registration fails, a screenshot will be saved to the project directory so you can inspect the screen.

When you are finished and have your tickets, you can go into the EC2 console and terminate the instance.

Good luck getting tickets and enjoy the show!