Rumpus simulates the grid-worlds common in Reinforcement Learning (RL) and allows client programs to interact with it over TCP sockets. Worlds are represented graphically (gifs) to make it is easy to define new learning problems. Socket based interaction means that users can develop clients that implement their favourite RL algorithms using their language of choice as long as the language supports sockets.

To facilitate the development of clients, the server comes with client libraries in C and Ruby. These libraries hide the details of the socket handling and the grid-world communication protocol. By including these libraries in their clients, developers can interact with the server through high-level procedures such as go(direction) and westOpen?. Examples of complete clients are also provided.

Released under the GNU General Public License, all code from Rumpus is free to use, distribute and modify. All contributions and feedback are greatly appreciated.

News:

  • 16/08/2006 : Released version 2.0.0 (alpha). This version contains major new features including:
    • An interactive client that lets a user guide a robot through a grid-world
    • Gifs rather than pngs for graphical world representation to avoid dependance in the imlib2 library. The commercial restrictions on gifs are now history it seems. This change introduces a dependence on tcl/tk, but a tcl/tk interface comes as standard with Ruby and tcl/tk is readily available for both Linux and Windows.
    • A number of command line options for different output levels and simulation modes.
    • A new world interpretation mode which allows 'thin' walls, i.e., walls between grid squares. Thin walls is common in RL problems, e.g., in Dietterich's Taxi world. This new mode is called 'walled' and is specified in the world file. The traditional mode is called 'occuption' mode and is the default.
    • World files are now in xml format, allowing automated world file parsing using the standard Ruby xml parser, REXML.
    • 3d worlds where each level is represented by a traditional 2d image. 3d world are also common in RL problems, e.g., Hengst's 10 story building. The server always runs in 3d mode by default, but the default client library rubyrumpus hides the extra details, providing backward compatibility for existing clients. A new client library, rubyrumpus3d, provides developers with full access to the new state and command structure through methods such as up!.
    • Restart and server-shutdown commands that give clients more control over the server.
    This is a 'quick and dirty' release as I now have a student helping me in my work(!), hence the alpha state.
  • 15/10/2005 : After a lot of work on rumpus I have decided to make a new majorm release rather that a bugfix release. It is not clear yet whether I will have to break backward compatibility or not (2.0.0 or 1.2.0). New features are: an interactive client, shutdown commands for clients and the server, a restart command for running multiple trials without disconnecting from the server, new representation of reward in the state message(this is the one that might break compatibility).
  • 06/02/2005 : Released version 1.1.0 which adds the possibility of unique state identifiers and noisy actions. This version also contains a C client library named crumpus.
  • 04/26/2004 : Released version 1.0.1 which reduces the level of output produced and fixes a bug in the 'blocked?(dir)' method in the rubyrumpus client library.
  • 04/21/2004 : Relased Rumpus version 1.0.0 (beta) and announced it on the comp.ai newsgroup and the MLnet mailing list.
  • 04/06/2004 : Moved Rumpus to RubyForge, still in pre-alpha stage, no releases.
Note! The logo is taken from Sutton, Precup and Singh's paper 'Between MDPs ans Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning' published in Artificial Intelligence, 112:181-211, 1999.