Tyler Streeter
verve project banner



Started development Fall 2004


Verve is an open source C++ software library for general purpose reinforcement learning. Its main component is an intelligent agent, with user-defined sensors, actions, and motivation (rewards and punishment). It is useful for situations where you need an intelligent control system to learn a complex control task, but you don't know the exact control algorithm. Simply by providing the agent with a scalar reinforcement signal as it explores the task, it will learn to optimize its behavior to achieve more rewards, thus learning the desired control policy. It combines several well-studied methods (radial basis functions, temporal difference learning, and planning) with a few experimental methods (uncertainty estimations and curiosity) into a novel learning architecture. See my master's thesis for more details.

Most of the images and videos below are from several physical simulations I wrote for my master's thesis. They test the library's capabilities on motor learning tasks. The 3rd video below, which was done after my thesis work, is a simulated playground for curious robot exploration.





The cart-pole/inverted pendulum reinforcement learning task A simulated playground for curious robot exploration A simulated robot with a green curiosity indicator Visualization of neural network-based value function and control policy, before and after learning
Visual representation of the value function over the 2D state space for the pendulum swing-up task, shown as a sequence over time as learning progresses Plot of learning performance on the pendulum swing-up task Plots of learning performance on an exploration task comparing various learning algorithms Poster for the 2005 Iowa State HCI Forum