Optical RL-Gym

OpenAI Gym is the de-facto interface for reinforcement learning environments. Optical RL-Gym builds on top of OpenAI Gym's interfaces to create a set of environments that model optical network problems such as resource management and reconfiguration. Optical RL-Gym can be used to quickly start experimenting with reinforcement learning in optical network problems. Later, you can use the pre-defined environments to create more specific environments for your particular use case.

Please use the following bibtex:

@inproceedings{optical-rl-gym,
  title = {The {Optical RL-Gym}: an open-source toolkit for applying reinforcement learning in optical networks},
  author = {Carlos Natalino and Paolo Monti},
  booktitle = {International Conference on Transparent Optical Networks (ICTON)},
  year = {2020},
  location = {Bari, Italy},
  month = {July},
  pages = {Mo.C1.1},
  url = {https://github.com/carlosnatalino/optical-rl-gym}
}

Content of this document

1. Installation
2. Environments
3. Examples
4. Resources

Installation

You can install the Optical RL-Gym with:

git clone https://github.com/carlosnatalino/optical-rl-gym.git
cd optical-rl-gym
pip install -e .

You will be able to run the examples right away.

You can see the dependencies in the setup.py file.

To traing reinforcement learning agents, you must create or install reinforcement learning agents. Here are some of the libraries containing RL agents:

• Stable baselines
• OpenAI Baselines -- in maintenance mode
• ChainerRL
• TensorFlow Agents

Environments

1. WDMEnv
2. EONEnv

Examples

Training a RL agent for one of the Optical RL-Gym environments can be done with a few lines of code.

For instance, you can use a Stable Baselines agent trained for the RMSA environment:

# define the parameters of the RMSA environment
env_args = dict(topology=topology, seed=10, allow_rejection=False, 
                load=50, episode_length=50)
# create the environment
env = gym.make('RMSA-v0', **env_args)
# create the agent
agent = PPO2(MlpPolicy, env)
# run 10k learning timesteps
agent.learn(total_timesteps=10000)

We provide a set of examples.

Resources

• Introductory paper The {Optical RL-Gym}: an open-source toolkit for applying reinforcement learning in optical networks [paper/pdf] [presentation/pdf] [presentation/YouTube].
• List of publications using Optical RL-Gym