Getting Started with Bonsai

Bonsai Key Concepts

Bonsai uses some unique vocabulary. Understanding these terms will help you understand the steps required to use Bonsai:

• Brain - a Bonsai brain is an Agent. When you train in Bonsai, the brain will learn how to meet the goals you set.
• Simulator - a Bonsai simulator is the environment the brain uses to learn. The brain will get observations from the simulator, and send actions to the simulator in response.
• Workspace - a Bonsai workspace contains all the brains and simulators you create. The workspace is where you'll create and define each brain. You can also connect a brain to a simulator, and then train it. In the workspace, you'll be able to monitor the brain's progress while it trains. You'll also be able to export the brain, so it can be deployed as a web service.
• Iteration - During training, a Bonsai brain will take an action. The simulator will run, and return an observation. Bonsai will take another action, moving the simulation along, and yielding another observation. Each action/observation cycle is an iteration. Training good agents in complicated systems can require millions of training iterations.
• Episode - During training, Bonsai will reset the simulator and perform many iterations. Bonsai has a built-in EpisodeIterationLimit. Bonsai will continue the episode until the iteration limit is reached, or until certain kinds of goals are satisfied. Then it will reset the simulation, and run another episode. Episodes continue until training is complete.

Brains

A Bonsai brain is defined by a special text file called an Inkling file. In the Inkling file, you can define the following aspects of the brain:

• What observations are visible to the brain.
• What actions are available to the brain.
• What you expect the brain to learn.

A full description of Inkling code is beyond the scope of this walkthrough. More information can be found in the Bonsai Documentation.

The FlexSim AI repository contains Bonsai examples. Each example includes an Inkling file with a .ink extension. Each example also includes a FlexSim model. The model and the Inkling file are configured to work together in the following ways:

• The model uses Reinforcement learning tool.
  • The Observation Space and Action Space are both set to use Bonsai.
  • The model sets the observation space parameters before requesting a decision.
  • The model uses the action space parameters in its logic.
  • The Reward Function must return an array where the second element (the "done" flag) is always zero. This flag may be used by Bonsai in the future.
• The Inkling file aligns with the model.
  • The Inkling file has a type that matches the observation space. The fields of that type have the exact same names and types as the parameters in the observation space.
  • The Inkling file has a type that matches the action space. The fields of that type have the exact same names and types as the parameters in the action space.

For example, the ChangeoverTimes example includes the parameters LastItemType, Time, and Throughput in the Observation space. It includes the parameter ItemType in the Action space. The Inkling file for that example includes two types: the ObservableState type, which matches the observation space, and the SimAction type, which matches the action space:

# State received from the simulator after each iteration
type ObservableState {
    LastItemType: number,
    Time: number,
    Throughput: number,
}

# Action provided as output by policy and sent as input to the simulator
type SimAction {
    ItemType: number<1 .. 5 step 1>
}

The names "ObservableState" and "SimAction" are arbitrary. The observation and action types can have any valid Bonsai name. However, the names of each field in each type must match the parameter names.

Managed and Unmanaged Simulators

There are two kinds of simulators in Bonsai: unmanaged and managed. The Bonsai brain always runs in the cloud, but an unmanaged simulator runs on your local computer. Consider the following diagram:

You can easily use FlexSim as an unmanaged simulator. This is the recommended way to test that your model and your Inkling code are lined up, before creating a managed simulator. You can also use FlexSim as an unmanaged simulator when you run assessments of the brain, during or after training.

For training, you'll need to create a managed simulator. A managed simulator allows Bonsai to use many instances of FlexSim to speed up training significantly. The exact number of instances will depend on the algorithm Bonsai chooses for learning. When Bonsai trains with a managed simulator, all activity occurs on the cloud:

Bonsai Setup

This section describes the steps you'll need to take in order to work with Bonsai and FlexSim together. You'll need the following resources:

• A Microsoft account
• An Azure Subscription that grants access to the account
• An Azure Resource Group attached to the subscription
• A Bonsai Workspace within the resource group
• The FlexSim for Bonsai offer to the subscription
• Values for SIM_WORKSPACE and SIM_ACCESS_KEY environment variables
• A licensed version of FlexSim

Creating a Bonsai Workspace

Follow the instructions for creating a Bonsai workspace on this page: Microsoft account setup for Bonsai. These instructions show you how to create a Bonsai workspace, including creating accounts, subscriptions, and resource groups.

Add Support for FlexSim to your Workspace

Follow these instructions to add support for FlexSim: Add simulator platform support to your Bonsai workspace. These instructions walk you through adding the FlexSim for Bonsai offer to your subscription. At the end of these steps, you'll need to note three values:

• Application Name
• Managed Resource Group
• Region

Set Environment Variables on your Computer

Follow these instructions to view your workspace details: Get your Bonsai workspace details

From the Workspace Details page, you'll be able to see the following values:

• Workspace ID
• Access Key - this value must be generated. When you generate the key, be sure to copy it into a text editor; you'll only be able to see the key once. However, if you lose it, you'll be able to generate another one. Also, be aware that access keys expire. You'll need to generate another one if that happens.

1. In Windows, open the Environment Variables window.
2. In the User Variables section, click the New button.
3. Set the Variable name to SIM_WORKSPACE.
4. Set the Variable value to your Workspace ID.
5. Click OK to add the new variable.
6. Click the New button again.
7. Set the Variable name to SIM_ACCESS_KEY.
8. Set the Variable value to a valid access key.
9. Click OK to add the new variable.
10. Click OK to close the Environment Variables window.

Samples using FlexSim and Bonsai

You can download the FlexSimAI repository. This repository contains Bonsai samples, including a model and a Brain definition file (a .ink file) that goes with the model. You can download a zip file for the latest version of FlexSim at this link: FlexSimAI-main.zip You can also visit the repository page to clone the repository: FlexSimAI. Be sure that you clone or download the files from the branch corresponding to your version of FlexSim.

Creating a Brain

To create a brain in Bonsai, follow these steps:

1. Open the Bonsai Interface in your browser, and go to your workspace. You may need to specify the subscription, resource group, and workspace that you plan to use.
2. On the left, find the Create Brain button:
3. Click that button, and choose Empty Brain:
4. Once the brain as been created, go to the Teach tab for that brain. Enter your Inkling code in the code editor. If you are using one of the samples, you can open the .ink file in a text editor to copy the code and paste it into the editor.

Using FlexSim as an Unmanaged Simulator

You can use FlexSim as an unmanaged simulator by following these steps:

1. Open the FlexSim model.
2. Open the Reinforcement Learning tool properties.
3. On the Bonsai tab, click the Register Simulator button.

Once you click the button, FlexSim will display its communication with Bonsai. If you go to the Bonsai UI in your browser, you'll see an unmanaged simulator called FlexSim Model appear in the list of simulators. If you click on that simulator, you can see the set of parameters that the model is configured to use in the Simulator Interface Details area of the page.

At this point, you should have a brain, as well as a simulator. With these two components, you can test training with the brain. Using FlexSim as a managed simulator allows you to quickly test that the Brain and the Model are configured correctly. To test training, simply train the brain with your local instance of FlexSim for a short amount of time. See the next topic for more information on training: Training With Bonsai.

While FlexSim is registered as a simulator, you won't be able to reset or run the model normally, since Bonsai is controlling FlexSim. However, you can click the Unregister Simulator button at any time to disconnect FlexSim from Bonsai.

Creating a Managed Simulator

To create a managed simulator, follow these steps:

1. Open the FlexSim model.
2. Open the Reinforcement Learning tool properties.
3. On the Bonsai tab, click the Export Simulator Package... button. This will save a .zip file containing your model.
4. In the Bonsai UI in your browser, click the + Add sim button.
5. Choose FlexSim from the list of available software.
6. Add the .zip file.
7. Enter an appropriate name for the simulator.
8. Enter the correct Resource group name, Application name, and Region. These values are obtained when you add support for FlexSim to your workspace.
9. Click the Create Simulator button.

The new simulator should appear in your list of Simulators immediately, although it may take several minutes before it is ready to use. Once it is ready, you can use it to train your brain. To learn more, see the next topic: Training with Bonsai.