Using a Trained Brain

Overview of Using a Trained Brain

Before reading this topic, be sure you have read the following topics:

Once you have finished training with Bonsai, you'll be able to export the brain. Then you'll need to deploy the exported brain, to make sure it is performing well. There are generally three phases of deployment:

  1. As a controller for the simulation model
  2. As an advisor for the real system
  3. As a controller for the real system

Exporting a Brain

Before deploying a brain, the first step is to export it. To do so, follow these steps:

  1. Go to your Bonsai workspace.
  2. Right-click on the version of the brain you want to export, and choose Export Version
  3. Once you have chosen a name, click the Export button.

After a few moments, the exported brain will appear in the menu on the right, in the Exported Brains section. The brain is now ready to be deployed.

Deploying a Brain

We recommend deploying the brain in phases. Each phase is described in the following sections.

Simulation Model Controller

It is always a good idea to evaluate the performance of the brain by allowing it to drive the simulation model. You can learn a lot about the brain by watching it in action. You can also use tools like the Experimenter to run many, many replications, so you can get a sense of the behavior of the brain over many decisions. To allow the brain to control your model, follow these steps:

  1. Be sure that Docker Desktop is running.
  2. Right-click the Docker Desktop icon in the system tray, and choose Switch to Linux containers. If you see an option to Switch to Windows containers, do not choose that option; just proceed to the next step.
  3. Go to your Bonsai workspace.
  4. In the Exported Brains sections, click the ... button for the brain you want to export.
  5. Choose View deployment instructions.
  6. A window will appear with commands for pulling and running a docker container. Run the commands in that window. This will download a container, and run it. The container is a web service: you send a request with the observation space, and it will return the action it has learned to take as a response.
  7. Open the simulation model.
  8. Open the properties for the Reinforcement Learning tool.
  9. From the pickoptions on the On Request Action field, choose Query a Server for an Action.
  10. In the popup, verify the following:
    • The Host field is set to 127.0.0.1
    • The Port field matches the port number in the docker run command, usually 5000
  11. Click the Apply button on the Reinforcement Learning tool.
  12. Reset and run your model.

As the model runs, it will send the observations to the brain running in the docker container. The brain will send its actions back to FlexSim. You can then verify that the brain is performing well, and that it isn't doing anything unexpected. In many cases, watching the model run will reveal a shortcoming in the brain. You'll be able to address that issue and train a better brain, before trying it in the real system.

System Advisor

If your brain behaves well in a simulated environment, you can proceed to deploy the brain in the real system in an advisory role. The exact details of how this is done will be completely unique for each system. the basic steps are as follows:

  1. Run the brain's container as a server.
  2. Read from the system's sensors or databases, and construct the same observations used to train the model.
  3. Send the observations to the brain's server.
  4. Provide a way to view the response action.

The goal of this step is to let experienced employees observe what the trained brain would choose to do, and validate that the brain is taking reasonable actions. If the trained brain is recommending unreasonable actions, it may need to be adjusted and retrained.

Some systems will stop deployment here. Because the brain knows how to make good data-driven decisions, employees can use it as a resource, making their jobs easier.

System Controller

In some systems, it may not be possible or necessary for an employee to apply the brain's instructions. In these cases, the brain can be used to make decisions without human supervision. The system and brain form a fully closed loop. As the system runs, it automatically sends observation data to the brain, and automatically applies the brain's decisions.