Skip to main content

1.2 - The Core Challenge - async vs sync

The primary technical hurdle in integrating python-sc2 with a library like stable-baselines3 is not specific to StarCraft II, but is a fundamental architectural conflict between their programming models: one is asynchronous and the other is synchronous.

A direct combination of these two libraries within the same process and thread is not feasible.

A Clash of Execution Models​

The two libraries operate on opposing assumptions about which component controls the main application loop.

Aspectpython-sc2 (Asynchronous)stable-baselines3 (Synchronous)
Control FlowAn asyncio event loop runs in the background. It calls our code (on_step) when an event occurs.A standard Python for loop runs in the foreground. It calls the environment's code (env.step()) when it chooses.
BlockingNon-blocking. await yields control to the event loop.Blocking. env.step() must return before the loop continues.
ParadigmEvent-Driven ("Tell me when the game state updates.")Imperative ("Do this step now and give me the result.")

The Technical Deadlock​

This clash creates a deadlock:

  • Running the synchronous RL training loop inside the asynchronous on_step would block the entire asyncio event loop, freezing the game.
  • Running the asynchronous python-sc2 game loop inside the synchronous env.step() method would fail, as there would be no running event loop to handle await calls.

The Solution- Inter-Process Communication (IPC)​

The standard architectural pattern to resolve this is to run each component in its own process and have them communicate via a messaging system, such as queues.

  • Process 1 (RL Trainer): Runs the synchronous stable-baselines3 training loop.
  • Process 2 (SC2 Game): Runs the asynchronous python-sc2 game loop.

This decouples the two loops, allowing them to run concurrently without blocking each other.

Architectural Blueprint​

This diagram illustrates the flow of information between the two processes.

       +------------------------------------+      Action Queue     +---------------------------------+
       |      Process 1: RL Trainer (Sync)  |---------------------->|  Process 2: SC2 Game (Async)    |
       |                                    |                       |                                 |
       |   # Blocks, waiting for obs        |                       |   # Blocks, waiting for action  |
       |   obs, ... = obs_q.get()           |                       |   action = action_q.get()       |
       |   action = model.predict(obs)      |                       |   ... execute action in SC2 ... |
       |   action_q.put(action)             |                       |   obs_q.put(new_obs)            |
       |                                    |                       |                                 |
       +------------------------------------+<----------------------+---------------------------------+
                                               Observation Queue

In the next chapter, we will implement this IPC bridge, creating a gymnasium environment that spawns the StarCraft II game in a separate process, ready for our RL agent to interact with.