The StarCraft Multi-Agent Challenge

In the last few years, deep multi-agent reinforcement learning (RL)\nhas become a highly active area of research. A particularly challenging class of problems in this area is partially observable, cooperative,\nmulti-agent learning, in which teams of agents must learn to coordinate their behaviour while conditioning only on their private\nobservations. This is an attractive research area since such problems are relevant to a large number of real-world systems and are\nalso more amenable to evaluation than general-sum problems.\nStandardised environments such as the ALE and MuJoCo have\nallowed single-agent RL to move beyond toy domains, such as grid\nworlds. However, there is no comparable benchmark for cooperative multi-agent RL. As a result, most papers in this field use one-off\ntoy problems, making it difficult to measure real progress. In this\npaper, we propose the StarCraft Multi-Agent Challenge (SMAC)\nas a benchmark problem to fill this gap.1 SMAC is based on the\npopular real-time strategy game StarCraft II and focuses on micromanagement challenges where each unit is controlled by an\nindependent agent that must act based on local observations. We\noffer a diverse set of challenge maps and recommendations for best\npractices in benchmarking and evaluations. We also open-source\na deep multi-agent RL learning framework including state-of-theart algorithms.2 We believe that SMAC can provide a standard\nbenchmark environment for years to come.\nVideos of our best agents for several SMAC scenarios are available at: https://youtu.be/VZ7zmQ_obZ0.\n