SLAY THE SPIRE SEEDS 2021 MOD
SLAY THE SPIRE SEEDS 2021 SERIES

Lessons Learned Break the problem down into achievable milestones These episodes were too long and the rewards too sparse. I rewarded the agent for completing encounters and punished it for dying.

SLAY THE SPIRE SEEDS 2021 SERIES

A dungeon is a series of about 15 encounters that could be combat, shops, events, or treasure, and ends with a boss combat. The learning environment at this point was the first dungeon of the game–Exordium. It’s much simpler if “pressing a button” always the same effect. What I mean by that is, it’s hard for an agent to learn that playing “card in hand at index 4” only does 6 damage to a monster when the environment indicates that the card at index 4 is Strike. I suspect this action space wasn’t good because there was indirection between “card in hand index” and the effect that playing that card had. Note that there’s no mention of a specific card type in that translation. For example, (4, 1) would translate to “play card in hand at index 4 against the monster at index 1: The part that represented playing cards in combat was essentially ×. This did not work.Īmong other problems, the action space I developed was probably too complicated. I thought I could toss the game at any RL trainer with no thought and get a good agent. I got overhyped on RL after hearing it learned to play Atari games from raw pixels. I started work on decapitate-the-spire, a headless python clone of the game. Lessons LearnedĬreate a way to play the game fast. I hooked up rllib up to the actual game via spirecomm with no other modifications. I considered stable-baselines, but its action mask support was still in the works at the time. I chose rllib because it supports action masking. May need lots of computation to train relative to other ideas like STS Battle AI’s game tree search.On-policy RL is sample inefficient–it has to play the game a lot to get good.If you do have lots of game replays, you can use them to give the agent a head start (transfer learning).No need for a large number of game replays to learn from.Ideally, the more it trains, the better it plays.Model-free RL doesn’t need to be told the game’s dynamics.

RL seems a natural choice when you can play a game many times. I chose model-free reinforcement learning (RL). It plays far better than any AI I’ve trained, although I’ll note that the “save scum” approach it uses to create a game decision tree exploits hidden game internals like RNG seed–it can see the future.

SLAY THE SPIRE SEEDS 2021 MOD

This is a combat only AI that uses a purpose-built mod to run A* search on the game decision tree as it’s played. In my opinion, this approach won’t scale to playing at a high level. It uses hardcoded strategies and priorities. This is, as far as I can tell, a demo for the mod that it’s packaged in. I hope it’ll help others who also want to create a Slay the Spire AI.

I’ll go through my tribulations over the months I’ve worked on this (incomplete) project. The roguelite traits that give the game replay value also make it hard to craft a single policy to play the game: randomness, procedural map generation, card drafting, convertible currencies (health, gold, cards, etc.), long reward horizon, powerful but rare synergies between relics and cards, and so on. Creating an AI to play Slay the Spire is not easy.

Slay the spire seeds 2021