Evolutionary Tree Development

For some time, I’ve been occasionally working on an AI-powered interactive fiction game. Think of it like Zork, but the world is generated by AI and populated by AI NPCs. I got to a point where I had a few variations of the game engine, and I needed to fine-tune the prompts and game mechanics.

The whole game was essentially a system of coordinating AI agents.

Testing agentic systems

One thing I realized quite early is that traditional testing approaches didn’t work with a non-deterministic system. I couldn’t just run the same test cases and expect the same results. I needed a way to explore the space of possible behaviors and outcomes. Manual testing is still essential, at least to force me to understand the nuances of the system and the various ways it can fail. However, it is not scalable, so I soon developed a way to automate the testing process by using player agents. They would start a game and play it, mimicking a human player. I even developed a command-line tool that would allow Claude Code (or any other coding agent) to play the game for testing specific features during development.

That quickly led to very useful traces of playthroughs that I could analyze to understand how the game performed.

Feedback Loop

I quickly took the next step and implemented an evaluation agent that would analyze the playthroughs and provide feedback on what worked and what didn’t. Based on this feedback, I would implement new features or tweak the prompts and mechanics. But this also felt automatable.

Another project I’m actively working on is an AI coding factory that orchestrates agents working on a shared codebase. My first idea was to generate user stories based on the evaluation of the playthroughs and let the factory take care of the implementation. But from past experience, I knew that not every change led to an improvement, and often I would create variations of a game type just to experiment with new and better ways of working.

I needed to somehow put the human feedback in there, but also avoid blocking the automation loop.

Evolutionary Tree

So I came up with the idea of creating an evolutionary tree of game versions. Each time I made a change, I would create a new branch in the tree. The evaluation agent would then analyze the playthroughs and identify strengths and weaknesses of each branch. The autonomous evolution process would then select the best performing branches to continue evolving, while discarding the less successful ones. This way, it can keep evolving in different directions and I can also intervene by marking certain branches as ‘good’ or ‘bad’ based on my own judgment.

Implementing the Evolution

Terminology

• Specimen: a game version (each version has a git branch stemming from its parent)
• Lineage: a sequence of specimens (a branch in the tree)
• Mutation: a focused change to a specimen
• Gene pool: a growing shared set of available mutations (never consumed)
• Genome: accumulated mutations in a lineage (prose, per specimen)
• Evolutionary Cycle: the process of creating a new specimen, evaluating it and deciding its fate

Evaluation

Each specimen is evaluated based on the playthrough of the agent. The evaluation agent produces an evaluation report that identifies the strengths and weaknesses of the specimen. The evaluation is not a score, because there are many different dimensions to consider, and they are often in conflict with each other. Also, it is not a vector of scores, because over time the numeric values may become meaningless as the game evolves and we might want to introduce new evaluation criteria. Instead, the evaluation report is a prose description of the specimen’s performance, highlighting what worked well and what didn’t. The evaluation report also includes a comparison with the parent specimen, identifying what has improved and what has regressed.

An evaluation leads to a verdict, which can be:

• Extinct: the specimen is discarded and not used for further evolution
• Dominant: the specimen is selected as the new baseline for further evolution, replacing its parent
• Divergent: the specimen is kept as a separate branch in the tree, coexisting with its parent and allowing for further evolution in a different direction

The Gene Pool

The gene pool is a shared repository of mutations that can be applied to any specimen. Rather than deciding what the mutation should be on the fly, I wanted to have some place where I could collect and organize potential mutations and reuse them across divergent specimens. This way, I can also have a better overview of the available mutations and their effects on different specimens. The gene pool is not consumed, meaning that applying a mutation does not remove it from the pool.

Evolutionary Cycle

On every evolutionary cycle, a base specimen is selected and a new one is created by applying a mutation from the gene pool. A mutation is a focused change that aims to improve the specimen in some way. The implementation involves running a coding agent like Claude Code with the mutation prompt.

Evaluation follows the implementation phase and we decide whether to discard the new specimen, replace the parent specimen with it or keep it as a divergent branch.

Early Results

As I am writing this, I have the Evolution System running and producing specimens with funny names such as “bacterium-hinticus”, “amoeba-pressurus”, or “paramecium-falsicus”. It is still early in the evolutionary tree, but the game produced is playable and has some interesting mechanics. The system seems to prefer long evolutionary lineages with a few divergent specimens, which means most mutations are improvements over the parent specimen.

It is too early to draw conclusions, but the approach seems promising and I am excited to see how the game evolves over time. I also believe that this approach can be applied to other domains beyond game development.

My Evolution vs Karpathy Loop

Recently, Andrej Karpathy made news by implementing the Autoresearch loop, which is a process of continuously improving an AI system by collecting data, training models, and deploying them at five-minute intervals. The two approaches share some similarities, but have some significant differences.

Karpathy’s loop works well in scenarios where you have a clear metric to optimize. If your system can be evaluated with a numeric score, you can easily assume that there will always be exactly one best candidate and you can always keep just the best candidate. In that case, you can afford to discard all the other candidates and focus on improving the best one.

If you are building something that needs to satisfy multiple criteria that are often in conflict with each other, you might want to keep multiple candidates and evolve them in different directions. In that case, you can afford to keep multiple candidates and explore different trade-offs between the criteria. The evolutionary tree approach allows for more diversity, as it does not assume that there is a single best solution. It also allows for more human intervention, as you can mark certain branches as good or bad based on your own judgment.

Ultimately, the Karpathy loop is a special case of the evolutionary tree approach, where you have a single lineage and you discard all the other branches. The evolutionary tree approach is more general and can be applied to a wider range of scenarios, including those where you have multiple criteria and you want to explore different trade-offs between them.

Further Work

My next steps are introducing a mechanism for controlling the number of divergent specimen and the size of the gene pool. I want to see how far can this approach scale.

Another direction that I’ll pursue is self-evolving the system, so instead of just modifying the game , I should modify the evolutionary system itself. I’ve already tried similar approaches on a smaller scale when migrating my blog, but now I’d like to see it applied on something more complex.

I still haven’t released the evolutionary game type, but while you wait you can still try out the base version here.