Artificial life simulations are full of small creatures that need brains: cells chasing food, swimmers, walkers, whole ecosystems of them. Those brains are usually evolved, and one of the nicest ways to do it came out of the field itself: ES-HyperNEAT (Risi & Stanley, 2012).
ES-HyperNEAT doesn’t evolve a network’s weights one by one. It evolves a small function that draws them from geometry. Each neuron gets coordinates, and the weight between two neurons is that function evaluated at their two positions, so moving a neuron changes all of its weights, and with them what it computes. That lets a creature’s brain mirror the layout of its body, and the same function decides where the hidden neurons go.
The same family of encodings has since evolved cellular automata that grow and copy patterns (Nichele et al., 2017), and whole ecosystems of neural cellular automata (Barbieux & Canaan, 2024).
ES-HyperNEAT’s brains are usually flat, though. Its search is a quadtree, built for a 2D sheet, while plenty of simulated creatures live in 3D. What I wanted to know is whether that matters. The test is a single cell chasing a drifting bit of food in 3D.
With the neurons laid out in 3D, the one-line rule “connect neurons that are close” steers about as well as a hand-built controller. Flatten the same neurons onto a plane and the rule falls apart in every layout I tried. In one of them the up and down thrusters end up on the same spot and cancel each other out, so the cell can never change height.
Evolving from scratch tells a similar story. ES-HyperNEAT found a good controller in every 3D run, and in about half the runs on the best 2D layout.
Every evolved run, one per chase. 3D always catches the food, the 2D map about half the time, and drop z can't change height.
The problem is cost. The search that places hidden neurons gets exponentially more expensive with every dimension you add, and Part 2 is about making it cheaper.
Epistemic status: one toy task, 8 seeds per setup, and a simplified ES-HyperNEAT written from scratch rather than the reference code. The hand-set rule result is big and clear. The evolution result is suggestive, with one open confound: the 3D networks grew about six times as many hidden neurons.
1. HyperNEAT computes weights from positions
HyperNEAT puts every neuron at a point in a geometric space, called the substrate, and computes each weight from the positions of the two neurons it connects:
\[w = f(\mathbf{p}, \mathbf{q})\]The function \(f\) is itself a small network, the CPPN (compositional pattern-producing network). Evolution never touches the weights directly. It evolves \(f\), originally with NEAT.
Because \(f\) is smooth, neurons that sit near each other get similar weights. And since \(f\) only ever sees coordinates, a rule like “connect neurons that are close” needs nothing more than the distance \(\lVert \mathbf{p} - \mathbf{q} \rVert\). (HyperNEAT is usually sold on symmetry and repetition. This post leans on something plainer, locality.)
So where do the coordinates come from? The designer places the input and output neurons to match the physical layout. A sensor that looks left goes on the left of the substrate, and a thruster that pushes up goes near the top. That’s the only place the physical world gets into the network.
Hidden neurons don’t correspond to anything physical. What matters is what they’re near. A hidden neuron’s weights are the CPPN evaluated between its position and each input and output, so under “connect what’s close”, one sitting between the left sensor and the left thruster mostly listens to the first and drives the second. Its position is its job.
ES-HyperNEAT (Risi & Stanley, 2012) goes a step further and lets \(f\) place the hidden neurons too. Only the inputs and outputs are fixed. Pin one end of a connection to an input neuron, say at \((0, -1)\), and \(f(0, -1, x, y)\) becomes a scalar field over the substrate: the weight a hidden neuron at \((x, y)\) would get from that input.
Hidden neurons go where this field varies. Where it’s flat, neighbouring neurons would get identical weights and compute the same thing, so extra ones would be wasted.
2. A quadtree finds where the field varies
To find those regions, ES-HyperNEAT keeps splitting the square into quarters:
- Evaluate the CPPN at the centres of a cell’s four quarters and take the variance of the four weights, starting with the whole square.
- If the variance is above a threshold \(\tau\), split the cell and repeat on each quarter, down to some maximum depth.
- Every leaf cell whose variance is still above a lower threshold gets a hidden neuron at its centre.
The explorer below runs this on the field of a random CPPN, \(w(x, y) = f(0, -1, x, y)\). Colour is the weight, lines are the leaf cells and dots are hidden neurons. Drag \(\tau\) down and the tree digs into the busy regions.
3. In 3D, the right controller is a one-line rule
Since the CPPN only sees coordinates, a substrate works well when its coordinates capture the geometry the task cares about. The weights you need are then a simple function of position, and simple functions are what evolution tends to find first, because it builds up a CPPN one mutation at a time.
The test task is deliberately 3D. A point agent (the cell above) chases a target (the food) along a random 3D curve. It has 14 sensors pointing in fixed directions \(\mathbf{u}_k\), the 6 faces and 8 corners of a cube, and sensor \(k\) reads \(s_k = \max(0, \mathbf{u}_k \cdot \hat{\mathbf{r}})\), where \(\hat{\mathbf{r}}\) points at the target. It steers with 6 thrusters, along \(\pm x\), \(\pm y\) and \(\pm z\), through a network with one hidden layer.
A hand-built controller that just thrusts towards the target scores 0.730. Zero thrust scores 0.225.
That controller is purely geometric. If a sensor fires, the food is roughly in that sensor’s direction, and the thrusters pointing that way will push the cell towards it. So thruster \(j\) should respond to sensor \(k\) in proportion to how well their directions line up, \(\mathbf{u}_k \cdot \mathbf{a}_j\). Put every sensor and thruster on the substrate at its real direction (sensors at radius 1, thrusters at radius 0.5) and “lined up” turns into “close together”, because at fixed radii
\[\lVert \mathbf{p} - \mathbf{q} \rVert^2 = \lVert \mathbf{p} \rVert^2 + \lVert \mathbf{q} \rVert^2 - 2\, \mathbf{p} \cdot \mathbf{q}\]shrinks as \(\mathbf{p} \cdot \mathbf{q}\) grows. So the locality rule
\[w = b - k \lVert \mathbf{p} - \mathbf{q} \rVert\]is the controller, routed through a grid of hidden neurons. Short connections are positive and long ones negative. A sensor excites the hidden neurons near it, and they excite the thrusters near them, which point the same way as the sensor.
A flat substrate can’t do this properly. The Borsuk–Ulam theorem says any continuous flattening of a sphere sends some pair of opposite directions to the same point. With only 14 sensors you can dodge the collision by tilting the projection, but the map still squashes some directions together and pulls others apart. The ring layout avoids collisions by giving up continuity altogether, so neighbours on the ring point in unrelated directions.
Try it below. Pick a substrate and play with \(k\) and \(b\), or hit the search button. The hidden neurons sit on a fixed grid here; in section 4 evolution gets to place them.
Live scores on 16 random chases. The table has the best settings from a grid search.
| Substrate | Best score with \(w = b - k\lVert \mathbf{p} - \mathbf{q} \rVert\) |
|---|---|
| 3D (true directions) | 0.745 |
| 2D, drop z | 0.317 |
| 2D, map projection (azimuthal equidistant) | 0.306 |
| 2D, sensors round a ring | 0.231 |
| reference: zero thrust / hand-built controller | 0.225 / 0.730 |
In 3D the locality rule does as well as the hand-built controller. In every 2D layout it lands much closer to zero thrust. A cleverer CPPN could encode exceptions to locality, but evolution would have to stumble on each one.
4. Evolution mostly agrees, with one telling failure
The hand-set rule used a fixed grid, though. The real test is letting ES-HyperNEAT do the whole job: evolve the CPPN from a random start and place the hidden neurons with its own quadtree.
How the experiment works: dynamics, learning algorithm, runs and code
- Task. The agent integrates its net thrust \(\mathbf{F}\) with damping, \(\mathbf{v} \leftarrow 0.8\,\mathbf{v} + 0.3\,\mathbf{F}\) and \(\mathbf{x} \leftarrow \mathbf{x} + 0.25\,\mathbf{v}\), so it overshoots if it thrusts too hard. A chase lasts 60 steps and scores \(1/(1 + \bar{d})\), where \(\bar{d}\) is the mean distance to the target.
- Learning. Each run starts from a randomly initialised CPPN and trains it with an evolution strategy. Every generation it evaluates 32 random perturbations of the CPPN’s parameters (16 antithetic \(\pm\) pairs), estimates the gradient of the score from their rank-weighted results, and takes an Adam step. The score is treated as a black box: no gradient passes through the simulation. A run lasts 120 generations.
- Scoring. Every 5 generations the current CPPN is tested on 16 held-out chases that are never used for training. That test score is what the charts show.
- Runs. A seed fixes a run’s random initialisation and random choices. There are 8 seeds for the 3D substrate and the 2D map projection, and 4 for each of the two cruder 2D layouts. Everything except the substrate is identical.
- Code. About 800 lines of NumPy, in /experiments/es-hyperneat/.
python run_all.pyreruns everything, andresults/holds each run’s raw output, including its final CPPN.
Top: mean test score, ±1 standard error. Bottom: every run's final score.
Every 3D run ended up level with the hand-built controller.
The 2D map projection split down the middle. Four of its eight runs got there too, just later, and the other four stalled somewhere between 0.27 and 0.54. The ring layout never got above 0.35.
The “drop z” layout is my favourite failure. All four runs finished at exactly 0.319. Dropping \(z\) sends the \(\pm z\) directions to the same point, which is exactly where a sensor pair and a thruster pair live. The two vertical thrusters always get identical weights, fire together and cancel, so the cell can’t change height whatever the CPPN does.
There’s one confound I can’t rule out. The 3D networks grew about 480 hidden neurons to 2D’s 75. That isn’t because the 2D tree ran out of room, but it does mean this experiment can’t separate “more dimensions” from “more neurons”.
5. Each dimension multiplies the search
In \(n\) dimensions the quadtree becomes a \(2^n\)-tree. A cell has 4 sub-cells in 2D, 8 in 3D and 64 in 6D, and both parts of the cost scale with that:
- every test evaluates the CPPN at \(2^n\) sub-cell centres;
- every split makes \(2^n\) new cells, each needing its own test.
If the field varies everywhere down to depth \(m\), that’s roughly \((2^n)^m\) cells at \(2^n\) evaluations each, for every input neuron.
Here’s what that looks like on random CPPNs in 2 to 7 dimensions, with trees of depth 2:
Median over 10 random CPPNs, log scale. Tree sizes vary, which is why 5D is no dearer than 4D.
Show the numbers
| n | cells tested | CPPN evaluations | time per search |
|---|---|---|---|
| 2 | 13 | 416 | 1.1 ms |
| 3 | 49 | 3,136 | 3.9 ms |
| 4 | 193 | 24,704 | 27 ms |
| 5 | 97 | 24,832 | 27 ms |
| 6 | 1,537 | 786,944 | 0.87 s |
| 7 | 6,209 | 6,358,016 | 8.8 s |
Timings from an AMD Ryzen 7 7840HS.
Going from 2D to 7D makes the work about 15,000 times bigger, and that’s one search on one network. Evolution runs a search for every candidate CPPN in every generation. Even in the steering experiment, moving from 2D to 3D stretched a 1.2-minute training run to 5.9 minutes.
6. Caveats
- The prediction was only half right. The hand-set rule made 2D look hopeless. Evolution then solved the 2D map projection half the time, so a mismatched substrate makes good solutions harder and less reliable to find. It doesn’t rule them out.
- Nothing learned at first. The first version never beat zero thrust. Random CPPNs saturate every neuron, and opposing thrusters cancel. Fan-in scaling and a zero-mean output shift, applied to every run before any comparison, fixed it. A hidden-neuron cap that was quietly bunching neurons into one corner came out as well.
Simplifications and scope
- Simplifications. This ES-HyperNEAT was written from scratch, not taken from the reference implementation. The CPPN has a fixed shape and is trained with an evolution strategy instead of NEAT. There’s one tree per network instead of one per input neuron, a single hidden layer, and a simplified version of the published pruning step. All of that is the same across substrates, but the absolute numbers would come out differently in the reference code.
- Narrow evidence. It’s one toy task, picked because its geometry makes the point, with 8 runs per substrate, so only fairly large differences show up. It also only compares 3D with 2D.
7. The dilemma
Matching the problem’s geometry argues for as many dimensions as the problem has, and most physical problems, from robot arms to drones, are 3D. Every dimension you add multiplies the cost of placing neurons.
There may be a way round it. The quadtree’s variance test is really asking how fast the weight field changes across a cell, which is a question about its gradient, and the CPPN is built from differentiable functions. Part 2: A gradient test for ES-HyperNEAT’s quadtree swaps the \(2^n\) samples for one gradient and checks whether it finds equally good networks for less work.
References
- Barbieux, A. & Canaan, R. (2024). Coralai: Intrinsic evolution of embodied neural cellular automata ecosystems. Proceedings of the Artificial Life Conference (ALIFE 2024). arXiv:2406.09654
- Nichele, S., Ose, M. B., Risi, S. & Tufte, G. (2017). CA-NEAT: Evolved compositional pattern producing networks for cellular automata morphogenesis and replication. IEEE Transactions on Cognitive and Developmental Systems. doi:10.1109/TCDS.2017.2737082
- Risi, S. & Stanley, K. O. (2012). An enhanced hypercube-based encoding for evolving the placement, density, and connectivity of neurons. Artificial Life, 18(4), 331–363. doi:10.1162/artl_a_00071