Matthieu Scharffe

TempleRun-RL

Teaching a reinforcement-learning agent to play the real Temple Run — on a physical phone

The goal

Temple Run is an endless runner: you sprint through crumbling ruins, jumping gaps, sliding under obstacles, strafing along narrow ledges, and turning at corners — faster and faster until you make one mistake and die. Playing it by hand, it took me a long, sweaty session to reach 10,000 m exactly once. That became the challenge: build an RL policy that reaches 10,000 m — reliably — on a real device. Not in a toy re-implementation, on the actual game people download. This is the log of everything it took to get there, including the dead ends, because the dead ends are where most of the engineering lived.

Dead end #1 — building my own Temple Run in Python

My first instinct was to own the whole game. So I built one: an original, Temple-Run-style endless runner from scratch in Python — a fast, deterministic simulation wrapped in a Gymnasium environment, with a full Panda3D 3D renderer that could be played by hand or produce 84×84 frames for a pixel-based policy. The track was a LEGO-style system of procedural tiles (straights, gaps, narrows, turns, hazards), and the world was an original Greek/Olympus theme with assets generated procedurally in Blender — deliberately IP-clean, using none of the real game's art.

My Panda3D clone — an original Greek/Olympus endless runner, playable by hand and wrapped in a Gymnasium RL interface.

It got surprisingly far — but it was the wrong tool for this goal. For a policy trained in my clone to transfer to the real Temple Run, the clone would have to match the real game closely, and two walls made that a project in itself. First, faithfully reproducing the game's exact procedural-generation engine — the precise way it lays out corners, gaps, and narrow sections — is its own deep reverse-engineering problem. Second, and more discouraging, matching the original's look was a rabbit hole: the game's rich temple surfaces come from a hand-painted texture atlas multiplied by a baked lightmap, and my flat-shaded slabs weren't close. I was sinking time into rendering and art that had nothing to do with the learning problem.

The lesson that reframed the whole project: I didn't need pixels — I needed the mechanics. A policy playing Temple Run doesn't see textures; it sees “is there a gap ahead, a corner coming, am I on the ground.” That realization is what eventually became the headless simulator further down — but first I went the other way entirely, and trained on the real game.

Injecting into the real game with a Frida gadget

Temple Run is a Unity / IL2CPP game shipped as native ARM64 code — no scripting layer to hook into. And my test phone is not rooted, so the usual frida-server (which needs root) was off the table. The workaround is a Frida gadget: a library you load inside the app's own process. To get it loaded without touching a line of the game's code, I use LIEF to add a single DT_NEEDED("libgadget.so") entry to the tiny Unity bootstrap library that loads first — so the dynamic linker pulls the gadget in automatically at launch. Merge the split APKs, re-sign with a debug key, and it's a normal installable app that happens to host an instrumentation agent.

From there, the gadget does two jobs, both by reading and calling directly into the game's memory (using offsets recovered from an IL2CPP dump of this exact build):

• Perception. Every frame it reads the player's state (dead, jumping, sliding, on-ground, lane offset, velocity, score) and walks the path graph one and two cells ahead to see what obstacle is coming. It streams all of this as newline-delimited JSON at 30 Hz over a TCP socket.
• Action. The host sends back one byte per action, and the gadget calls the game's own Jump, Slide, and turn methods on the main thread. Crucially it does not use adb input — MIUI blocks synthetic touch events, which is exactly why the in-process approach exists.

Two translations took some reverse-engineering to get right. Lateral movement isn't a button in Temple Run — you tilt the phone — so the gadget overrides the accelerometer reading the game polls, feeding it a commanded tilt so the game's own physics glide the runner left or right exactly as a tilting hand would. And turns are routed through the game's own delayed-turn swipe buffer: instead of forcing a turn on a specific frame, the policy just says “I want to turn left,” and the game fires it at the corner with native timing. That second detail turns out to matter enormously later.

Dead end #2 — parallel phones and emulators

Training on the real game works, but it's slow: one phone is one environment running in real time at roughly 10 decisions per second, with short episodes. RL is hungry — it wants millions of steps — and there is simply no way to collect data faster than wall-clock on a single device. My first fix was to plug several phones into the computer and run them in parallel. It helped a little, but nowhere near enough: the data rate a handful of real phones can produce is a rounding error next to what training needs.

So, an emulator on the PC? That ran into a hardware wall. Temple Run ships as ARM64 native code, and my machine is x86_64:

• An x86_64 emulator runs the game through ARM-to-x86 translation — but that hides the game's native library from Frida (the module lookup returns null), so none of my hooks resolve.
• An ARM64 virtual device won't boot on an x86_64 host at all — the emulator refuses the architecture outright.

That left two honest options: rent an ARM64 cloud machine and run real game instances there, or stop relying on the game entirely and build my own fast simulator. I chose to build.

The simulator — reverse-engineering the mechanics

This is where the Python-clone lesson paid off. The simulator is deliberately headless — no graphics, no attempt to re-create the game's exact level generation. It reproduces only the two things the policy actually depends on: the observation it would see on the phone, and the physics and death rules that decide whether an action keeps you alive. Every constant in it — how velocity ramps from 100 to 210 over ~4,200 m, how long a jump stays airborne, the lateral lag when you tilt, where a narrow section's safe strip sits — is calibrated against real measurements pulled off the phone with Frida, cross-checked against the generation parameters I recovered from the game.

The payoff is speed: the sim runs 10,000+ steps per second and vectorizes across CPU cores, so a training run that would take days of real-time phone play finishes in minutes. Honestly, this simulator was the project — the file is a 1,800-line calibration ledger, and around it sits a whole toolchain whose only job is to keep it honest: a fidelity scorecard that compares dozens of sim quantities against a frozen device reference, and replay tools that push a real recorded run through the sim and diff it step by step. The gap between sim and reality — the sim-to-real gap — is the enemy this entire project is fighting.

Teaching the policy to remember its last moves

The deployed policy is a small, memoryless network: it sees one frame and picks one action. That caused a very specific, very fatal bug. At a corner, the “turn now” signal is visible for about two frames — so a memoryless policy fires the turn on both frames, the second one over-rotates the runner past the corner, and it falls off the edge. Same story for double-jumps.

The fix is borrowed in spirit from sim-to-real work in legged robotics, where a reactive controller is fed a short history of what it just did: I append the one-hot encoding of the last 4 actions to the observation. Now the policy can see “I already turned last frame” and hold off. A small turn-spam penalty and a host-side cooldown reinforce it. Interestingly, feeding past observations instead (classic frame-stacking) actually hurt in my tests — the useful memory here is what the agent did, not what it saw.

Curriculum — drilling one obstacle at a time

In the real game a run is a random mix of every obstacle, which makes rare, hard skills slow to learn. One of the bigger breakthroughs was to isolate a single obstacle type and train focused episodes on just that — I can force both the simulator and, via the gadget, the real game to generate only narrows, or only turns, or only gaps. Narrow sections, which used to be a reliable death, got cracked with a narrow-only curriculum and then warm-started back into the full game.

But curriculum is a double-edged sword, and I want to be honest about it: when I trained on crevasses only, the policy's turning skill atrophied and regressed on device. The rule I landed on is “isolate to teach a new skill, but keep turns practiced in every phase” — you can specialize, but you can't let the agent forget what it already knew.

Reward shaping

The base reward is simple: reward = forward distance gained, minus a penalty on death. One subtlety worth its own debugging session — I reward on the game's distance traveled, not its on-screen score, because the score is inflated by a challenge multiplier that ramps up over a run and would quietly distort the signal. On top of that sits a stack of optional shaping terms — a cost per unnecessary action, a bonus for reacting correctly to an obstacle, a penalty for spamming turns, a pull toward the safe strip in narrows — and every one of them is justified by a measurement taken off the real device. Reward shaping is the least glamorous and most time-consuming part of RL, and this was no exception.

The honest twist — instrumentation beat retraining

Here's the most counter-intuitive thing I learned. At one point I ran a dozen consecutive retrains that all looked great in the simulator and all regressed on the phone. Meanwhile, the two single biggest jumps in real-world performance — around +40% median distance each — came from not retraining at all: they were the instrumentation fixes above (native swipe-queue turns, 30 Hz state). On a live device, perception latency dominated model capacity. A bigger, cleverer network wasn't the lever; fresher, better-timed inputs were. That's a lesson I'll carry well beyond this game.

Watch it run

Left: one of the earliest policies — flailing, dying almost immediately. Right: a trained policy on the real phone, nearly reaching 2,000 m. The gap between these two clips is where all of the above went.

Where it stands, and what's next

The best run so far reached 10,545 m — it cleared 180 turns before finally dying to a flame tower — which means the 10,000 m goal is reachable. The problem is that it happened once. The median run is around 2,600 m and the variance is high, so the entire next phase is about turning a lucky best into a dependable median. Concretely, that means:

• Reliability over peak. Optimize for the median 10k, not the record.
• Closing the last sim-to-real gaps — a handful of named edge cases the sim still doesn't model well (sliding under flame towers at top speed, the rhythm of back-to-back gaps, strafe lag in fast narrows).
• A GPU-accelerated simulator. Today the sim is fast but CPU-bound; moving it onto the GPU to unlock another order of magnitude of throughput is the next big build.

It's not finished — that's rather the point of building in public. The code, warts and calibration notes included, is on GitHub.