Coding agents are the ultimate sandbox game

It turns out the Environment Agency have done a LiDAR scan of England. They provide point clouds with 1 metre lat/lon resolution, showing the measured height of the terrain at each point, across the entire country.

They provide both a DTM ("digital terrain model") and DSM ("digital surface model"), where the DTM has been somehow processed to only tell you actual terrain, while the DSM tells you heights including vegetation and buildings etc.; there is also a combined point cloud that gives you DSM heights but includes a classification of each point to say whether they think it is ground or vegetation or a roof.

James Nation recently posted a 3d reconstruction of Bruton Dovecote, using the Environment Agency LiDAR data to provide the shape of the surrounding terrain, this is how I found out about the LiDAR data.

So this sounds like an incredibly powerful dataset, but what can we do with it? I started trying to just make a viewer so that I could explore the point cloud dataset and hopefully think of something cool to do, and you know how it is with LLM coding agents, one thing led to another and before I knew it I was flying a biplane around Somerset.

Flight simulator

This is a game using the Godot engine, plus a Python script that takes in Environment Agency point clouds and OpenStreetMap data, and constructs meshes of the terrain suitable for Godot.

I currently only have a small area of the country, I'd love to do the whole thing but it is 60 gigabytes of Godot assets already.

About 15 years ago I tried to play FlightGear, and found I didn't really get on with it. It was super slow on my PC, plus you have to spend a million years pretending to switch on magnetos and fuel pumps etc. before you can even fly the plane.

I had much more fun playing GL-117. It is visually a much less impressive game, but you get to experience the joy of flight with less faffing around. It has a crosshair controlling the "stick", and you basically just point the mouse where you want the plane to fly. So I copied that interaction model for my game.

The ultimate sandbox game

I realised while I was playing with this that the flight simulator is not the real game here.

I briefly harboured the delusion that I was making a flight simulator that I might some day release, but I now realise that that is not what it's about. It's never going to be "finished". Adding things to the game and then flying around the local area is the game, for me. Treating it as a finished work that is just played as-is is missing most of the fun.

Plus it's 60 gigabytes, and bulk-downloading the Environment Agency data is super inconvenient, they make you download it in 5km chunks and have some anti-bot stuff that makes it difficult (not impossible) for the LLM to download them automatically. All this stuff is just annoying for other people to have to deal with. (But apparently there is an easier way to get it).

So I think "finishing the game" and "releasing it" are not worthwhile. If you want to play the game I've been playing, start up your favourite coding agent and get started. The joy is in making your own.

Traditionally a game involves a "gameplay loop", which is a sequence of things that take place within the game that are (intended to be) fun for the player. For example in Mario you run through the level collecting coins and avoiding enemies and trying to reach the end, and repeat that for every level.

In my flight simulator the gameplay loop is to fly around in the game, have an idea occur to me, ask Codex to implement it, and then fly around some more now with the benefit of the cool new feature. It's a very personal type of gameplay.

I also realised that a lot of the joy I have got from programming over the years has been this kind of "gameplay", it's just that before coding agents I had to focus much more on the programming and much less on the new ideas.

AI games

4 years ago I wrote in The future of virtual reality about this kind of thing being possible, although it isn't (yet?) playing out how I expected. I expected that you'd have some kind of AI-powered 3d world model that would behave more like an image generation model that you can walk around, and that the coding agent would have to do only a tiny amount of work in plugging together pre-existing parts. That's not really what's happened. Instead coding agents have been absolutely wildly successful, to the point that they just implement everything you ask for by hand and you don't need an AI involved at runtime.

The biplane in the game is not even an asset I downloaded from somewhere, I just asked the bot for a cartoon-style biplane and it made me one. I didn't know whether it made a static triangle mesh or whether it was constructing it at runtime. (As it happens, it's made at runtime out of Godot primitives for spheres, cylinders, and boxes, with procedural shaders for fabric and metal seams).

My flight simulator is still in the realm of "classical games programmed by AI" rather than "AI games" as such.

I still think that an AI-driven world will be a material step-up in capabilities, but the required models don't exist yet, and end-user hardware wouldn't currently be able to run them anyway.