Can a language model fly a drone?
A language model that has never seen an image flies a drone, and it gets better with practice, without any training. We challenged Ripple, a text-only model, to fly a simulated drone through a scanned factory, set up as a search-and-rescue drill, with no camera and no map. Flying is our test of whether a model that only reads text can reason about space.
So how does a model that reads only text find its way through a building?
It sees no images. Only words.
What Ripple gets, every second
What Ripple sends back
A velocity: east, up, north. Shown is the one flown.
On 48 flights Ripple steered itself, delaying its commands by one second cut its success from 29% to 8%. It flies by the text it reads.
Read a full prompt
On another factory route, H22, Ripple had 12 metres of clear space to the south and 2.5 to the north, and it asked for 4.0 metres per second south. The command only sets a target velocity, so momentum and wind still affect where the drone actually goes.
Route H22 · second 2
Blue arrow · requested move. Dotted line · actual path.
- Instrument reading
{ "position": [-16.1, 0.7, 6.7], "velocity": [-0.4, 0.1, -3.8], "goal": {"distance_m": 16.3, "in_line_of_sight": false}, "free_space_m": { "east": 3, "north_east": 3.5, "north": 2.5, "north_west": 3.8, "west": 1.5, "south_west": 11.8, "south": 12, "south_east": 4.5, "up": 1, "down": 0.8 } }- Ripple’s reply
{ "velocity": [0, 0, -4], "say": "Flying south toward goal, south clear, north blocked" }
The full prompt for this move
You are piloting a small drone through an indoor building. Every second you receive what the drone senses and you give the next command. Reply with one JSON object and nothing else.
The building is a 3D scan of a real factory: halls, rooms, pillars, machinery and clutter. Coordinates are [x, y, z] in metres: +x is east, +z is north, +y is up.
You have no map. You sense your position and velocity, the goal's position, whether the goal is in direct line of sight, and free space (metres, up to 12) in 8 compass directions plus up and down.
Your command is a velocity the drone holds for the next second. Nothing plans a path for you.
The drone is a sphere of radius 0.25 m with top speed 5.5 m/s; random wind gusts push it. Touching anything counts as a wall contact (it slows you and counts against you). You reach the goal within 1 m of it. You have 90 s. You give a new command every second.
Your last commands: [{"t":0,"command":{"velocity":[0,0,4]},"said":"Moving north toward goal, no line of sight yet"},{"t":1,"command":{"velocity":[0,0,-4]},"said":"Flying south toward goal, east blocked, south is clear"}]
What the drone senses now:
{"time_s":2,"time_left_s":88,"wall_contacts_so_far":2,"position":[-16.1,0.7,6.7],"velocity":[-0.4,0.1,-3.8],"goal":{"position":[-1.4,0.6,-0.4],"distance_m":16.3,"in_line_of_sight":false},"free_space_m":{"east":3,"north_east":3.5,"north":2.5,"north_west":3.8,"west":1.5,"south_west":11.8,"south":12,"south_east":4.5,"up":1,"down":0.8}}
Return JSON with these keys:
"velocity": [vx, vy, vz] in m/s (vector length at most 5.5)
"predict_position_in_1s": [x, y, z] where you expect to be one second from now
"predict_goal_distance_in_1s": the distance to the goal in metres you expect one second from now
"say": at most 12 words: what you are doing and why
The eight horizontal readings and two vertical readings each reach up to 12 metres. The simulator checks collisions in a grid of 25 cm cubes and updates the drone’s motion 60 times a second. Ripple can ask for any velocity within the flight limits, including a diagonal move between two sensor directions.
We used Ricursive Intelligence’s Ripple-65B-Preview. Model announcement · Model weights
Two checks on what Ripple read and said
Would the same commands work in a different room?
Ripple’s original commands reached the goal on 29% of 48 development flights. Replaying those commands on a different route brought that down to 0%, and even delaying them by one second cut it to 8%.
| Controller on the same 48 dev flights | Reached | Time touching walls |
|---|---|---|
| Hover in place | 0% | 12% |
| Random direction every second | 0% | 26% |
| Straight at the goal, no avoidance | 2% | 95% |
| Ripple's own commands | 29% | 46% |
| …the same commands, flown on a different route | 0% | 62% |
| …the same commands, one second late | 8% | 55% |
| …its first command, held with no feedback | 0% | 90% |
Did Ripple always turn the way it said it would?
At second four on route H22, Ripple said southeast and commanded south. The table highlights disagreements between the rounded compass labels. Across seven development runs, the unrounded headings differed by more than 67.5° on 20–27% of decisions where both directions could be measured.
| t | What it said | Said | Commanded | Same label? |
|---|---|---|---|---|
| 0s | “Moving north toward goal, no line of sight yet” | north | north | Yes |
| 1s | “Flying south toward goal, east blocked, south is clear” | south | south | Yes |
| 2s | “Flying south toward goal, south clear, north blocked” | south | south | Yes |
| 3s | “Flying south toward goal, keeping altitude low, checking sight” | south | south | Yes |
| 4s | “Flying south-east toward goal, keeping low, avoiding west wall” | south-east | south | No |
| 5s | “Flying south, slight rise and east, avoiding west wall” | south | south | Yes |
| 6s | “Slowing south, rising toward goal, avoiding west wall” | south | north-west | No |
| 7s | “Flying east and south toward goal, keeping altitude, avoiding walls” | south-east | north-east | No |
| 8s | “Flying east and south toward goal, keeping altitude, avoiding walls” | south-east | north-east | No |
| 9s | “Slowing east, flying south toward goal, avoiding walls.” | east | south-west | No |
| 10s | “Fly east and north toward goal, keeping altitude.” | north-east | north-east | Yes |
| 11s | “Flying west and north toward goal, avoiding walls.” | north-west | north-west | Yes |
Can it get better with practice?
Ripple was built to write code, so it practiced by writing. It wrote the drone’s flight program, flew it on 12 practice routes, and rewrote it after each round from the logs of its failed flights. Nothing about the model changes while it practices: its weights stay fixed, and what improves is the flight program it writes.
What went wrong
* Multiple flights ended with the drone pressed against a *corner* or a wall while the goal was in a direction the policy considered "admissible" but the clearance there was tiny (0.5-1.2 m). The drone then oscillated against the same surface for the rest of the flight.
[…]
A pure "max alignment with goal" score cannot see around the corner.
What it changed
New design (v7):
[…]
* Added a "stuck detector" that fires when the drone has not reduced its distance to the goal over the last N ticks AND its velocity is small or it is pressed against a wall: on firing, it forces a 90-degree turn away from the wall for a few ticks (memory.escapeUntil).
* Added a "corner detector": if two perpendicular sensors are blocked (e.g. east and south both 0), the drone is in a corner; it must turn toward the third open direction, not slide along either wall.
Rewriting its own code from the logs of its failed flights took Ripple’s program from 0 to 8/12 practice routes in six rewrites. Of the eight coding runs we charted, seven ended better than they began.
Explore the practice scores of all the coding runs
Each line is one coding run of 12 practice routes and eight versions. These are 8 of the 24 runs considered for the test. Choose a run to see its scores.
Run 1 finished 0, 2, 2, 2, 0, 2, 6 and 2 of the 12 practice routes, version by version.
Run 2 finished 0, 3, 1, 3, 2, 3, 3 and 3 of the 12 practice routes, version by version.
Run 3 finished 0, 0, 4, 1, 4, 3, 7 and 4 of the 12 practice routes, version by version.
Run 4 finished 0, 0, 0, 0, 0, 0, 0 and 0 of the 12 practice routes, version by version.
Run 5 finished 1, 3, 4, 3, 4, 1, 4 and 4 of the 12 practice routes, version by version.
Run 6 finished 3, 0, 3, 3, 2, 3, 2 and 4 of the 12 practice routes, version by version.
Run 7, whose program went to the test, finished 0, 2, 1, 5, 2, 1, 8 and 7 of the 12 practice routes, version by version.
Run 8 finished 0, 2, 3, 4, 5, 0, 2 and 1 of the 12 practice routes, version by version.
Fixing one route could break another. In one run, Ripple changed how much clearance its program checked before a turn. That finished a previously stuck flight in ten seconds, but cut the practice goals it reached from five to two.
Ripple’s notes on a difficult corner
Flight 1 is the clearest example: at [-12.8,0.5,-1.4] the sensors show north = 6.3, north_west = 12, west = 12, and the goal is at [-10.6,-0.9], i.e. to the north-east. The policy returned vz = +4.42 (north) but the drone was sliding along the north wall; it needed to go *west* first (west = 12 m free) to get around the corner, then north. A pure "max alignment with goal" score cannot see around the corner.
From the selected program’s change log
How far can we push it?
Can it fly routes it has never seen?
We froze Ripple’s best program, so the model was out of the loop, and flew it on 48 factory routes it had never practiced. Here are four of the routes it finished.
Ripple’s program reached the goal on 65% of flights it had never practiced, close to the 67% it scored on its practice routes.
Can it fly somewhere new?
We took the program out of the factory. In a second building it flew unchanged. In a canyon and among cottages, Ripple practiced again in each place. The clips show an example flight in each place.