Spatial Arena

Can a language model fly a drone?

A language model that has never seen an image flies a drone, and it gets better with practice, without any training. We challenged Ripple, a text-only model, to fly a simulated drone through a scanned factory, set up as a search-and-rescue drill, with no camera and no map. Flying is our test of whether a model that only reads text can reason about space.

So how does a model that reads only text find its way through a building?

It sees no images. Only words.

A drone in a factory hall with an adversary chasing it. The photographic scan is on the left and the collision grid on the right, where lines from the drone show the free space around it.

What Ripple gets, every second

What Ripple sends back

A velocity: east, up, north. Shown is the one flown.

On 48 flights Ripple steered itself, delaying its commands by one second cut its success from 29% to 8%. It flies by the text it reads.

Read a full prompt

On another factory route, H22, Ripple had 12 metres of clear space to the south and 2.5 to the north, and it asked for 4.0 metres per second south. The command only sets a target velocity, so momentum and wind still affect where the drone actually goes.

Route H22 · second 2

3 m3.5 m2.5 m3.8 m1.5 m11.8 m12+ m4.5 mN5 m

Blue arrow · requested move. Dotted line · actual path.

Instrument reading
{
  "position": [-16.1, 0.7, 6.7],
  "velocity": [-0.4, 0.1, -3.8],
  "goal": {"distance_m": 16.3, "in_line_of_sight": false},
  "free_space_m": {
    "east": 3, "north_east": 3.5,
    "north": 2.5, "north_west": 3.8,
    "west": 1.5, "south_west": 11.8,
    "south": 12, "south_east": 4.5,
    "up": 1, "down": 0.8
  }
}
Ripple’s reply
{
  "velocity": [0, 0, -4],
  "say": "Flying south toward goal, south clear, north blocked"
}

The full prompt for this move

You are piloting a small drone through an indoor building. Every second you receive what the drone senses and you give the next command. Reply with one JSON object and nothing else.

The building is a 3D scan of a real factory: halls, rooms, pillars, machinery and clutter. Coordinates are [x, y, z] in metres: +x is east, +z is north, +y is up.
You have no map. You sense your position and velocity, the goal's position, whether the goal is in direct line of sight, and free space (metres, up to 12) in 8 compass directions plus up and down.
Your command is a velocity the drone holds for the next second. Nothing plans a path for you.
The drone is a sphere of radius 0.25 m with top speed 5.5 m/s; random wind gusts push it. Touching anything counts as a wall contact (it slows you and counts against you). You reach the goal within 1 m of it. You have 90 s. You give a new command every second.

Your last commands: [{"t":0,"command":{"velocity":[0,0,4]},"said":"Moving north toward goal, no line of sight yet"},{"t":1,"command":{"velocity":[0,0,-4]},"said":"Flying south toward goal, east blocked, south is clear"}]

What the drone senses now:
{"time_s":2,"time_left_s":88,"wall_contacts_so_far":2,"position":[-16.1,0.7,6.7],"velocity":[-0.4,0.1,-3.8],"goal":{"position":[-1.4,0.6,-0.4],"distance_m":16.3,"in_line_of_sight":false},"free_space_m":{"east":3,"north_east":3.5,"north":2.5,"north_west":3.8,"west":1.5,"south_west":11.8,"south":12,"south_east":4.5,"up":1,"down":0.8}}

Return JSON with these keys:
  "velocity": [vx, vy, vz] in m/s (vector length at most 5.5)
  "predict_position_in_1s": [x, y, z] where you expect to be one second from now
  "predict_goal_distance_in_1s": the distance to the goal in metres you expect one second from now
  "say": at most 12 words: what you are doing and why

The eight horizontal readings and two vertical readings each reach up to 12 metres. The simulator checks collisions in a grid of 25 cm cubes and updates the drone’s motion 60 times a second. Ripple can ask for any velocity within the flight limits, including a diagonal move between two sensor directions.

We used Ricursive Intelligence’s Ripple-65B-Preview. Model announcement · Model weights

Two checks on what Ripple read and said

Would the same commands work in a different room?

Ripple’s original commands reached the goal on 29% of 48 development flights. Replaying those commands on a different route brought that down to 0%, and even delaying them by one second cut it to 8%.

Controller on the same 48 dev flightsReachedTime touching walls
Hover in place0%12%
Random direction every second0%26%
Straight at the goal, no avoidance2%95%
Ripple's own commands29%46%
…the same commands, flown on a different route0%62%
…the same commands, one second late8%55%
…its first command, held with no feedback0%90%

Did Ripple always turn the way it said it would?

At second four on route H22, Ripple said southeast and commanded south. The table highlights disagreements between the rounded compass labels. Across seven development runs, the unrounded headings differed by more than 67.5° on 20–27% of decisions where both directions could be measured.

tWhat it saidSaidCommandedSame label?
0s“Moving north toward goal, no line of sight yet”northnorthYes
1s“Flying south toward goal, east blocked, south is clear”southsouthYes
2s“Flying south toward goal, south clear, north blocked”southsouthYes
3s“Flying south toward goal, keeping altitude low, checking sight”southsouthYes
4s“Flying south-east toward goal, keeping low, avoiding west wall”south-eastsouthNo
5s“Flying south, slight rise and east, avoiding west wall”southsouthYes
6s“Slowing south, rising toward goal, avoiding west wall”southnorth-westNo
7s“Flying east and south toward goal, keeping altitude, avoiding walls”south-eastnorth-eastNo
8s“Flying east and south toward goal, keeping altitude, avoiding walls”south-eastnorth-eastNo
9s“Slowing east, flying south toward goal, avoiding walls.”eastsouth-westNo
10s“Fly east and north toward goal, keeping altitude.”north-eastnorth-eastYes
11s“Flying west and north toward goal, avoiding walls.”north-westnorth-westYes

Can it get better with practice?

Ripple was built to write code, so it practiced by writing. It wrote the drone’s flight program, flew it on 12 practice routes, and rewrote it after each round from the logs of its failed flights. Nothing about the model changes while it practices: its weights stay fixed, and what improves is the flight program it writes.

Start Goal
What Ripple wrote before its next attemptComment at the top of version 7, shortened where marked […]

What went wrong

* Multiple flights ended with the drone pressed against a *corner* or a wall while the goal was in a direction the policy considered "admissible" but the clearance there was tiny (0.5-1.2 m). The drone then oscillated against the same surface for the rest of the flight.

[…]

A pure "max alignment with goal" score cannot see around the corner.

What it changed

New design (v7):

[…]

* Added a "stuck detector" that fires when the drone has not reduced its distance to the goal over the last N ticks AND its velocity is small or it is pressed against a wall: on firing, it forces a 90-degree turn away from the wall for a few ticks (memory.escapeUntil).

* Added a "corner detector": if two perpendicular sensors are blocked (e.g. east and south both 0), the drone is in a corner; it must turn toward the third open direction, not slide along either wall.

Practice took the program from 0 to 8 of 12 practice routes
0%25%50%75%100%v1v2v3v4v5v6v7v8 Version of the program 0% 17% 8% 42% 17% 8% 67% 58% Version 7 went to the test
Best version so farScore of each new version

Rewriting its own code from the logs of its failed flights took Ripple’s program from 0 to 8/12 practice routes in six rewrites. Of the eight coding runs we charted, seven ended better than they began.

Explore the practice scores of all the coding runs

Each line is one coding run of 12 practice routes and eight versions. These are 8 of the 24 runs considered for the test. Choose a run to see its scores.

0%25%50%75%100%v1v2v3v4v5v6v7v8 Version of the program Practice routes finished 0%17%17%17%0%17%50%17%0%25%8%25%17%25%25%25%0%0%33%8%33%25%58%33%0%0%0%0%0%0%0%0%8%25%33%25%33%8%33%33%25%0%25%25%17%25%17%33%0%17%8%42%17%8%67%58%0%17%25%33%42%0%17%8%
Best version so farScore of each new versionThe other runs

Run 1 finished 0, 2, 2, 2, 0, 2, 6 and 2 of the 12 practice routes, version by version.

Run 2 finished 0, 3, 1, 3, 2, 3, 3 and 3 of the 12 practice routes, version by version.

Run 3 finished 0, 0, 4, 1, 4, 3, 7 and 4 of the 12 practice routes, version by version.

Run 4 finished 0, 0, 0, 0, 0, 0, 0 and 0 of the 12 practice routes, version by version.

Run 5 finished 1, 3, 4, 3, 4, 1, 4 and 4 of the 12 practice routes, version by version.

Run 6 finished 3, 0, 3, 3, 2, 3, 2 and 4 of the 12 practice routes, version by version.

Run 7, whose program went to the test, finished 0, 2, 1, 5, 2, 1, 8 and 7 of the 12 practice routes, version by version.

Run 8 finished 0, 2, 3, 4, 5, 0, 2 and 1 of the 12 practice routes, version by version.

Fixing one route could break another. In one run, Ripple changed how much clearance its program checked before a turn. That finished a previously stuck flight in ten seconds, but cut the practice goals it reached from five to two.

Ripple’s notes on a difficult corner

Flight 1 is the clearest example: at [-12.8,0.5,-1.4] the sensors show north = 6.3, north_west = 12, west = 12, and the goal is at [-10.6,-0.9], i.e. to the north-east. The policy returned vz = +4.42 (north) but the drone was sliding along the north wall; it needed to go *west* first (west = 12 m free) to get around the corner, then north. A pure "max alignment with goal" score cannot see around the corner.

From the selected program’s change log

How far can we push it?

Can it fly routes it has never seen?

We froze Ripple’s best program, so the model was out of the loop, and flew it on 48 factory routes it had never practiced. Here are four of the routes it finished.

1234
1Route 1
2Route 2
3Route 3
4Route 4

Ripple’s program reached the goal on 65% of flights it had never practiced, close to the 67% it scored on its practice routes.

Can it fly somewhere new?

We took the program out of the factory. In a second building it flew unchanged. In a canyon and among cottages, Ripple practiced again in each place. The clips show an example flight in each place.

Unchanged, the program reached 36% of its goals in the second building and 0% outdoors. After practice in each outdoor place, it reached 69% in the canyon and 76% at the cottages.

We also sent a bird after the drone. Ripple read its heading correctly from five positions in 39 of 44 plans, but its plans reached the goal on only 42 of 192 flights.

How did it compare with other pilots?

Two other pilots flew the same 96 flights: a pilot we wrote and tuned by hand, and Ripple steering live.

  1. Ripple’s programThe flight code Ripple wrote, flying with no model in the loop.
    65%
  2. A hand-written pilotAn obstacle-avoiding program we wrote and tuned ourselves.
    55%
  3. Ripple steering liveRipple reads the instruments and sends one command every second.
    44%
Can it get past an adversary?

We added an adversary, a bird that chases the drone. Ripple read two seconds of the adversary’s position as text, wrote down what it thought the adversary was doing, and chose a route, a speed and a dodging distance. A flight controller flew that plan.

Start Goal

What Ripple read

Two seconds of the adversary’s position, as text.

0.0 s   [10.4, 1.0, -12.9]
0.5 s   [10.0, 1.0, -13.5]
1.0 s   [9.5, 1.1, -15.2]
1.5 s   [9.1, 1.2, -17.2]
2.0 s   [8.6, 1.2, -19.2]

What Ripple wrote

“The bird's velocity is roughly (-1.7, 0.1, -2.5) m/s, moving toward the drone's current position, so it is pursuing. The drone should fly away from the bird's heading and use the building's floors and atrium to break line of sight. Route R1 is the shortest and avoids the bird's likely path, so it is chosen.”

Route R1 at 4.6 m/s, dodging inside 6 m.

What happened

Ripple read the adversary as chasing it, and it was. A flight controller flew the plan and did the dodging. Ripple was not told the wind.

Gap to the adversary
To the goal
Wind

From five positions, Ripple got the adversary’s heading right to within 30° in 39 of the 44 plans where it was moving. It named what the adversary would do next in only 12 of 48 plans, and its plans reached the goal on 42 of 192 flights. Eight practice encounters beforehand made no reliable difference.

In the factory Ripple read two seconds of the adversary’s position and chose a route, a speed and a dodging distance once per flight, after which a flight controller flew the plan. Those plans reached the goal on 42 of 192 flights, while flying the shortest route with no dodging reached it on 16 of 96. Most of the flights that failed ended against a wall rather than with the adversary catching the drone.

In the two-floor building Ripple chose a new waypoint every second. Across 72 flights against a programmed guard it reached the goal 30 times, and a pilot that always takes the shortest route reached it 31 times. When a second copy of Ripple flew the adversary, it caught the drone 18 times against the guard’s 42.

What did the outdoor trials test?

The navigation test used 48 new routes per place, each in 2 winds. In each place we picked one program from 24 independent coding runs using practice scores alone, and the average across all of those programs was much lower.

Practice in each place took the program from 0% to most of its goals

Zohar canyon

48 routes · 96 flights per pilot

  • Ripple’s factory program, unchanged0% 0 of 96 flights. 95% interval 0% to 0%.
  • Ripple’s program after practice here69% 66 of 96 flights. 95% interval 57% to 82%.
  • Tuned hand-written pilot56% 54 of 96 flights. 95% interval 45% to 71%.
  • Average of 24 programs practiced here28% 95% interval 22% to 34%.

Cottages

48 routes · 96 flights per pilot

  • Ripple’s factory program, unchanged0% 0 of 96 flights. 95% interval 0% to 0%.
  • Ripple’s program after practice here76% 73 of 96 flights. 95% interval 66% to 85%.
  • Tuned hand-written pilot76% 73 of 96 flights. 95% interval 67% to 85%.
  • Average of 24 programs practiced here36% 95% interval 32% to 40%.
Each dot marks the share of flights that reached the goal, and the thin line shows the 95% interval across routes.
The same numbers as a table, with the factory and the second building
PlaceFactory program, unchangedProgram practiced hereTuned hand-written pilotFlights per pilot
Factory65% [53%–78%]65% [53%–78%]55% [45%–69%]96
Zohar canyon0% [0%–0%]69% [57%–82%]56% [45%–71%]96
Cottages0% [0%–0%]76% [66%–85%]76% [67%–85%]96
Two-floor building36% [24%–53%]Not tested50% [38%–68%]80

The intervals come from resampling whole routes with both winds together, and the factory row of the table reuses the earlier test. The outdoor study specified 18 comparisons across navigation, search and delivery. We changed its analysis after seeing the results, using one-sided tests with a Holm correction for those directional comparisons. The gains over the unchanged factory program passed that correction; none of the comparisons with a tuned hand-written pilot did.

The clips in the tabs are examples, and none is a flight from the navigation test. The factory clip is a development flight that Ripple steered live. The canyon and cottage clips are demonstrations on a route and a pad we picked by eye, each flown by a frozen program from a single coding run rather than the program we tested: a navigation run along the well-scanned canyon floor, and a parcel delivery to the rear porch, where the simulator released the parcel after a steady two-second hover and checked where it landed. Both of those clips smooth the recorded positions for display, which removes the once-a-second wobble of the one-hertz commands; the start, the end and the parcel release are unchanged, and the smoothed paths stay within 1.5 m of the recordings and clear of the collision map. Scan scales are provisional, and every flight is simulated.

How we tested

Flights that reached the destination

Factory

48 routes · 96 flights per pilot

  • Ripple’s selected program65% 95% interval 53% to 78%.
  • Tuned hand-written pilot55% 95% interval 45% to 69%.
  • Ripple steering live44% 95% interval 34% to 55%.
  • Average of 24 programs43% 95% interval 35% to 52%.

Two-floor building

40 routes · 80 flights per pilot

  • Ripple’s selected program36% 95% interval 24% to 53%.
  • Tuned hand-written pilot50% 95% interval 38% to 68%.
  • Ripple steering liveNot tested here
  • Average of 24 programs36% 95% interval 28% to 45%.
Each dot marks the share of flights that arrived within one metre of the goal in 90 seconds. The thin line shows the 95% interval across routes.

We chose each pilot on 12 practice routes. For live steering we compared 13 prompts. For the hand-written pilot we tried 193 settings. From 24 coding runs, we picked the program that reached the most practice goals, breaking ties by progress. Both live Ripple and the frozen code chose a velocity every simulated second. During the frozen program’s flights there were no model calls.

We specified four comparisons before the test. The same route and wind were paired across pilots. To estimate uncertainty we resampled whole routes, keeping the two winds together, and adjusted the hypothesis tests for four comparisons.

Factory · 96 flights per pilot
PilotReached (95% interval)ProgressTime touching wallsReached without contact
Planner with the full map100% (100%–100%)96%5%68%
Ripple’s selected program65% (53%–78%)82%5%2%
Hand-written pilot, tuned55% (45%–69%)78%4%8%
Hand-written pilot, default47% (35%–63%)70%8%6%
Ripple steering live, selected prompt44% (34%–55%)72%34%1%
Ripple steering live, plain prompt33% (24%–47%)67%53%0%
Average across Ripple’s programs43% (35%–52%)69%20%3%
Straight-line flyer11% (3%–22%)43%88%2%
Two-floor building · 80 flights per pilot
PilotReached (95% interval)ProgressTime touching wallsReached without contact
Planner with the full map100% (100%–100%)95%0%99%
Ripple’s selected program36% (24%–53%)84%20%4%
Hand-written pilot, tuned50% (38%–68%)77%50%13%
Hand-written pilot, default36% (24%–53%)72%55%18%
Average across Ripple’s programs36% (28%–45%)81%30%7%
Straight-line flyer29% (16%–44%)71%75%1%

Against tuned live steering the selected program gained +21 pp (95% paired interval [+6, +35] percentage points). Against the tuned hand-written pilot it gained +9 pp ([−4, +22]). The factory comparison with live steering was the only one of the four to pass the adjusted test. The average across all programs includes one frozen program from every coding run, and that average reached 43%. Arrivals were rough: the tested program reached the goal without touching a wall on 2% of its factory flights.

A planner given the full factory map reached 100% of its goals. The other solo pilots used instrument readings. We ran Ripple-65B-Preview through OpenRouter on PrimeIntellect in bf16 with temperature 0.7 and thinking off. Replaying the recorded commands reproduced 5,120 of 5,120 final evaluation episodes exactly.

Replay three pilots from the same starting point

Route H22 comes from an earlier evaluation. The goal is 17 metres away in a straight line, with obstacles in between. All three flights used the same wind.

Ripple’s programLive steeringHand-written pilot
startgoalN5 m
The program reached the goal in 12.6 s with 2 wall contacts. Live Ripple with the plain prompt ran out of time 61% of the way there, with 101 wall contacts. The default hand-written pilot ran out of time 33% of the way there, with 59 wall contacts. Dark gray marks walls. Light gray marks clutter. Wind seed 830212.
Could better instructions help Ripple steer live?

On 48 development flights, we tried 13 changes including a local text map, flight history and collision warnings. None consistently brought the drone closer to the goal.

−40−30−20−100+10+20plain promptGoal bearing−11 ppList blocked directions−5 ppLocal text map−5 ppRecent flight history−5 ppAll four together+1 ppCompass commands−10 ppCollision warning−15 ppCompass commands + warning−32 ppRipple’s flight notes−6 ppGeneric flying advice−13 ppShort prompt−6 ppShort prompt + notes−8 ppTwo-second commands−12 pp
Progress measures the closest the drone came. Starting 20 metres away and getting within five counts as 75% progress. Red marks comparisons whose 95% interval lies entirely below zero.
ChangeWhat changedReachedProgress Δ
Goal bearingGive the direction and distance to the goal relative to the drone.29% → 25%−11 pp
List blocked directionsAsk Ripple to name the goal direction, list blocked directions and pick an open one.29% → 25%−5 pp
Local text mapDraw nearby obstacles on a 15 × 15 grid at the drone’s height.29% → 33%−5 pp
Recent flight historyGive the positions and number of wall contacts from the last ten seconds.29% → 35%−5 pp
All four togetherCombine the goal bearing, direction checklist, local map and recent flight history.29% → 35%+1 pp
Compass commandsAsk for a compass direction, speed and climb rate. The simulator converts these to a velocity.29% → 19%−10 pp
Collision warningGive the distance to the nearest obstacle and warn when the current velocity will lead to contact.29% → 21%−15 pp
Compass commands + warningCombine compass commands with the collision warning.29% → 8%−32 pp
Ripple’s flight notesInclude notes Ripple wrote after reviewing practice flights.29% → 38%−6 pp
Generic flying adviceSubstitute generic advice of the same length as Ripple’s notes to test the cost of extra text.29% → 31%−13 pp
Short promptShorten the rules and sensor readings and show only the previous command.29% → 29%−6 pp
Short prompt + notesAdd five facts condensed from Ripple’s flight notes to the short prompt.29% → 31%−8 pp
Two-second commandsHold each velocity for two seconds before asking for another move.29% → 29%−12 pp

On fresh factory routes, the practice-selected prompt reached 44% of goals and the plain prompt reached 33%. The difference was not significant.

So what does this mean?

A model trained on language can hold a working picture of three-dimensional space. Ripple has never seen an image or a map. Everything it knew about the factory reached it as numbers in text, and it steered by them: delaying its commands by one second cut its success from 29% to 8%.

It can reason about a place from a description alone. From the log of a failed flight, Ripple worked out that its program could not see around a corner. After six rewrites like that, the program went from 0 to 8/12 practice routes.

What it learned was general. The program reached the goal on 65% of flights over 48 routes it had never practiced, close to the 67% it scored on its practice routes, which a program that had memorised those routes could not do. In a canyon and among cottages the factory program reached 0%, and after practice there Ripple’s programs reached 69% and 76%.

Working only from text, Ripple reasons about space well enough to write a pilot that flies routes it has never seen, and to make that pilot better by reading where it failed.

Where this goes

Everything here happened in simulation, with a drone that senses distances and has no camera. The open question is how far this goes as language models get better at acting in physical space.

We also want to try flying in new places without practice and getting several drones to work together. We have search and rescue in mind.

We would love to see what you try. Send us a mission, a building or a better pilot.