HarnessVLN
Unifying Training-Free Embodied Navigation
through an Agent Harness
Video overview
4 min 57 sec · English narration
Training-freeSuccess rate
Abstract
Embodied navigation requires an agent to interpret visual observations, accumulate spatial knowledge, and execute actions over long horizons. Existing training-free methods benefit from multimodal large language models, but their task-specific pipelines often lack explicit mechanisms to reconcile proposals with spatial evidence, task progress, and execution failures.
HarnessVLN introduces an Agent Harness that coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface. Hierarchical event memory tracks progress and execution history, while a persistent Spatiotemporal Graph organizes reusable evidence for target verification and recovery.
Method
Framework overview
The multimodal language model proposes operations. The Agent Harness assembles context, validates proposals, dispatches tools, and integrates execution feedback.
Hierarchical event memory
Working, progress, and reflection memory represent the current task context, completed subgoals, and execution failures.
Spatiotemporal graph
Places, entities, navigable relations, evidence provenance, and failure annotations persist across subgoals.
Validated execution
Operation proposals are validated against available evidence, geometric feasibility, task consistency, and prior failures.
Method
Unified tool interface
Eight tools provide a common interface for perception, retrieval, grounding, navigation, recovery, and termination.
01observePerception
Capture current RGB-D and pose.
02observe_panoramaPerception
Capture predefined headings.
03retrieve_memoryRetrieval
Recall task-relevant evidence.
04ground_targetGrounding
Map semantics to an image region.
05query_depthPerception
Estimate target depth and uncertainty.
06navigate_toNavigation
Execute a validated spatial target.
07backtrackRecovery
Return to a verified visited location.
08request_stopTermination
Verify completion before stopping.
Evaluation
Quantitative results
On the full val-unseen splits, HarnessVLN achieves 59.6% SR on R2R and 51.4% on RxR, exceeding HSGM by 11.7 and 9.6 percentage points. On R2R Rand100, the matched GPT-5.5 comparison improves SR by 3.0 points and OSR by 12.0 points over AgenticNav, while reducing NE from 5.19 m to 4.24 m.
Training-free SR59.6%
R2R full val-unseen
| Method | VLN-CE R2R | VLN-CE RxR | ||||||
|---|---|---|---|---|---|---|---|---|
| NE ↓ | OSR ↑ | SR ↑ | SPL ↑ | NE ↓ | SR ↑ | SPL ↑ | nDTW ↑ | |
| Supervised learning · Training-based | ||||||||
| NaVidRSS24 | 5.47 | 49.1 | 37.4 | 35.9 | 8.41 | 34.5 | 23.8 | — |
| Uni-NaVidRSS25 | 5.58 | 53.3 | 47.0 | 42.7 | 6.24 | 48.7 | 40.9 | — |
| NaVILARSS25 | 5.22 | 62.5 | 54.0 | 49.0 | 6.77 | 49.3 | 44.0 | 58.8 |
| ETPNavTPAMI24 | 4.71 | 65.0 | 57.0 | 49.0 | 5.64 | 54.7 | 44.8 | — |
| InternVLA-N1ICLR26 | 4.83 | 63.3 | 58.2 | 54.0 | 5.91 | 53.5 | 46.1 | 65.3 |
| NavFoMICLR26 | 3.78 | 70.8 | 61.7 | 55.3 | 3.83 | 64.4 | 56.2 | — |
| ABot-N0CVPR26 | 3.78 | 70.8 | 66.4 | 63.9 | 3.83 | 69.3 | 60.0 | — |
| Zero-shot · Training-free | ||||||||
| Open-Nav†ICRA25 | 6.70 | 23.0 | 19.0 | 16.1 | — | — | — | — |
| CA-NavTPAMI25 | 7.58 | 48.0 | 25.3 | 10.8 | 10.40 | 19.0 | 6.0 | 13.5 |
| InstructNav†CoRL24 | 6.89 | 47.0 | 31.0 | 24.0 | — | — | — | — |
| GC-VLNCoRL25 | 7.30 | 41.8 | 33.6 | 16.3 | 8.80 | 33.8 | 13.8 | — |
| Uni-LaViRA-GPT-5.5†arXiv26 | 6.11 | 58.0 | 42.0 | 25.7 | — | — | — | — |
| EvoNav-Gemini-2.5-pro†CVPR26 | 5.04 | 51.0 | 43.0 | 37.8 | — | — | — | — |
| SmartWay-GPT-5.5†IROS2025 | 5.16 | 60.0 | 44.0 | 35.0 | — | — | — | — |
| HSGM-GPT-5CVPR26 | 5.42 | 58.7 | 47.9 | 32.8 | 7.43 | 41.8 | 25.1 | 54.9 |
| mini-swe-agent-GPT-5.5†arXiv26 | 7.29 | 57.0 | 52.0 | 44.2 | — | — | — | — |
| AgenticNav-GPT-5.5†arXiv26 | 5.19 | 65.0 | 55.0 | 48.4 | — | — | — | — |
| HarnessVLN-GPT-5.5†Ours | 4.24 | 77.0 | 58.0 | 35.8 | — | — | — | — |
| HarnessVLN-GPT-5.5Ours | 4.08 | 70.7 | 59.6 | 43.6 | 6.50 | 51.4 | 38.1 | 55.6 |
NE is in meters; all other metrics are percentages. † denotes R2R Rand100; unmarked rows use the full val-unseen splits. Compare methods within the same split. Shading identifies our method and does not imply the best value in every column. — denotes an unreported metric.
Training-free SR76.0%
HM3D-v2
| Method | Training | HM3D-v2 | HM3D-OVON | ||
|---|---|---|---|---|---|
| SR ↑ | SPL ↑ | SR ↑ | SPL ↑ | ||
| Uni-NaVidRSS25 | Yes | 73.7 | 37.1 | 39.5 | 19.8 |
| FiLM-NavarXiv25 | Yes | 77.0 | 41.3 | 40.8 | 24.4 |
| MTU3DICCV25 | Yes | — | — | 40.8 | 12.1 |
| NavFoMICLR26 | Yes | — | — | 43.6 | 31.3 |
| ABot-N0CVPR26 | Yes | — | — | 54.0 | 30.5 |
| InstructNavCoRL24 | No | 58.0 | 20.9 | — | — |
| TANGOICRA25 | No | — | — | 35.5 | 19.5 |
| VLFMICRA24 | No | 63.6 | 32.5 | 38.5 | 22.2 |
| STEGNavarXiv26 | No | 69.4 | 28.2 | — | — |
| DRIVE-NavarXiv26 | No | 72.4 | 41.3 | 50.2 | 32.6 |
| MSGNavCVPR26 | No | 74.4 | 33.4 | 48.3 | 27.0 |
| HarnessVLNOurs | No | 76.0 | 37.9 | 59.3 | 36.6 |
All metrics are percentages. Training indicates task-specific navigation training. HarnessVLN has the highest listed training-free SR on HM3D-v2 (76.0%) and the highest listed SR / SPL on HM3D-OVON (59.3% / 36.6%). Efficiency gains vary across benchmarks.
Component ablation
Cumulative component contributions
Fixed 100-episode subsets use identical episodes, base models, and evaluation settings within each task. Adding memory, the ST Graph, and stop validation raises SR from 43.0% to 58.0% on R2R and from 45.0% to 55.0% on HM3D-OVON.
R2R success rate (%) · fixed 100-episode subset
| Cumulative configuration | R2R | HM3D-OVON | ||||
|---|---|---|---|---|---|---|
| SR ↑ | SPL ↑ | OSR ↑ | SR ↑ | SPL ↑ | OSR ↑ | |
| Base agent | 43.0 | 24.3 | 65.0 | 45.0 | 32.0 | 72.0 |
| + Event memory | 50.0 | 30.9 | 73.0 | 52.0 | 32.4 | 72.0 |
| + ST Graph | 56.0 | 32.4 | 76.0 | 53.0 | 34.2 | 71.0 |
| + Stop validation | 58.0 | 35.9 | 77.0 | 55.0 | 33.0 | 74.0 |
Each row retains the components enabled above it. These are incremental contributions, conditional on earlier components. Stop validation improves SR on both tasks; HM3D-OVON SPL decreases slightly.
Qualitative results
Simulation demonstrations
Recorded trajectories show egocentric observations, spatial maps, and task-state updates on four navigation benchmarks.
R2R01 / 04 · 30 s
Instruction following with stair traversal
The agent ascends the stairs and enters the first doorway on the right.
RxR02 / 04 · 31 s
Multi-stage instruction following
The agent follows an instruction involving an archway, a change of direction, and a dining-table landmark.
HM3D-v203 / 04 · 45 s
Object-goal navigation: bed
An extended search trajectory culminating in successful navigation to a bed.
HM3D-OVON04 / 04 · 30 s
Open-vocabulary navigation: picture
The agent explores the scene and navigates to a picture specified as the target.
Selected successful episodes from the September 9 demo set. Playback is temporally condensed; the HM3D-v2 clip also uses uniform frame subsampling. These examples illustrate behavior and do not constitute an aggregate evaluation.
Robot experiments
Real-world deployment
HarnessVLN is deployed on a 1.74 m AgiBot A3U humanoid equipped with stereo vision, 3D LiDAR, onboard NVIDIA Thor compute, and a local path planner.
01 / 03Combined navigation task
Stop at the entrance signboard, then continue to the vending machine through one shared Harness protocol.
BibTeX
@misc{harnessvln2026,
title={HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness},
author={Anonymous},
note={Manuscript under review},
year={2026}
}