CAIRN

Cross-Room 3D Scene Understanding with
Topology-Aware Large Multimodal Models

1University of Oxford 2Microsoft Research 3Simon Fraser University
†Part of this work was done during an internship at Microsoft Research.   *Corresponding author.
Hierarchical scene graph. Object tokens carry room-local relational context from a GNN; learned room tokens abstract each room.
Topology-aware attention. A hierarchical mask with geometric bias routes information along the house layout.
CAIRN-MR benchmark. 673 multi-room HM3D scenes, 238K annotations, six tasks from intra-room perception to cross-room reasoning.
Figure 1 · interactive Pick a task, or drag the house to look around
Assembling the house…

CAIRN-MR assembles full households from HM3D rooms (R1–R5). CAIRN reads the house as a room / object hierarchy and answers six task types. Every question is a real CAIRN-MR validation sample from this scene; grounding and QA answers are CAIRN's actual predictions (the captioning card shows the reference caption).

TL;DR — We introduce CAIRN, a topology-aware 3D-LLM that reasons over full multi-room households by aligning attention with scene hierarchy, together with CAIRN-MR, a benchmark of six tasks spanning intra-room perception to cross-room reasoning. CAIRN outperforms prior 3D-LLMs by a large margin on CAIRN-MR while staying competitive on five single-room benchmarks.

Abstract

Existing 3D scene-grounded large language models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the ability to reason over real-world household environments containing multiple interconnected rooms and diverse object categories. We introduce CAIRN, a topology-aware 3D-LLM for multi-room 3D scene understanding. CAIRN aligns transformer attention with scene hierarchy, giving the model explicit awareness of object-level relations and room-level connectivity. It enriches object tokens with room-local relational context via a graph neural network, introduces learned room tokens for room-level abstraction, and applies a hierarchical attention mask with geometric bias to route information according to scene topology. CAIRN is developed on CAIRN-MR, a benchmark we introduce on HM3D for multi-room 3D scene understanding, covering grounding, captioning, and four question-answering tasks that progressively evaluate from intra-room perception to cross-room reasoning. Experiments show that CAIRN outperforms prior 3D-LLMs by a large margin across all CAIRN-MR tasks while remaining competitive on five single-room benchmarks.

Click any icon to jump to the corresponding section.

CAIRN-MR Benchmark

0multi-room scenes
0rooms per scene
0objects per scene
0task annotations
0task types
CAIRN-MR benchmark: six task types over multi-room HM3D scenes with dataset composition charts.
Figure 2. CAIRN-MR covers grounding, captioning, and four QA tasks over assembled multi-room HM3D scenes. Right: benchmark scale and task composition. Click to zoom.
Visual groundingfind the object a free-form description refers to, among all rooms
Dense captioningdescribe an object and its spatial context
Object identificationwhich object satisfies relations inside an implicitly described room
Countinghow many instances a described room contains
Existence verificationis a category present in the described room
Inter-room comparisonwhich of two described rooms has more of a category

CAIRN-MR is built on HM3D residential environments and contains 673 multi-room scenes (on average 4.5 rooms and ~115 object instances each) with 238K task annotations. Crucially, the target room is never explicitly named: models must first localize the relevant room(s) from implicit object-level spatial constraints before performing downstream reasoning. The six tasks progressively evaluate from intra-room perception to cross-room comparison. Referring expressions are verified to be unique at the full-scene level, and QA tasks are calibrated against random, frequency, and human baselines (human accuracy 86.8–94.5%).

Method

CAIRN architecture: hierarchical scene graph construction, tokenization, and masked attention with geometric bias.
Figure 3. CAIRN builds a hierarchical scene graph (bottom), tokenizes it into object and room tokens (middle), and feeds them to an LLM with hierarchical masked attention and geometric bias (top). Click to zoom.

Given a 3D scene, CAIRN constructs a two-layer hierarchical scene graph capturing object relations and room adjacencies, tokenizes it into object and room tokens, and feeds them to an LLM with hierarchical masked attention and geometric bias. The mask routes information along scene topology — object tokens interact only within their room, while cross-room communication is mediated through learned room tokens — and learned bias terms inject spatial priors (relative pose, distance, room adjacency) into attention logits. This produces a block-sparse attention pattern that reduces object-object attention from O(N²) to per-room cost.

Try it: the hierarchical attention mask

room ↔ room mRR object ↔ own room mOR/RO object ↔ object, same room mOO blocked

Hover a row (or a node) to see what that token can attend to.

Results

Trained under the same two-stage protocol with a Qwen3-8B backbone, CAIRN achieves the largest gains on grounding (+5.4 Acc@0.25) and captioning (+14.9 CIDEr, ~19% relative) over the strongest baseline, with QA gains scaling with the degree of room-level reasoning required. CAIRN also matches or exceeds prior methods on five single-room ScanNet benchmarks, achieving the best or second-best results on 7 of 9 metrics.

Method Grounding
A@0.25 ↑
Grounding
A@0.5 ↑
Captioning
CIDEr ↑
Ident.
EM ↑
Count.
MRA ↑
Exist.
EM ↑
Comp.
EM ↑
Chat-Scene19.619.478.361.358.969.563.6
3DGraphLLM18.918.779.661.958.372.762.3
Inst3D-LMM20.420.079.561.858.571.663.7
CAIRN (ours)25.825.494.562.161.176.170.4
Δ vs. best baseline+5.4+5.4+14.9+0.2+2.2+3.4+6.7

Multi-room performance on CAIRN-MR. All methods use Qwen3-8B and the same two-stage training protocol.

Qualitative Visualization

Explore six CAIRN-MR validation households in 3D. Every question below is one CAIRN answers correctly and the Chat-Scene baseline‡ gets wrong. Selecting a question highlights the room(s) it implicitly describes and the objects the answer depends on: green for the ground truth (and CAIRN), red for the baseline's grounding prediction.

Loading scene…
drag to orbit · scroll to zoom · right-drag to pan
 

‡Chat-Scene architecture (no scene graph, no room tokens, no topology mask) re-trained with the same Qwen3-8B backbone and two-stage protocol as CAIRN. Point clouds are HM3D scans with ceilings cut away; room names R1–R5 are neutral labels.

Qualitative comparison on a multi-room scene with visually similar living rooms.
Figure 4. From the paper: on a multi-room scene with visually similar living rooms, CAIRN (green) grounds the correct object and resolves the cross-room comparison, where the best baseline, Inst3D-LMM (red), fails.

BibTeX

@article{liang2026cairn,
  title   = {CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models},
  author  = {Liang, He and Ma, Chenyang and Zhang, Yiming and Shin, Sangyun and Markham, Andrew and Trigoni, Niki and He, Yuhang},
  journal = {arXiv preprint arXiv:2607.06534},
  year    = {2026}
}