EndoComparer is a vision-language model focused on cross-case surgical video comparison: given two surgical videos from different cases, it answers questions about whether and how they differ in anatomy, tools, actions and skill. EndoDiffVQA is the benchmark we built for this task.
Purpose: Surgical training benefits from explicit comparison between cases: in patient condition, in technique and in skill proficiency. Surgical vision-language models (VLMs) offer a promising and flexible interface for this purpose. Yet current VLMs typically see one video at a time, and they often fall short when asked to compare videos.
Methods: We propose EndoComparer, a VLM for comparative surgical video question answering with two components. (a) A Pseudo-Siamese Visual LoRA (PSVL) gives each video slot its own low-rank adapter over a shared frozen vision encoder, separating the two videos in the visual pathway at the cost of a single encoder pass. (b) A Perceive-then-Compare chain of thought (PtC-CoT) trains the model to first state what each video shows, then derive the answer from these statements. We further curate EndoDiffVQA, a benchmark of nearly 11K multiple-choice questions on video pairs over cholecystectomy and gastric bypass procedures, covering anatomy, tool, action and skill.
Results: EndoComparer outperforms strong generalist and surgical VLMs on detecting and specifying differences between videos. Ablations show that the two proposed components contribute complementarily, and the model remains more robust under a video-swap control that exposes confusion between the two videos.
Conclusion: Providing a dedicated visual reasoning path through PSVL and PtC-CoT makes cross-case comparison of surgical videos more robust for VLMs. EndoDiffVQA and EndoComparer provide a benchmark and a strong baseline for this task.
| Model | 4-subset macro | Strict per subset | |||||
|---|---|---|---|---|---|---|---|
| Detect | Specify | Strict | Chole-Cap | LRYGB-Cap | Chole-Skill | LRYGB-Skill | |
| Zero-shot | |||||||
| Qwen3.5-9B | 49.8 | 57.3 | 35.4 | 41.3 | 34.4 | 31.6 | 34.3 |
| Qwen3.8-27B | 52.8 | 58.9† | 35.2 | 38.7 | 33.8 | 44.7 | 23.6 |
| InternVL3.5-8B | 49.7 | 54.5† | 30.6 | 44.7 | 25.5 | 28.6 | 23.6 |
| InternVL3.5-30B | 56.6 | 57.1 | 37.2 | 44.5 | 32.0 | 36.3 | 36.1 |
| SurgVidLM | 51.5 | 49.2† | 32.1 | 32.5 | 24.9 | 39.2 | 32.0 |
| GPT-5.5 | 49.5 | 56.3 | 32.9 | 45.8 | 25.9 | 35.2 | 24.7 |
| Gemini 3.6 Flash | 59.1 | 75.1 | 46.9 | 53.9 | 46.9 | 43.1 | 43.7 |
| Fine-tuned on EndoDiffVQA | |||||||
| InternVL3.5-8B | 61.6 | 85.3† | 47.0 | 66.3 | 51.7 | 46.4 | 23.6 |
| SurgVidLM | 60.0 | 88.3† | 45.1 | 62.6 | 52.2 | 38.0 | 27.5 |
| SurgTEMP | 60.9 | 88.9† | 46.8 | 64.3 | 52.9 | 42.6 | 27.5 |
| SurgViVQA | 52.4 | 37.5 | 32.6 | 32.9 | 33.1 | 36.9 | 27.4 |
| PeskaVLP | 61.3 | 63.3† | 44.6 | 53.9 | 44.8 | 47.7 | 31.9 |
| SurgCLIP | 61.4 | 57.3 | 43.5 | 54.0 | 45.4 | 47.5 | 27.2 |
| Qwen3.5-9B (backbone) | 63.6 | 77.6 | 48.3 | 64.6 | 49.4 | 43.3 | 31.1 |
| w/o PSVL | 64.0 | 78.0 | 49.5 | 65.3 | 51.6 | 44.9 | 33.4 |
| w/o PtC-CoT | 65.6 | 80.3 | 50.9 | 68.3 | 52.5 | 43.3 | 32.6 |
| EndoComparer (ours) | 67.7 | 83.5 | 54.0 | 69.6 | 54.5 | 51.9 | 40.1 |
2,300 test items, %. Detect, Specify and Strict are 4-subset macros; the last four columns give Strict per subset. † Specify macro over fewer than four subsets, because the model called fewer than 20 items Different in the others; not highlighted. Best per group in bold; the fine-tuned group includes ours. Standard deviations are given in the paper.
| Subset | Procedure | Source annotation | Questions | Clip length |
|---|---|---|---|---|
| Chole-Cap-DQA | Laparoscopic cholecystectomy | CholecT50 scene captions | Anatomy · Tool · Action | seconds to tens of seconds |
| LRYGB-Cap-DQA | Roux-en-Y gastric bypass | MultiBypass140 scene captions (newly curated) | Anatomy · Tool · Action | seconds to tens of seconds |
| Chole-Skill-DQA | Laparoscopic cholecystectomy | CholeScore OSATS-style criteria (1–5) | Skill | tens of minutes |
| LRYGB-Skill-DQA | Roux-en-Y gastric bypass | MultiBypass140 GOALS domains (newly annotated) | Skill | tens of minutes |
Is the falciform ligament visible in both videos?
Is the gallbladder the same color in both videos?
Is the gallbladder in the same part of the view in both videos?
Is the cystic artery cut in both videos?
Is a hook used in both videos?
Does the grasper enter from the same side in both videos?
Do both videos show the same number of instruments engaging the gallbladder?
Is the grasper actively engaged with the cystic pedicle in both videos?
Is the same interaction with the tissue shown in both videos?
Does a grasper hold the same part of the gallbladder in both videos?
Is the surgeon pursuing the same goal in both videos?
Is Calot's triangle exposed to the same standard in both videos?
Is two-handed coordination of the same standard in both videos during the gastric pouch creation?
| Gemini 3.6 Flashzero-shot | (b) D3 substitute | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (c) D3 substitute | ✗ |
| EndoComparerours | “Video 1: visible. Video 2: visible. (a)” | ✓ |
| Gemini 3.6 Flashzero-shot | (b) D1 swap | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (b) D1 swap | ✗ |
| EndoComparerours | “Video 1: blue-green. Video 2: light yellow. (c)” | ✓ |
| Gemini 3.6 Flashzero-shot | (c) D1 swap | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (c) D1 swap | ✗ |
| EndoComparerours | “Video 1: left. Video 2: center. (d)” | ✓ |
| Gemini 3.6 Flashzero-shot | (a) D2 collapse | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (a) D2 collapse | ✗ |
| EndoComparerours | “Video 1: coagulated. Video 2: not coagulated. (b)” | ✓ |
| Gemini 3.6 Flashzero-shot | (c) D3 substitute | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (a) D3 substitute | ✗ |
| EndoComparerours | “Video 1: used. Video 2: used. (b)” | ✓ |
| Gemini 3.6 Flashzero-shot | (c) D2 collapse | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (c) D2 collapse | ✗ |
| EndoComparerours | “Video 1: right. Video 2: right. (d)” | ✓ |
| Gemini 3.6 Flashzero-shot | (a) D1 swap | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (a) D1 swap | ✗ |
| EndoComparerours | “Video 1: 0 instruments. Video 2: 1 instrument. (b)” | ✓ |
| Gemini 3.6 Flashzero-shot | (a) D4 absent | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (c) D1 swap | ✗ |
| EndoComparerours | “Video 1: not engaged. Video 2: engaged. (b)” | ✓ |
| Gemini 3.6 Flashzero-shot | (d) D4 absent | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (b) D1 swap | ✗ |
| EndoComparerours | “Video 1: a stapler grasps the stomach. Video 2: an energy device dissects the stomach. (a)” | ✓ |
| Gemini 3.6 Flashzero-shot | (a) D1 swap | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (a) D1 swap | ✗ |
| EndoComparerours | “Video 1: proximal part. Video 2: proximal part. (d)” | ✓ |
| Gemini 3.6 Flashzero-shot | (a) D4 absent | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (b) D1 swap | ✗ |
| EndoComparerours | “Video 1: cutting the biliary limb. Video 2: suturing the gastrojejunal defect. (d)” | ✓ |
| Gemini 3.6 Flashzero-shot | (b) D1 swap | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (b) D1 swap | ✗ |
| EndoComparerours | “Video 1: score 3. Video 2: score 5. (c)” | ✓ |
| Gemini 3.6 Flashzero-shot | (d) D1 swap | ✗ |
|---|---|---|
| Qwen3.5-9Bfine-tuned backbone | (c) D2 collapse | ✗ |
| EndoComparerours | “Video 1: score 5. Video 2: score 3. (a)” | ✓ |
To be added.