The convolutional network that powered YOLO and its maritime descendants sees the image the way a microscope sees it — local windows, built up. The Vision Transformer instead attends everywhere at once: every patch of sky can inform every patch of sea. On terrestrial benchmarks the trade was decisive; attention outperformed convolutions once data and compute were sufficient.
Maritime vision sits in the opposite regime — small datasets, shipboard compute, domain shift — and the question of whether ViT's advantages survive those constraints is genuinely open in the literature.
Key findings
- ViT's global attention is theoretically well-matched to the maritime problem — the horizon, sea-state texture and target context are global features. Proposed
- ViTs are more data-hungry than CNNs; maritime corpora are 10–100× smaller than the benchmarks ViTs won on. Established
- Shipboard compute budgets (inference latency, power, no datacentre) favour efficient models — a constraint ViT variants are only beginning to address. Established
- Published maritime ViT work exists but is thin compared to the YOLO/CNN mainstream; no clear win is established. Research demonstration
- Hybrid architectures — convolutional locality plus transformer attention — are the direction the research is actually converging on. Research demonstration
Why attention should matter at sea
A maritime scene is a global-context problem in miniature. The horizon is the organising reference; sea-state texture tells which pixels are water and which are targets; a ship's class is readable from the spread of its parts across the frame. Local-window convolutions build that context indirectly through depth; attention reads it directly. On paper, the sea is exactly the picture ViT was designed for. Maritime-GPT analysis
The two constraints that bite
- <strong>Data scarcity.</strong> Transformers lack the convolution's built-in inductive bias toward local structure; they must learn it from data. Maritime corpora — tens of thousands of frames, not millions — are the regime where that bias was worth the keep. Established
- <strong>Compute.</strong> Attention scales quadratically in resolution; a shipboard GPU running a real-time fusion stack does not have datacentre headroom. The efficiency research (lightweight ViTs, windowed attention like Swin) is exactly what's being tested against the constraint. Established
- <strong>Domain shift amplification.</strong> A more expressive model can also memorise a narrower domain's quirks harder — generalisation across sea states is where the extra capacity can cost. Maritime-GPT analysis
| Architecture | Strength | Maritime weakness | Status |
|---|---|---|---|
| CNN / YOLO | Efficient, inductive local bias | Limited global context | Proven baseline |
| ViT / DETR | Global attention, strong scaling | Data-hungry, compute-heavy | Demonstrated, not dominant |
| Swin / windowed ViT | Local windows + hierarchical attention | Still data-hungry | The compromise being tested |
| CNN+ViT hybrid | Locality + global context | Complexity, tuning burden | Research frontier |
Where ViT already earns its place
Satellite and aerial maritime imagery — SAR ship detection, ocean-surveillance, high-altitude reconnaissance — is the regime where the dataset scale is better and the global context is most valuable. ViT and hybrid architectures are advancing there. The bridge-camera real-time problem is different: tighter compute, scarcer labels, and a domain where a CNN's bias may still be worth more than attention's flexibility. The honest summary is not 'ViT replaces CNN' but 'ViT leads where data exists, trails where it doesn't'. Maritime-GPT analysis
What would settle it
- A large bridge-perspective maritime dataset at the scale transformers need.
- Published shipboard-inference benchmarks — latency and power, not just mAP.
- Domain-shift evaluation: trained on one sea, tested on another.
What we know
- Transformers' global context is theoretically well-suited to maritime scenes.
- Maritime data scarcity and shipboard compute are the binding constraints.
- Published maritime ViT results exist but are not yet dominant over tuned CNN baselines.
What we don't
- Whether attention's advantage survives quantisation and the compute budget aboard.
- How ViTs generalise across the domain shifts that define maritime deployment.
- Whether a large-scale maritime pretraining corpus would flip the balance.
Being tested
- Lightweight/hybrid ViTs for shipboard inference.
- Self-supervised maritime pretraining to overcome label scarcity.
- ViT-based fusion heads over multi-sensor inputs.
Maritime-GPT analysis
The transformer doesn't lose at sea — it hasn't yet had the data to win. Where the datasets are large (satellite, surveillance), attention advances. Where they're scarce (bridge footage, small craft), the convolution's built-in bias is still doing more work than the transformer's flexibility can buy. The open question is whether maritime vision will ever generate the data scale ViT needs — or whether the domain will be won by hybrids that give the model the bias it can't afford to learn. Maritime-GPT analysis
This is a research feature, not operational guidance. Simulation results are reported as simulation; demonstrations are not deployments; company claims are attributed as such. Verify safety-critical decisions against official nautical publications and flag-state requirements.
Sources & evidence
The ViT paper — the architecture family maritime work is adapting.
The domain-constraint baseline any new architecture must beat.