net-inspection-cv

CI Python Tests Lint License

🐟 Open-source computer vision for fish-farm net inspection

Free, MIT-licensed tooling for net-damage inspection in aquaculture, built on Norwegian open data and released for anyone working on oppdrettsfisk along the coast — farmers, ROV operators, suppliers and researchers alike.

Why nets specifically: across 305 Norwegian escape incidents from 2010–2018, holes in nets accounted for 75% of all escaped fish and 44% of incidents (SINTEF 2019:00669), and SINTEF's first-listed countermeasure is verbatim "hyppig og god inspeksjon av not" — frequent, good net inspection. Escaped farmed fish is a shared environmental problem, so the tooling for catching it earlier should be shared too.

Built entirely on public sources: SOLAQUA ROV footage (SINTEF Ocean, CC BY-SA 4.0), the Fiskeridirektoratet locality register and MET Norway ocean forecasts — all NLOD/CC-licensed, none requiring a key. Contributions, corrections and real labelled damage data are all very welcome.

What this is not: a validated product. Every damage number here comes from synthetic damage; performance on real holes has never been measured, and the repository says so everywhere it matters. It is a working pipeline and an evaluation discipline, offered as a starting point rather than an answer.

A research-grade computer-vision toolkit for detecting possible damage (holes, tears, abnormal regions) in aquaculture fish-farm net imagery, built as an honest exploration framework. It ingests images / video / ROS bags, preprocesses underwater footage, and compares five detection approaches — a classical OpenCV heuristic, a hand-crafted anomaly model, a foundation-model anomaly detector (PatchCore), a supervised YOLOv8 detector/segmenter, and a det∧seg agreement ensemble — with a rigorous, adversarial evaluation, temporal video reasoning, an interactive web console, ONNX export, real-data (SINTEF SOLAQUA) ingestion, and CI. Its defining feature is intellectual honesty: it never claims real-world damage-detection performance it cannot prove, and it reports the experiments that failed alongside the ones that worked.

This is five things, not one. Detection is the most visible but not the largest, and the rest is easy to miss in a README this long:

temporal confirmation, live

Real SOLAQUA ROV footage, the shipped detector, 44 consecutive frames. Left: every frame scored on its own. Right: the same stream, where a detection must survive 3 frames before it becomes an alert. 80 raw detections become 24 — a 70% cut — and because this net is undamaged, every box on the left is a false alarm and the right-hand panel going quiet is the system working.

Look at what it fires on: the thin bright mooring cords rigged around the calibration markers. Not the mesh. That is the whole false-alarm story in one loop, and finding it is why the headline number in this README was corrected downwards rather than up.

Regenerate: python scripts/make_temporal_gif.py

Where to look: code in src/netinspect/, CLIs in scripts/, the full write-up in reports/PROTOTYPE_REPORT.md, result tables/figures in reports/results/, models in models/, and the web UI in web/. For the reasoning trail: the research decision log (what external research was adopted vs rejected and why) and a code-grounded technical Q&A. For deployment: model card, deployment runbook + SLOs, data-collection protocol, and the honest production-readiness scorecard.

Status: prototype. The repository ships with a small synthetic dataset so the full pipeline runs out of the box, and an integration with the real SOLAQUA ROV dataset (SINTEF). Synthetic results only verify that the code path works. SOLAQUA is real footage but of undamaged nets with no damage labels, so it is used for real-image preprocessing, false-positive / robustness analysis, and anomaly detection — not for measuring damage detection accuracy. Validated damage-detection numbers still require real, labelled footage (see What a farm operator would need to provide).

what the detector fires on

Two clips, same day, same site, same camera, near-identical standoff — and both show undamaged net, so every box is a false positive. The top clip produces a 31% false-alarm rate; the bottom produces zero.

The boxes do not land on the calibration markers. They land on the thin bright mooring cords rigged around them — elongated high-contrast structures that resemble the synthetic tears the detector was trained on. The bottom clip also carries hardware (floats, smaller markers) further away and fires nothing, so the trigger is not "equipment in frame" but a particular shape at a particular scale.

Reproduce: python scripts/make_failure_figure.py

what drives false alarms

638 real frames, 4 clips, 2 days. A pre-registered hypothesis — that the long-running "different-day" gap was an operating-envelope violation driven by standoff distance — was tested and rejected: the clip spanning the widest standoff range (0.19–1.31 m) produces zero false alarms, and standoff shows no within-clip association on the held-out day (all p > 0.67).

The last three rows matter: frames within a clip are strongly correlated, so a per-frame confidence interval overstates how well any of this generalises to a new net. Reproduce: python scripts/analyze_operating_envelope.py

site planning

Detection models answer "is there damage in this frame". An operator also needs "is it worth flying today, and was the footage captured under conditions where that answer means anything". This joins the Fiskeridirektoratet cod-locality register to the MET Norway ocean forecast — both open, neither needing a key — to rate inspection windows per site, and shows the size-dependent cod thermal optimum against today's sea temperature.

Reproduce: python scripts/site_planner.py --operator ODE --forecast

A localhost inspection console (FastAPI + a custom web UI) ties it all together and is built to be self-explanatory: each frame source carries a one-line "what am I looking at", each detector a plain-language explainer, and an OOD-gate toggle shows a live in-distribution / review-needed badge per frame (the deploy-time safety net). Browse real SOLAQUA frames, switch detector (classical / anomaly / PatchCore / YOLO / ensemble), adjust confidence live, and compare methods — with a detection table and latency. python scripts/serve.py then open http://127.0.0.1:8000.

web console

(Shown: YOLO flagging 4 damage regions on composited-on-real net. On real undamaged frames it correctly shows zero — consistent with the adversarial eval.)

The console has three input modes:

Enter 0 for the first USB camera, an rtsp://... URL for an IP camera or ROV feed, or a path to a video file (which can loop, so a recorded clip stands in for a camera). Annotated frames arrive in the browser as MJPEG — no plugin, no client-side decoding, and a page refresh just reconnects.

python scripts/serve.py # then open the Live tab
python scripts/live_inspect.py --source 0 --display # or headless/desktop
python scripts/live_inspect.py --source rtsp://CAM/stream --method ensemble --out outputs/live

Both paths drive the same netinspect.live module, so the browser and the desktop runner cannot drift apart.

What makes it real-time rather than just "runs on video":

• Capture and inference run on separate threads, and stale frames are dropped. A camera delivers frames on its own schedule; a model does not. The inference thread always takes the newest frame and discards the backlog. Measured on a×ばつ720 clip: 1359 frames captured, 138 inferred, 1219 dropped, 22.5 fps, 38 ms latency. A single-loop design would instead have shown smooth video of a steadily older world.
• One alert per persisting defect, not one per frame. Temporal confirmation requires a detection to survive N frames. On the 31%-false-alarm clip above, that reduces 40 frames of flicker to zero confirmed alerts.
• Unfamiliar frames are flagged, not silently scored. The OOD gate marks a new site or camera for human review instead of producing confident boxes on a domain nothing here was characterised on.

A live feed from a real net will trip the OOD gate, and that is the correct behaviour, not a bug. These models learned synthetic damage. Run in shadow mode and fine-tune on real labels before anything alerts on its own.

Nobody can send a diver to "frame 1247". Every detector in this repo — and most of the literature — reports damage in frame coordinates, which is not an actionable answer. This maps detections onto the net itself: metres along the sweep, metres across it, depth, and a physical size in millimetres.

inspection map

python scripts/map_inspection.py --clip 2024年08月22日_14-29-05

107 per-frame alerts became 6 distinct sites — ×ばつ fewer things to look at, one of them supported by 72 sightings from different viewpoints.

Three things here are not the typical approach:

• Scale calibrates itself — no chessboard, no camera intrinsics. Total visual displacement over the pass is compared against the distance telemetry says the vehicle travelled. On the reference clip: 1.36 mm/px at 1 m, i.e. 0.83 mm/px at the observed 0.61 m standoff. The check that this is measuring something real rather than fitting noise is the implied 82° horizontal field of view — a sane number for an underwater camera. Ground sampling distance is then propagated per frame, so a detection seen from 1.4 m is not mistaken for the same physical size as one seen from 0.6 m.
• Spatial confirmation, which is strictly stronger than temporal. Temporal tracking loses a defect the moment the camera pans away. A position on the net survives the vehicle leaving and coming back, so repeat sightings from different angles become evidence that a thing is real and distinct.
• Coverage is reported, so a clean result means something. The map names the bands of net the camera never photographed. Without that, "no damage found" and "we never looked there" are indistinguishable — the failure mode that makes a clean inspection report dangerous.

where the camera flew

The same pass in three dimensions — and every axis is measured, none reconstructed: along-track from visual odometry, standoff from the net-plane sensor, depth from the pressure sensor. It shows the vehicle flying alongside the wall at 0.23–0.88 m off the net, with sites pinned on the wall rather than on the path. That matters operationally, because standoff is what sets ground sampling distance: the same defect at 0.88 m is resolved at half the detail it is at 0.23 m, so how the pass was flown determines what the pass could possibly have found.

Why the wall is drawn flat, and what a real 3-D pen would take. It would be easy — and dishonest — to wrap this strip onto a cylinder and call it a net model. Over this 5.5 m arc the deviation from a straight line sits inside USBL noise, so the pen radius simply is not in the data; fitting a cylinder would be inventing geometry rather than measuring it. Depth varies by only 0.15 m across the whole pass, so this is one horizontal band at one depth, not a pen. A genuine 3-D net model needs (a) a full circumnavigation so the curvature exceeds the noise, (b) passes stacked at multiple depths to cover the wall vertically, and (c) an absolute anchor — a USBL fix or a fiducial on the net — to register separate passes to each other. SOLAQUA has none of the three, so the code maps one pass honestly and the map format is per-pass, ready to compose when an anchor exists.

The map and the 3-D pass above are static figures. The console turns them into something you can fly around: python scripts/serve.py → the Net 3D tab.

the cage, turning

A full orbit of the console's 3-D view, rendered by the viewer itself (python scripts/make_cage_gif.py) rather than screen-recorded — so it cannot drift from the code. The dashed cage is declared by the operator; the solid band and the orange sites are measured from the footage. The inspected band really is that small: 5.5 m of a 160 m ring.

the 3-D cage view

Both panels are drawn by the console's own renderer, exported headlessly (node web/net3d.render.mjs) so the figure cannot drift from the code. Top: the whole cage, with the inspected band a sliver on the north wall and the feed barge moored alongside. Bottom: flown to the best-evidenced site.

Drag to orbit, scroll to zoom, click a marker to see the actual photograph of what was detected, and the camera flies to that panel of the net. A real Norwegian sea cage is modelled — a floating collar, a cylindrical net wall, and the cone that tapers to the centre weight — with the feed barge moored alongside as the landmark everything is described against. So a site does not read as "3.1 m along the sweep" but as:

3 m clockwise around the ring from the feed barge, 1.7 m deep on the wall.

Cage dimensions (circumference, wall depth, cone depth, barge bearing, the bearing the pass started at) are yours to set in the panel — a pen is a purchased object with a stated circumference, so the operator knows them even though the footage never will.

The rendering enforces the honesty, it does not just caption it. Declared geometry — the cage shell, the barge — is drawn thin, muted and dashed. Measured geometry — the swept band, the defect sites — is solid and saturated. The two never share a colour or a line weight, so the picture cannot quietly imply a net was reconstructed when it was not. That rule is covered by a test, not just a convention: node web/net3d.test.mjs asserts the shell is dashed and the band is filled, and CI runs it.

The viewer is ~450 lines of hand-rolled canvas 3-D with no dependency — no WebGL library, nothing from a CDN — so the console still works offline on a boat.

And it changes the headline. Placing 5.5 m of swept net on a real 160 m cage turns a result that sounds thorough into one that is honest:

0.077% of the cage was looked at — 3.5 m2 swept of 4,589 m2 of netting (wall + cone) · 3.4% of the ring · ~30 passes to circle it once, at this depth alone.

That number is the strongest argument in the repo for why a single clean pass is not a clean net, and it only exists because the pass is placed on a whole cage.

Live wiring. Tick "wire the live feed in" before starting a live session and the same visual odometry runs on the incoming stream, so a confirmed defect is placed on the cage as it is found — drawn dashed, because live scale rests on a declared standoff rather than telemetry. Positions are as good as that number and the UI says so.

Why feature matching works here at all: a net mesh is repetitive, which is normally fatal to feature matching. Biofouling supplies the non-repeating texture that saves it — measured on real SOLAQUA frames, ~1200 RANSAC inliers per consecutive pair at a 79% inlier ratio.

Why vision and telemetry are split the way they are: frame-to-frame, visual motion and integrated net-relative velocity correlate at only r = +0.26 — too weak to fuse per frame without injecting noise. In aggregate they agree, so telemetry supplies scale and anchoring and vision supplies motion. That division is a measurement, not a design preference. The independent cross-check: visual path 6.29 m vs telemetry 5.88 m over the pass (ratio 1.07).

What this is not. It maps the inspected strip, not a pen. A 3-D model of a whole net needs a full circumnavigation; one clip's arc is far too straight to even recover the pen radius — over a 6 m sweep the deviation from a straight line sits inside USBL noise, so fitting a cylinder to it would be inventing geometry. Positions drift with distance from the start and are reported with an error bar (~5% of distance travelled), never as exact points. And on SOLAQUA the net is undamaged, so every site mapped above is a false positive — the map is the mechanism; the damage is synthetic.

Measured across 557 real undamaged frames — every detection is a false alarm by construction — plus the labelled held-out split for recall:

This corrects what an earlier version of this README implied. The AGPL-free path is not simply a worse model bought for a better licence — on the axis this project argues decides adoption, it is ×ばつ better, cutting the worst clip from 31% to 10%. It costs 12 points of recall. Which operating point is right depends on whether a missed defect or an ignored alarm is more expensive at your site, and netinspect gate lets you write that down and enforce it.

Physics-based water augmentation (netinspect.water) synthesises Jerlov water types onto training frames — the idea behind UWCNN's training set, reimplemented from the model rather than the code. The hypothesis was that the between-clip false-alarm spread is a water-condition sensitivity.

It is not. With 60% of frames degraded: worst clip unchanged at 10%, held-out day 0% → 2%, recall flat at 88%. An earlier attempt with photometric jitter also failed to improve on it (17.5% → 18.5%).

Both attempts above tried to make the detector robust to how a scene looks. Neither ever showed it the thing it actually fires on. So the obvious next lever — and one that needs no damage labels at all, because every frame in bag3 is undamaged net and every detection there is a false positive by construction — is to train on those frames as explicit negatives.

It made things worse, reproducibly (eval_hard_negatives.py):

Recall was unchanged at 83.3%, so this is not a precision/recall trade — it is strictly worse. Every treatment seed scored worse than every baseline seed, so the ranges do not overlap and this is not seed noise.

Two controls make that readable. The negatives are split from the mined clip temporally with a discarded buffer, because adjacent video frames are near-duplicates and a random split would report memorisation as generalisation. And optimizer steps are matched, not epochs — adding 119 frames makes an epoch ×ばつ longer, and the first version of this experiment made exactly that mistake, appearing to gain recall that was really just more training. That run was discarded rather than published.

Three independent strategies have now failed to close this gap, one of them the strategy that most directly targets it. That is the strongest evidence in the repo that the between-clip spread is not a data-diversity problem — so the next step was to stop feeding the model pixels and find out what it is firing on.

All three are kept and documented because a failed hypothesis that was actually measured is worth more than an untested one that sounds plausible.

analyse_false_alarms.py measured all 109 false-alarm boxes against the 284 labelled damage boxes, and refuted this repo's own stated explanation. The culprit is the mooring cords — a contact sheet of the 24 highest-scoring false alarms shows bright rope in 22 of them — but they are bright, and the damage the detector trained on is dark:

The 24 highest-scoring false alarms on bag3: bright mooring rope, not dark holes

The model is firing across the polarity boundary — most likely on the oriented-edge signature a bright rope and a dark tear share, since both interrupt an otherwise regular mesh. That also explains all three failures above: two changed appearance without changing edge structure, and the third asked the model to suppress the very signature that marks real damage too.

And the two populations are separable with no retraining at all. A hole shows unlit water and is dark; a rope is an object in front of the net and is bright. netinspect.polarity drops detections brighter than the net around them (sweep):

Halving the false alarms is free; a ×ばつ reduction costs 12 points of recall. After three failed attempts to fix this with more data, the thing that worked was one afternoon of measuring what the model actually does.

Read the recall column sceptically. It is measured on synthetic damage, which the generator paints dark by construction — the same assumption the filter makes, so it is being graded on its own premise. The false-alarm column is real net and stands on its own; the recall column only shows the filter does not obviously break the detector. Real damage is not always dark (backscatter, a fish behind the hole, a partly-occluded tear), which is why this is off by default and opt-in, and why validating it needs real labelled damage this project does not have.

The underwater-vision literature is full of enhancement methods — UWCNN, colour correction, dehazing — and they unarguably produce nicer pictures. Before adopting one, this project asked the only question that matters here: does the detector get better?

It does not. Applied at inference to a detector trained on raw frames, which is how a preprocessing step would actually be deployed:

199 real undamaged SOLAQUA frames for the false-alarm rate, 29 labelled frames for recall, conf ≥ 0.25.

Recall was already saturated, so there was nothing to gain — and contrast stretching amplifies exactly the thin bright mooring cords this detector already false-fires on. The mechanism and the failure mode are the same one the corrected 31% headline is about.

The lesson generalises past this repo: enhancement that helps a human read an image is not the same thing as enhancement that helps a model, and bolting it in front of a model trained without it introduces a distribution mismatch that costs more than the enhancement gains. If you want the benefit, enhance the training set and retrain — do not add it as a deployment-time filter.

The split is mechanistic, not incidental. Contrast operations (CLAHE) hurt; colour-cast operations do not. Red-channel compensation is the only variant that reduces spurious boxes at all — 95 against raw's 107 — because it borrows structure from the surviving green channel instead of stretching local contrast into the bright rigging. It is still not adopted at inference: +1% on 199 frames is inside the noise, and the frames-alarming rate is what decides whether an operator trusts the system.

Reproduce: python scripts/eval_preprocessing.py

A tool-calling assistant answers operational questions over the repo's real artifacts (inspection results, ROV telemetry, the evidence ledger) and cites the file behind every number. Its defining behaviour is what it won't do: asked "how accurate is this on real damage?", it states that recall on real damage has never been measured rather than reaching for the synthetic-proxy F1.

That is enforced in two independent layers — a machine-readable evidence ledger compiled into the system prompt, and a deterministic post-check that flags an answer touching unvalidated capability without the caveat. The second layer is plain string matching precisely so it holds when the model doesn't, and is testable with no API key.

python scripts/ask.py --backend ollama "How accurate is this on real damage?"
python scripts/eval_assistant.py --backend ollama --model qwen3:14b

The backend is swappable — Anthropic, local Ollama, or any OpenAI-compatible endpoint — with identical tool schemas and an identical guardrail. So the same 12-case adversarial suite turns a design claim into a measurement, across six models, three families and a ×ばつ size range:

Two findings, and the second is the one I did not expect.

Boundary disclosure held at 100% on every model, down to 3B. The safety property — refusing to answer past the evidence — is carried by the system, not by the model's capacity. That is the claim this architecture makes, and a 3B model honouring it is much better evidence than a frontier model doing so. Caveat stated plainly: 5 boundary cases per model is a small denominator, and 100% of 5 is not 100% of 500.

Tool grounding does not track model size at all. An 8B scores 100% while both 15B models sit at 50%; a 3B beats a 7B of the same family. Whatever decides whether a model verifies rather than answering from its prompt, it is not parameter count — so "use a bigger model" is not the fix, and picking one on size alone would have chosen worse here. Every remaining failure is the same mode: a correct answer produced without checking. Right answer, wrong process.

Two of those "failures" were originally mine, not the models': a must_mention check on the bare substring "not" scored a correct "the hypothesis has been rejected" as a miss, and the missing-data marker list lacked "not known". Both were corrected and the saved answers re-scored with --rescore, which calls no model — fixing a brittle check and quietly re-running the models would have reported different numbers for the wrong reason.

YOLOv8 detecting damage on real net (live scripts/infer.py output, confidences shown):

yolo detection

Live stream (scripts/live_inspect.py) — real-time overlay with temporal confirmation + OOD gate:

live inspect

Status bar shows method · confirmed-defect count · OOD verdict. Here the ensemble boxes a damage tear (0.96) and correctly ignores the instrument housing, while the OOD gate flags this different-day frame for review — the deploy-time safety net.

YOLOv8 trained on synthetic damage composited onto real net, evaluated on a held-out clip:

Method comparison (synthetic damage on real backgrounds; IoU 0.30, class-agnostic):

Adversarial "is it cheating?" check — the trained YOLO on real undamaged net (no damage present, so any detection is a false alarm):

Read the third row before the fourth. An earlier version of this table omitted bag3 entirely — the evaluation set happened to contain the two clean same-day clips and not the dirty one, so "0% on its own training backgrounds" was true of the subset and false of the day. That is a sampling artifact of my own making, the same class as the 1%→11% one recorded in §5.7, and it is the third such correction in this project. All four undamaged clips on disk are now evaluated (scripts/adversarial_eval.py).

The corrected numbers say something more interesting than the original claim did. bag1, bag2 and bag3 are the same day, same site, same camera, flown at near-identical commanded standoff (0.65 / 0.60 / 0.60 m — verified against the ROV telemetry in scripts/extract_telemetry.py). Two of them produce zero false alarms and one produces 31%. The between-clip spread on one day is roughly thirty times the different-day effect. Whatever these models are sensitive to, it is a property of the scene, not of the calendar — which is why the pre-registered hypothesis that standoff distance explained the different-day gap was tested and rejected (scripts/analyze_operating_envelope.py).

It also means the per-frame confidence intervals in this repo are too narrow. Frames within a clip are strongly correlated (ICC ≈ 0.31 for the detector), so 638 frames carry roughly 13 frames' worth of independent information. The honest interval on the detector's overall false-alarm rate is the clip-clustered [0.00, 0.25], not the naive per-frame [0.08, 0.12].

Temporal confirmation removes a further 70% of transient false alarms on real undamaged video.

Closing the different-day gap — a precision/recall trade-off (measured on 200 frames). The nano segmenter fires on 18% of different-day undamaged frames (vs the detector's 1%). Two honest options, opposite ends of the operating curve:

The ensemble (detector proposes, segmenter confirms) drives false alarms to 0.5% — one frame in 200, not zero — and adds masks, but inherits the detector's caution (~0.57 recall — it only confirms damage the detector already found). The bigger seg-gpu catches 0.98 of the damage at an 11% false-alarm cost. Pick by which error is more expensive — a missed defect or a false alarm. (Recall here is on synthetic damage and a small split, so it's a noisy proxy.)

ensemble false-positive suppression

Real different-day undamaged net. Left: seg v3 false-alarms on an instrument housing. Right: the det∧seg ensemble stays clean — same frame, no retraining.

Read these honestly. Synthetic metrics only verify the pipeline. SOLAQUA frames are real but undamaged/unlabelled (false-alarm & anomaly behaviour only). The YOLO numbers are on synthetic damage composited on real backgrounds — the damage appearance comes from one generator (same in train and test) and the cross-clip set is the same site/camera, so this shows "learns this damage model and transfers across clips/backgrounds", not "detects real damage." Real labelled damage is still required to claim real-world performance. Full detail in the report.

Generated from the committed result JSONs by python scripts/make_plots.py (figures in reports/results/plots/).

The robustness story, end to end: the first segmentation model (seg v2) regressed badly out-of-distribution (35.5% different-day false alarms) — a failure the evaluation caught. Multi-clip training (seg v3) cut that to 18%; a stronger-augmentation follow-up (seg v4) failed to improve it (22% — kept as an honest negative); and a bigger YOLOv8s-seg on a GPU (seg-gpu) reached the best recall (0.98) at an 11% false-alarm cost. The honest conclusion is a precision/recall trade-off, not a single winner: the det v1 / ensemble end (0–1% false alarms, lower recall) vs the seg-gpu end (0.98 recall, 11% false alarms) — chosen by which error costs more. (One flattering 1% number along the way turned out to be a small-sample artifact, re-checked on 200 frames and corrected.) Separately, a label-free backbone ablation for the anomaly detector (reports/results/ssl_dino/): off-the-shelf DINOv2 features are competitive and cleaner than ImageNet-ResNet18 but not a free OOD win; and domain SSL — a SimCLR ResNet18 I pretrained from scratch on the unlabelled SOLAQUA frames (ssl_pretrain.py) — underperforms both (AUROC 0.80/0.61 vs 0.98/0.82 and 1.00/0.93), an honest reminder that self-supervised learning needs scale (508 frames isn't it; the GPU path to the full video is built in).

1. Screen inspection video for suspicious regions (offline review).

python scripts/infer.py --method yolo --yolo-weights models/yolo_damage_v1.pt --source clip.mp4 --out outputs/review --every-n 5
# fewer false alarms via temporal confirmation on a contiguous sequence:
python scripts/run_temporal.py --method yolo --yolo-weights models/yolo_damage_v1.pt --source data/processed/frames --out outputs/review_temporal

1b. Live camera / ROV feed (real-time). live_inspect.py runs any method on an RTSP/HTTP IP camera, a USB webcam, or a video file, confirms defects over time (one alert per persisting defect, not per frame), and flags out-of-distribution frames for human review:

# RTSP/ROV stream, ensemble detector, live window + event log:
python scripts/live_inspect.py --source rtsp://CAMERA_IP/stream --method ensemble \
 --yolo-weights models/yolo_damage_v1.pt --seg-weights models/yolo_damage_seg_v3.pt \
 --patchcore-model models/patchcore_normal_net --display --out outputs/live
# USB webcam 0: --source 0 # any video file: --source clip.mp4

Emits damage_confirmed events (events.jsonl) with bbox, score and an ood_review flag. For a ROS-publishing ROV, swap the source for an rclpy/rospy image subscriber and call the same facade per message (see solaqua.py for the pattern).

1c. Turn a completed pass into a map (where is it, how big, what was missed).

python scripts/map_inspection.py --clip 2024-08-22_14-29-05
python scripts/map_inspection.py --clip 2024-08-22_14-29-05 --method seg_gpu --merge-radius 0.4 --no-figure

Writes a map JSON with every detection placed in metres along/across the net, its size in mm, a drift estimate, the distinct sites repeat sightings collapse to, and the coverage gaps. Needs a frame index for the clip and its _data.bag telemetry (see §"Where is it?").

2. Only have normal-net footage, want to flag anything unusual (label-free).

python scripts/train_patchcore.py --images data/processed/normal_frames --out models/patchcore_mynet
python scripts/run_anomaly.py --images data/processed/new_frames --model models/patchcore_mynet --out outputs/anomaly # heatmaps + boxes

3. Have labelled damage → train and rank approaches.

python scripts/prepare_data.py --images data/raw/images --labels data/raw/labels --out data/processed --yolo-split
python scripts/train_yolo.py --data data/processed/yolo/dataset.yaml --epochs 60
python scripts/compare_methods.py --images <test_imgs> --labels <test_lbls> --yolo-weights runs/detect/train/weights/best.pt --out outputs/comparison

4. Decide whether to trust a model (before deploying).

python scripts/adversarial_eval.py --yolo-weights models/yolo_damage_v1.pt --out reports/results/adversarial # FP on undamaged + FROC
# Combine the robust detector with the segmenter (det proposes, seg confirms):
python scripts/eval_ensemble.py --det models/yolo_damage_v1.pt --seg models/yolo_damage_seg_v3.pt --out reports/results/ensemble
# Out-of-distribution gate — flag unfamiliar frames for human review:
python scripts/eval_ood_gate.py --patchcore-model models/patchcore_normal_net --out reports/results/ood_gate
# Subtle-damage stress test (--hard = small, low-contrast damage) + sensitivity dial:
python scripts/make_real_dataset.py --frames data/processed/solaqua_diffday --out data/processed/hard_composite --hard
python scripts/eval_sensitivity.py --yolo-weights models/yolo_damage_seg_gpu.pt \
 --composited data/processed/hard_composite/images/test data/processed/hard_composite/labels/test \
 --undamaged data/processed/solaqua_diffday --out reports/results/sensitivity_hard # recall vs false-alarms by conf
python scripts/make_plots.py # turn results into figures

5. Serve / explore.

python scripts/serve.py --yolo-weights models/yolo_damage_v1.pt --patchcore-model models/patchcore_normal_net # HTTP API
streamlit run streamlit_app.py # interactive viewer

Fish-farm nets develop holes, tears, deformations and abnormal regions over time. Detecting them early matters: undetected damage risks fish escape, with operational, financial and environmental consequences. Today this is largely a manual review task — an operator or ROV pilot watches underwater camera footage and judges whether something looks wrong.

The question this prototype explores is: can computer vision flag suspicious regions in net footage to support (not replace) human inspection?

This is not a clean image-classification problem. A real system must cope with:

• Underwater optics — light attenuation, colour casts, turbidity, backscatter.
• Variable conditions — lighting, camera distance/angle, net deformation, motion blur.
• Confounders — marine growth/biofouling, ropes, shadows, fish occlusion, background structure.
• Data scarcity & imbalance — damage is rare; most frames are intact net.
• Ambiguous labels — "damage" is partly a judgement call; annotators disagree.
• Asymmetric error costs — false positives waste operator time; false negatives can miss real damage. The acceptable trade-off is a business decision, not a default.

These are exactly the reasons the prototype favours explainable baselines, careful evaluation, and visual failure analysis over a single black-box model.

The repo is organised around the workflow I would actually follow on the job:

1. Inspect & understand the data — counts, sizes, class balance, label sanity (prepare_data.py).
2. Define labels — agree damage classes and a labelling guideline with domain experts (configs/).
3. Classical baseline first — a fast, explainable OpenCV method to gauge how separable damage is and to surface data difficulty (run_classical_baseline.py).
4. Train a detection/segmentation baseline — YOLOv8 / YOLOv8-seg once labelled data exists (train_yolo.py).
5. Evaluate false positives / false negatives — precision/recall/F1/AP, image-level decisions, confidence sweeps, per-image failure overlays (evaluate.py).
6. Produce a prototype + recommendations — a report covering assumptions, results, limitations and next steps (reports/).

The classical and anomaly methods are screening / difficulty probes; the ML path is the route to real performance once labelled data exists. All three share one prediction schema and are compared with the same metrics (compare_methods.py).

net-inspection-cv/
 README.md this file
 pyproject.toml packaging + optional deps (cv / ml / dev)
 requirements.txt pinned-ish runtime deps
 configs/
 baseline.yaml classical + preprocessing + eval params
 yolo_dataset.yaml YOLO dataset config (placeholder)
 src/netinspect/
 data.py dataset discovery, YOLO label parsing, summaries
 preprocess.py CLAHE, white balance, denoise, resize
 classical_baseline.py explainable OpenCV anomaly baseline
 model_baseline.py YOLOv8 detect/segment wrapper (graceful if missing)
 anomaly.py normal-net anomaly model (patch Mahalanobis)
 patchcore.py foundation-model anomaly (pretrained-CNN PatchCore)
 dino_backbone.py self-supervised DINOv2 backbone for PatchCore (SSL-vs-supervised ablation)
 ssl_pretrain.py from-scratch SimCLR — domain-pretrain a ResNet18 on unlabelled SOLAQUA
 ensemble.py det-proposes / seg-confirms agreement ensemble (det-v1 robustness + masks)
 ood_gate.py out-of-distribution gate — defer unfamiliar frames to human review
 temporal.py IoU tracker — confirm detections that persist
 mapping.py net-frame localisation: visual odometry, self-calibrating scale, coverage
 live.py real-time camera/RTSP/ROV capture + inference (threaded, drops stale frames)
 netmodel.py place a measured strip on a declared sea cage; coverage vs the whole net
 cli.py the `netinspect` command — the front door for everything below
 dataset.py bring-your-own-data: ingest, audit, leakage-safe grouped split
 acceptance.py release gate + threshold calibration against an operating point
 compose.py composite photorealistic damage + hard negatives onto REAL frames
 inference.py unified facade over all methods (used everywhere)
 onnx_infer.py ONNX Runtime inference path
 evaluate.py detection / segmentation / image-level metrics
 visualize.py overlays, comparisons, galleries
 video.py video frame extraction
 solaqua.py SOLAQUA client + ROS-bag camera & sonar extraction
 telemetry.py 13 canonical ROV streams from the data bags, sensor-suite drift normalised
 frame_sync.py recover frame timestamps and join frames to telemetry on the bag clock
 envelope.py operating-envelope gate — when is a frame's verdict trustworthy
 image_quality.py per-frame capture quality (sharpness, contrast, brightness, saturation)
 sites.py Fiskeridirektoratet locality register
 ocean.py MET Norway ocean forecast + cod thermal optimum
 warehouse.py DuckDB view layer over every artifact
 assistant/ grounded tool-calling Q&A: evidence ledger, tools, backends, eval suite
 coco.py COCO -> YOLO adapter (real labelled data drop-in)
 synthetic.py placeholder data generator (testing only)
 utils.py IO, geometry, optional-dependency handling
 scripts/ CLI entry points (see below)
 incl. make_cage_gif.py / make_temporal_gif.py /
 make_workflow_figure.py — every README figure and
 animation regenerates from the code it documents
 web/ interactive console (index.html / style.css / app.js)
 + net3d.js (3-D cage viewer), net3d.test.mjs, net3d.render.mjs
 streamlit_app.py alternative interactive viewer
 models/ committed prototype models (.pt/.npz/.onnx) + NOTICE
 Dockerfile CPU container (built + verified; see the file header)
 operating_point.example.yaml the acceptance contract the release gate enforces
 .github/workflows/ci.yml CI: run tests on push/PR
 tests/ pytest: data loading + metrics
 reports/PROTOTYPE_REPORT.md
 data/ outputs/ runs/ data, predictions, visualisations, training runs

This matters more than anything else on this page if you are evaluating it for commercial use, so it is not buried in a footer:

AGPL-3.0 is viral over a network, and running netinspect serve is exactly the network use that triggers it. Most corporate legal teams block AGPL for that reason; Ultralytics sells a commercial licence that removes it.

There is a way out, and it is implemented. netinspect ships a second detector path built on torchvision (BSD-3-Clause) with no Ultralytics import anywhere in it — asserted by a test that parses the module's imports rather than grepping its prose:

python scripts/train_permissive.py --data data/mysite --epochs 30
netinspect gate --data data/mysite --method permissive --weights models/permissive_v1.pt

The cost is measured rather than asserted. Both detectors, same held-out split, same threshold:

That is 21 of 24 damaged frames against 24 of 24, on 29 frames. The shipped checkpoint has had 30 epochs on a GPU (final loss 0.108 — see the sidecar models/permissive_v1.json, written by the training run itself), so the gap is not simply undertraining; the denominator is still small, so read it as "the path works and costs something", not as a definitive gap. Ultralytics is the better training stack; this trades some recall for a licence you can deploy. Measure both on your own data and decide with numbers.

You do not have to take any of this on trust. netinspect sbom writes a CycloneDX 1.5 bill of materials and classifies every installed dependency's licence from its own package metadata — never a hand-maintained table, which goes stale silently:

netinspect sbom # -> reports/results/sbom.cyclonedx.json
netinspect sbom --fail-on copyleft # exit 1 if anything strong-copyleft is present

On the full development environment that is 120 components: 115 permissive, 2 strong copyleft — ultralytics and ultralytics-thop, exactly as claimed above — plus certifi (MPL-2.0) and pi_heif (LGPLv3), both file/library-level obligations that do not reach your code.

Running that audit exposed a packaging gap worth stating, because it made the claim above untrue in practice: the ml extra bundles torch and torchvision together with ultralytics, so there was no way to install the permissive detector without also installing the AGPL one. There is now a permissive extra that carries the torchvision path alone:

pip install -e ".[cv,permissive,serve]" # torchvision detector + PatchCore, no AGPL
netinspect sbom --fail-on copyleft # exit 0 — verify it in YOUR environment

Both variants were built as containers and audited from inside them:

docker build --build-arg EXTRAS=cv,permissive,serve -t netinspect-permissive .
docker run --rm netinspect-permissive netinspect sbom --fail-on copyleft # exit 0
docker run --rm netinspect netinspect sbom --fail-on copyleft # exit 1

That is the difference between a licence claim and a licence guarantee. It also found a bug in the gate itself: netinspect sbom --fail-on copyleft printed FAILED and then exited 0, because the CLI's delegation did int(exc.code or 0) on a SystemExit whose payload was a message string. A pipeline would have read that as a pass. Fixed, and pinned by a test.

One caveat neither path removes: weights fitted on SOLAQUA frames inherit CC BY-SA 4.0 regardless of framework. For a fully unencumbered artifact, train on footage you own. Full detail in models/NOTICE.md. Not legal advice; a statement of what the upstream licences say, so the question reaches your counsel before your product.

Everything above is measured on public data. The point of the toolkit is what happens when you bring labelled footage from your own sites. One command per step, and the last one can refuse.

pip install -e ".[cv,ml,serve]"
netinspect doctor # what's installed, what's missing
netinspect onboard ./my_footage --out data/mysite
netinspect train --data data/mysite/dataset.yaml --epochs 80
netinspect calibrate --data data/mysite --weights runs/detect/train/weights/best.pt
netinspect gate --data data/mysite --weights runs/detect/train/weights/best.pt \
 --operating-point operating_point.yaml

the release gate, run for real

Captured from live runs against the labelled dataset in this repo (python scripts/make_workflow_figure.py). Same model, same data, both outcomes: at default thresholds the gate refuses because 5 clean frames cannot support a false-alarm rate, and exits 1. Told explicitly that 5 is acceptable, it passes — 24 of 24 damaged frames caught, 0 of 5 clean frames alarmed. The exit code is the point: CI can act on it.

onboard takes whatever your annotation tool exports — YOLO .txt, COCO .json, Pascal VOC .xml, or images with no labels at all — detects the format, audits it, and writes a trainable dataset plus a data_health.json. It splits grouped by clip by default, because inspection footage is video: a random image-level split puts frame 100 in train and frame 101 in test, and returns an F1 of 0.99 that means nothing. It also perceptually hashes every frame to catch the same footage exported twice under different names — grouping does not catch that, and nothing else will tell you.

It refuses rather than quietly proceeding: pixel coordinates in a normalised field, zero-area boxes, or too few clips to form three splits all stop it, with the offending files named.

calibrate picks the confidence threshold on your validation split against a false-alarm budget you set. This is the largest honest accuracy gain available without collecting more data — and the repo default of 0.25 was tuned on SOLAQUA footage, so on your camera and water it is close to arbitrary.

gate is the one that matters. It measures the model against an operating point you wrote down beforehand (see operating_point.example.yaml) and exits non-zero if it does not meet it, so CI can refuse to promote it:

FAIL — must not be deployed
 [ok ] clean frames: 28 (needs >= 20)
 [ok ] damaged frames: 10 (needs >= 5)
 [ok ] false alarm rate: 0 (needs <= 0.05)
 [FAIL] recall: 0 (needs >= 0.8) — 0 of 10 damaged frames were caught

It fails closed. A test set with no clean frames cannot produce a false-alarm rate, and the gate treats "not measurable" as a failure rather than a pass — silence there is exactly how an unvalidated model reaches a boat. It also refuses to certify on a handful of frames, because a rate over eight images is noise wearing a percentage sign.

Why the primary axis is frame-level false alarms rather than mAP: a crew watching a screen abandons a system that cries wolf, so the number that decides whether this gets used at all is what fraction of clean frames raised an alert. Recall is second, and meaningless without the first — a detector that fires on everything has perfect recall.

What this does and does not give you. The engineering is production-shaped: one CLI, a data audit that refuses bad input, a release gate with an exit code, an OOD gate for unfamiliar domains, health/readiness/metrics endpoints, and a container. The shipped weights are not a validated product — they learned synthetic damage and their recall on real holes has never been measured. Run netinspect gate against your own labelled footage; if it fails, that is the tool working. Shadow-mode first, then alerting.

The service is secure by default and refuses to be published insecurely.

export NETINSPECT_API_KEY=$(openssl rand -hex 24)
export NETINSPECT_LIVE_ALLOW='rtsp://cam-*.farm.local/*'
export NETINSPECT_MEDIA_ROOT=/srv/inspection/clips
netinspect serve --host 0.0.0.0
# then open the console WITH the key — it is stored per-session and stripped
# from the address bar, so it does not linger in browser history:
# http://your-host:8000/?key=$NETINSPECT_API_KEY

Without a key it will not bind anything but loopback — local development still just works, but publishing an unauthenticated inference endpoint is not something you can do by forgetting:

Refusing to bind 0.0.0.0 without authentication.
Set NETINSPECT_API_KEY to a secret, or bind 127.0.0.1 for local use.

POST /api/live/start used to take a free-form source string and hand it to OpenCV — a read of any file the process could open and an outbound request to anywhere the host could reach. It is now default-deny: camera indices, files under NETINSPECT_MEDIA_ROOT, and URLs matching NETINSPECT_LIVE_ALLOW. Private, loopback and link-local addresses are refused even when a pattern matches, because 169.254.169.254 is the standard route from "can reach an internal endpoint" to "has your cloud credentials".

GET /api/version reports the service version, the git commit, and a SHA-256 digest of every model file — so an inspection report can be tied to the exact artefacts that produced it, which a version string alone cannot do.

GET /api/ready answers a narrower question than it looks: can this process serve what it was deployed for? It compares the methods that were configured against the ones that actually resolved, and 503s naming the difference. The earlier version asked whether the always-present classical method was available, so it could never fail — a pod whose model volume never mounted would stay in the load-balancer and 500 every request against the method it existed to serve.

Long-lived MJPEG viewers are streamed from a coroutine rather than a sync generator. That is not a style preference: Starlette runs a sync streaming generator in the threadpool, so each viewer held one worker for the whole life of the stream, and inference shares that pool. A few dozen open browser tabs would have starved every /predict in the process indefinitely.

Still needed for a hardened deployment, and deliberately not faked here: TLS and rate limiting via a reverse proxy. It is single-process, so the in-memory metrics do not aggregate across workers, and they are counters and averages rather than histograms — no percentiles.

A container is provided in Dockerfile. It has been built and run — Docker 29.7.2, 4.47 GB — and verified end to end: no key exits 1 with the InsecureBinding reason, a key brings it up and the HEALTHCHECK reaches healthy, /api/health and /api/ready answer unauthenticated, /api/version is 401 → 200 → 401 for absent/valid/wrong keys, it runs as uid 10001, and all six methods resolve. CI does not build it, so treat a change to it as unverified until someone rebuilds.

Building it was worth doing, because the first build was broken in a way reading could not have caught. The apt list was libglib2.0-0 libgomp1, chosen by guesswork. pyproject.toml asks for opencv-python-headless, but ultralytics depends on the full opencv-python, so both wheels install and the non-headless one wins import cv2 — which needs libGL and libxcb. cv2 failed to import, taking the classical detector, video decoding and YOLO with it, and the container still started and reported healthy, advertising classical as available because that string was hardcoded rather than checked. /api/ready caught it (503, naming the missing methods) — which is the entire reason the readiness probe was rewritten to be able to fail. Both are fixed: the apt line is correct, and available_methods() now actually checks that OpenCV imports.

Python 3.10+ (developed and verified on 3.14). Core data/metric code needs only NumPy/Pandas/Pillow; OpenCV and Ultralytics are optional and the code degrades gracefully without them.

python -m venv .venv
.\.venv\Scripts\Activate.ps1 # Windows; use source .venv/bin/activate on Linux/Mac
pip install -e ".[cv,ml,data,serve,dev]" # cv=OpenCV; ml=ultralytics/torch; data=rosbags; serve=FastAPI/Streamlit

GPU training (Colab). Heavy runs (multi-clip, bigger yolov8s/m-seg, longer schedules) are slow on CPU. notebooks/colab_train_gpu.ipynb runs the identical pipeline on a Colab GPU — clone → pull SOLAQUA → rebuild the composited dataset → train → held-out adversarial eval → download weights — in minutes, and sketches the GPU-only SSL-pretraining-on-SOLAQUA next step.

If you only want the classical baseline + evaluation: pip install -e ".[cv,dev]". Extras are modular: data (rosbags, for SOLAQUA .bag), ml (YOLO), serve (API + viewer) are each optional and degrade gracefully when absent. If ultralytics is unavailable, the YOLO scripts print clear guidance instead of crashing.

This runs the whole pipeline end-to-end on generated placeholder data:

# 1. Generate synthetic net images with injected "damage" (+ YOLO labels)
python scripts/make_demo_data.py --out data/sample --n 10
# 2. Inspect the dataset (counts, sizes, class balance, label errors)
python scripts/prepare_data.py --images data/sample/images --labels data/sample/labels --out data/processed
# 3. Run the classical baseline -> predictions + overlays
python scripts/run_classical_baseline.py --images data/sample/images --out outputs/classical --config configs/baseline.yaml
# 4. Evaluate against ground truth -> metrics + failure overlays
python scripts/evaluate.py --preds outputs/classical/preds.json --images data/sample/images --labels data/sample/labels --out outputs/eval
# 5. Collect report assets (gallery, failure cases, metrics.json) in one command
python scripts/make_report_assets.py

Reminder: the synthetic demo only verifies that the pipeline works. It does not prove real aquaculture performance.

# Build a YOLO train/val split (also writes a ready-to-use dataset.yaml)
python scripts/prepare_data.py --images data/sample/images --labels data/sample/labels --out data/processed --yolo-split
# Train (needs ultralytics). On real data use more epochs / a larger model.
python scripts/train_yolo.py --data data/processed/yolo/dataset.yaml --epochs 50
# Inference with trained weights, then evaluate the same way as the classical path
python scripts/run_inference.py --images data/processed/images --weights runs/detect/train/weights/best.pt --out outputs/predictions
python scripts/evaluate.py --preds outputs/predictions/preds.json --images data/processed/images --labels data/processed/labels --out outputs/eval

SOLAQUA (paper, CC BY-SA 4.0) is real ROV footage of operational Norwegian net pens. The camera data is inside ROS .bag files and the nets are undamaged with no damage labels, so we use it honestly: real preprocessing, false-positive analysis, and anomaly detection.

# List files (sizes), then download the smallest video bag and extract ~40 frames
python scripts/fetch_solaqua.py --list
python scripts/fetch_solaqua.py --smallest-video --frames-out data/processed/solaqua_frames
# (a) Robustness / false-positive analysis: nets are undamaged, so every
# classical-baseline detection is a false alarm. This quantifies that.
python scripts/run_real_analysis.py --images data/processed/solaqua_frames --out outputs/real_analysis --config configs/baseline.yaml
# (b) Anomaly detection: learn "normal net", flag deviations (heatmaps + boxes)
python scripts/train_anomaly.py --images data/processed/solaqua_frames --out outputs/anomaly/model
python scripts/run_anomaly.py --images data/processed/solaqua_frames --model outputs/anomaly/model --out outputs/anomaly

Observed on 38 extracted frames (real, undamaged net): the classical baseline raises a false alarm on ~76% of frames (≈3 detections/frame), and the anomaly model's heatmaps light up on biofouling and lighting hotspots — both honest illustrations of why a naive heuristic is insufficient and why a learned model plus a calibrated FP/FN threshold are needed. These numbers describe false-alarm behaviour on undamaged net; they are not damage-detection metrics (there is no damage to detect here).

When real labelled damage isn't available, we composite plausible damage onto real SOLAQUA net frames (real texture/biofouling/lighting; labelled holes/tears) to get a believable training/eval set — clearly not real damage.

# Composite damage onto real frames -> YOLO train/val/test (disjoint backgrounds)
python scripts/make_real_dataset.py --frames data/processed/solaqua_frames_dense --out data/processed/real_composite
# Train YOLOv8 on real backgrounds, then compare all 3 methods on the test split
python scripts/train_yolo.py --data data/processed/real_composite/dataset.yaml --epochs 60 --imgsz 480
python scripts/compare_methods.py --images data/processed/real_composite/images/test \
 --labels data/processed/real_composite/labels/test --config configs/baseline.yaml \
 --anomaly-model outputs/anomaly/model --yolo-weights runs/detect/train/weights/best.pt \
 --out outputs/comparison

Measured: YOLO F1 ≈ 0.97 vs classical 0.50 vs anomaly 0.12, holding up on a held-out clip (F1 ≈ 0.98) — see the caveats above and in the report.

# Foundation-model anomaly (label-free, F1 ~0.78): pretrained-CNN PatchCore
python scripts/train_patchcore.py --images data/processed/solaqua_frames_dense --out models/patchcore_normal_net
# Self-supervised vs supervised features (same detector, only the backbone differs):
python scripts/train_patchcore.py --images data/processed/solaqua_frames_dense --backbone dinov2_vits14 --out models/patchcore_dino_vits14
python scripts/compare_anomaly_backbones.py --models resnet18=models/patchcore_resnet18 dinov2=models/patchcore_dino_vits14 --out reports/results/ssl_dino
# Domain SSL: SimCLR-pretrain a ResNet18 on the UNLABELLED SOLAQUA frames (GPU=minutes),
# then use that backbone in PatchCore vs ImageNet-supervised vs off-the-shelf DINOv2:
python scripts/pretrain_ssl.py --frames data/processed/solaqua_frames_dense data/processed/solaqua_bag2 data/processed/solaqua_bag3 --epochs 400 --batch 256 --device cuda --out models/ssl_resnet18_solaqua.pt
python scripts/train_patchcore.py --images data/processed/solaqua_frames_dense --backbone-weights models/ssl_resnet18_solaqua.pt --out models/patchcore_ssl_solaqua
# Adversarial "is it cheating?" suite: FP on REAL undamaged net + different-day + FROC
python scripts/adversarial_eval.py --yolo-weights models/yolo_damage_v1.pt --out reports/results/adversarial_yolo
# Temporal confirmation on a CONTIGUOUS sequence: removes ~70% of transient false alarms
python scripts/fetch_solaqua.py --bag <clip>.bag --frames-out data/processed/solaqua_seq --every-n 1 --max-frames 120
python scripts/run_temporal.py --method classical --source data/processed/solaqua_seq --out outputs/temporal --config configs/baseline.yaml

Committed result tables: reports/results/ — 4-method comparison, adversarial eval, temporal.

# Real labels usually arrive as COCO -> convert to YOLO (the drop-in slot for
# an operator's eventual labelled damage). Also handles SeaClear/TrashCan etc.
python scripts/convert_coco.py --coco anns.json --images imgs --out data/processed/real --single-class
python scripts/train_yolo.py --data data/processed/real/dataset.yaml --epochs 60
# Export to ONNX (portable; TensorRT/INT8 on-device) + latency benchmark
python scripts/export_onnx.py --weights models/yolo_damage_v1.pt --imgsz 480

Honest note on public datasets. SeaClear / TrashCan / Trash-ICRA19 are marine debris (plastic, animals, plants) — useful as a real-image smoke test of the COCO adapter and for transfer/pretraining, not a net-damage proxy. Verify each dataset's licence before downloading. The COCO adapter exists so that real labelled net-damage data drops straight in when it's available.

Deployment. ONNX export works and is benchmarked (reports/results/onnx_benchmark.json); on this CPU dev box ONNX Runtime wasn't faster than PyTorch — the real speedup is TensorRT (FP16/INT8) on the target device (e.g. Jetson), which is documented rather than run here (needs the device + calibration data).

The classical baseline's max_region_density_ratio (texture gate) trades recall for fewer false alarms on real net. Lowering it from the untuned setting cut false detections on undamaged frames ~46–75% (config configs/baseline.yaml).

Trained prototype models are committed under models/ (see models/NOTICE.md for provenance/licensing), so the YOLO and anomaly paths run out of the box:

# FastAPI inference service using the committed models
python scripts/serve.py --anomaly-model models/anomaly_normal_net --yolo-weights models/yolo_damage_v1.pt
# curl -F file=@frame.jpg "http://localhost:8000/predict?method=yolo"
# Or containerised (serves YOLO + anomaly + classical). The key is NOT optional:
# CMD binds 0.0.0.0, and the service refuses a non-loopback bind without auth,
# so without it the container exits at startup instead of coming up open.
docker build -t netinspect .
$key = python -c "import secrets; print(secrets.token_hex(24))"
docker run -p 8000:8000 -e NETINSPECT_API_KEY=$key netinspect
# then browse to http://localhost:8000/?key=$key
# Interactive viewer: browse frames, switch methods, live thresholds
streamlit run streamlit_app.py
# Unified batch/video/bag inference for any method
python scripts/infer.py --method yolo --yolo-weights models/yolo_damage_v1.pt \
 --source data/processed/solaqua_frames --out outputs/infer

Measured results are committed under reports/results/ (in-clip and cross-clip method comparisons, real-frame false-positive analysis).

python scripts/fetch_solaqua.py --bag data/raw/solaqua/<clip>.bag --sonar-out data/processed/sonar

Sonar (SonoptixECHO) decodes to a ×ばつ512 intensity image — a complementary modality that sees through turbidity. Experimental; not RGB damage detection.

python scripts/extract_frames.py --video data/raw/video.mp4 --out data/processed/frames --fps 2

Implemented and working:

• Image/video ingestion, YOLO detection and segmentation label parsing, dataset summaries with invalid-label reporting.
• Underwater preprocessing (gray-world white balance, CLAHE, denoise, resize).
• Classical baseline producing boxes + scores + overlays + JSON/CSV.
• YOLOv8 detect/segment training & inference wrapper (graceful when ultralytics is absent).
• Evaluation: IoU-matched precision/recall/F1, VOC-style AP, confidence sweep, image-level "contains damage?", mask IoU, explicit FP/FN lists, failure overlays.
• Visualisation: prediction/GT overlays, colour-coded match overlays, markdown gallery.
• Real-data ingestion (SOLAQUA): public-API client, resumable download, ROS .bag camera and multibeam-sonar extraction.
• Two anomaly baselines: hand-crafted patch-Mahalanobis, and a foundation-model PatchCore (pretrained CNN) reaching F1 0.78 label-free (×ばつ the hand-crafted one).
• Realistic training data: photorealistic damage (seamless blend, frayed fibres) + hard negatives composited on real backgrounds (compose.py).
• Four-method comparison + a measured improvement story (classical FP reduction; PatchCore; YOLO F1≈0.97).
• Adversarial "is it cheating?" evaluation: false positives on real undamaged net, different-day generalization, FROC curve.
• Temporal reasoning: persistence tracking that removes ~70% of transient false alarms on real video.
• Segmentation (YOLOv8-seg) for masks; COCO ingestion adapter (real labelled data drops in); ONNX export + latency benchmark.
• Interactive web console (FastAPI + custom UI) with self-explanatory source/method descriptions and a live OOD-gate badge; Streamlit viewer; batch/video/bag runner; live_inspect.py real-time RTSP/USB/ROV streaming with temporal confirmation + per-defect alerts; per-class evaluation; CI.
• Production-shaped serving: path-traversal-safe, upload validation, structured logging + request IDs, /health /ready /metrics, no-leak error handling.
• Torch-free ONNX inference (onnx_infer.py, parity-verified) + a streaming pipeline (stream_inspect.py) that emits one confirmed-damage alert per new track.
• Ops artifacts: model card, deployment/SLO runbook, data-collection protocol, and a self-critical production-readiness scorecard.
• Tests (647, in 36 files) for data, metrics, anomaly, compositing, inference, temporal, PatchCore, COCO, per-class, the synthetic label generator, the polarity filter, service security/integration, deployment/container contract, licence/SBOM, ONNX, the DINOv2 backbone, and SimCLR pretraining.

Placeholder / synthetic (clearly marked):

• data/sample/ and the synthetic.py generator are procedural placeholders. Any metric computed on them measures the pipeline, not real-world skill.
• configs/yolo_dataset.yaml points at the demo split; classes are a starting point to be agreed with domain experts.
• The classical baseline parameters in configs/baseline.yaml are tuned for the synthetic data and will need re-tuning (or replacing with the ML model) on real footage.

• Representative footage — images/video from real inspections across sites, seasons, depths, lighting and camera setups (ideally the operator's own inspection-camera or ROV streams).
• Real damage examples — actual holes/tears/abnormal regions, plus plenty of normal net under varied conditions.
• Metadata where available — camera type, site, depth, date/time, lighting, environment.
• Labelling guideline + domain feedback — what counts as "damage", class definitions, edge cases.
• An agreed error trade-off — acceptable false-positive vs false-negative rates for the intended use.
• The deployment target — real-time on-ROV, offline review, operator decision support, alerting, or integration with existing inspection software via its API.

See reports/PROTOTYPE_REPORT.md for the full write-up: assumptions, approach, results, failure modes and next steps.

pytest -q

This prototype can download and process the SOLAQUA dataset:

Ohrem, S. J., Haugaløkken, B. O. A., et al. SOLAQUA: SINTEF Ocean Large Aquaculture Robotics Dataset. SINTEF Ocean. Data: https://data.sintef.no/product/dp-7141fcd5-0fb8-4be3-b9ce-e5f7f5bb4a58 — arXiv:2504.01790. Licensed CC BY-SA 4.0.

SOLAQUA data is not redistributed in this repository; it is downloaded on demand via scripts/fetch_solaqua.py. Any reuse must comply with CC BY-SA 4.0 (attribution + share-alike).

MIT for this prototype's code. Downloaded SOLAQUA data remains under CC BY-SA 4.0.