feat: rebuild internal MatAnyone2 workspace #13

Original file line number	Diff line number	Diff line change
Expand Up		@@ -83,6 +83,21 @@
		pip3 install -r hugging_face/requirements.txt
		```

	### Windows `.venv` Setup For Internal Web App
	If you are running the internal web app on a Windows workstation, a local `.venv` is the most reliable setup path:

	```powershell
	py -3.10 -m venv .venv
	.\.venv\Scripts\Activate.ps1
	python -m pip install --upgrade pip
	python -m pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128
	python -m pip install -e .
	python -m pip install decord
	python -m pip install -e D:\my_app\lens_hunter2\models\sam3\sam3_repo --no-deps
	```

	The gradio demo keeps its extra dependencies in [`hugging_face/requirements.txt`](./hugging_face/requirements.txt), so install those separately only when you need the demo UI.

		## 🔥 Inference

		### Download Model
Expand Down Expand Up		@@ -138,6 +153,110 @@ By launching, an interactive interface will appear as follow.

		![overall_teaser](assets/teaser_demo.gif)

	## 🧪 Internal Web App
	The repository now also includes a desktop-first internal web app for the queue-based review workflow.

	Current flow:

	1. Upload a short source clip from the `New Session` page.
	2. Enter the `Annotation Workbench` and build one or more target layers from the template frame.
	3. Submit the queued job and inspect `source / overlay / alpha / foreground` from the `Result Review` page before downloading artifacts.

	Current outputs:

	- `foreground.mp4`
	- `alpha.mp4`
	- `rgba_png.zip`
	- `output_prores4444.mov`

	### Annotation Workbench

	The current workbench is a single-page desktop UI built around `SAM3 -> key-frame mask -> MatAnyone2 bidirectional video matting`.

	Workbench capabilities:

	- multi-target layer management
	- arbitrary template-frame selection before annotation
	- `Positive / Negative` point placement
	- `Balanced / Hair / Edge / Motion` refine presets
	- `Add / Remove / Feather` brush cleanup
	- numeric `Preset strength / Motion softness / Temporal stability` controls
	- `Source / Overlay / Mask` review modes
	- export-mask selection before queue submission

	The internal web app now uses local `SAM3` as the default interactive segmentation backend for the annotation workbench, while `MatAnyone2` remains the video matting backend after submission. When the selected template frame is not frame `0`, the worker now splits inference into forward and backward passes from that frame and stitches the result before export. You can still override the segmentation backend or checkpoint at launch time:

	```powershell
	$env:MATANYONE2_WEBAPP_SAM_BACKEND = "sam3"
	$env:MATANYONE2_WEBAPP_SAM3_CHECKPOINT_PATH = "D:\my_app\lens_hunter2\models\sam3\checkpoints\sam3.pt"
	# fallback: switch back to SAM2 if needed
	$env:MATANYONE2_WEBAPP_SAM_BACKEND = "sam2"
	$env:MATANYONE2_WEBAPP_SAM2_VARIANT = "sam2.1_hiera_large"
	```

	### Start For Testing

	Recommended on Windows:

	```powershell
	.\scripts\start_internal_webapp.ps1
	```

	Open:

	```text
	http://127.0.0.1:8010
	```

	Check or stop the service:

	```powershell
	.\scripts\check_internal_webapp.ps1
	.\scripts\stop_internal_webapp.ps1
	```

	Manual split-process launch is still available:

	```shell
	python scripts/run_internal_webapp.py
	```

	Run the worker in a separate process:

	```shell
	python scripts/run_internal_worker.py
	```

	### Quick Test Path

	1. Upload a short clip.
	2. Choose the best template frame for segmentation.
	3. Use positive and negative clicks to isolate the person.
	4. Switch the preset to `Hair`, `Edge`, `Motion`, or `Balanced` based on the edge you care about.
	5. Tune `Preset strength`, `Motion softness`, and `Temporal stability`.
	6. Use `Add / Remove / Feather` to manually correct the key-frame mask.
	7. Save one or more targets, choose which saved masks to export, and submit the job.
	6. Review `Overlay / Alpha / Foreground` on the result page before downloading artifacts.

	### Smoke Test

	Run an end-to-end smoke test that launches the web app, launches the worker, submits two back-to-back jobs, and verifies queueing plus artifact export:

	```shell
	python scripts/smoke_internal_webapp.py --copies 2
	```

	Windows one-click helpers are also available:

	```powershell
	.\scripts\start_internal_webapp.ps1
	.\scripts\check_internal_webapp.ps1
	.\scripts\stop_internal_webapp.ps1
	.\scripts\smoke_internal_webapp.ps1 --copies 2
	```

	The smoke flow submits jobs through the same HTTP workflow used by the UI, carries the active SAM backend settings into the temporary service environment, and verifies that the queue reaches a completed result with generated artifacts.

		## 📊 Evaluation
		Please refer to the [evaluation documentation](docs/EVAL.md) for details.

Expand Down

Navigation Menu

feat: rebuild internal MatAnyone2 workspace #13

Are you sure you want to change the base?

Uh oh!

feat: rebuild internal MatAnyone2 workspace #13

Filter by extension

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!