title	DamageClaims
emoji	🚀
colorFrom	indigo
colorTo	gray
sdk	docker
app_port	8000
pinned	false
short_description	DamageClaims is a real-world OpenEnv environment

DamageClaims

Automated freight insurance adjudication via reinforcement learning.

DamageClaims is a production-ready OpenEnv environment that models the end-to-end claims investigation workflow used by freight insurance adjusters. An LLM agent gathers evidence documents, reasons over multi-party liability, detects fraud, and submits a structured payout decision — all within a strict turn budget.

This is not a toy benchmark. It models operational constraints from real logistics and insurance workflows.

The Problem

Freight damage claims are expensive to process manually. A single adjuster must:

Collect and cross-reference 5–10 documents per claim (bills of lading, inspection reports, delivery receipts, photos)
Attribute liability across three parties — carrier, warehouse, and shipper — whose responsibilities overlap
Calculate accurate payouts against declared and verified cargo values
Flag inflated or fraudulent claims before issuing payment

At scale, this process is slow, inconsistent, and error-prone. DamageClaims trains an agent to handle this automatically — making grounded, auditable decisions from evidence under time pressure.

Architecture

flowchart TD
 A[LLM Agent] -->|Builds prompt from observation| B[inference.py]
 B -->|Parses JSON action| C[DamageClaimsEnv Client]
 C -->|HTTP POST /reset, /step| D[FastAPI Server]
 D --> E[DamageClaimsEnvironment]
 E -->|Loads task config| F[data/tasks.json]
 E -->|Scores decision| G[Deterministic Grader]
 G -->|approval + liability + payout + efficiency - doc_penalty| E
 E -->|DamageClaimsObservation| D
 D -->|observation, reward, done| C
 C --> B
 B -->|START, STEP, END logs| H[stdout / CI pipeline]

Episode Sequence

sequenceDiagram
 participant Agent
 participant Client as EnvClient
 participant Server as FastAPI
 participant Env as DamageClaimsEnvironment
 Agent->>Client: reset(task_id)
 Client->>Server: POST /reset
 Server->>Env: reset()
 Env-->>Client: DamageClaimsObservation (turn=0, done=False)
 loop Max 8 turns
 Agent->>Client: step(action)
 Client->>Server: POST /step
 Server->>Env: step(action)
 Env->>Env: apply action + grade + reward
 Env-->>Client: observation, reward, done
 Client-->>Agent: updated state
 end
 Note over Agent,Env: submit_decision triggers grader and ends episode

Task Suite

Task	Difficulty	Claim ID	Scenario	Ground Truth
`simple_carrier_fault`	Easy	CLM-001	3 laptops with cracked screens damaged in transit	100% carrier liability
`split_liability`	Medium	CLM-002	Pharma cold-chain breach during shipping and storage	60% carrier / 40% warehouse
`fraud_detection`	Hard	CLM-003	Luxury watch shipment with inflated declared value and partial theft	Partial approval, reduced payout

All tasks are deterministic. Each specifies available_documents, critical_documents, ground_truth targets, and question_hints.

Scoring

The grader computes a single score in [0.0, 1.0] from five components:

score = approval_score + liability_score + payout_score + efficiency_bonus - doc_penalty

Component	Weight	Condition
Approval correct	+0.40	`approved` matches ground truth
Liability per party (×ばつ3)	+0.10 each	Within 15 percentage points of ground truth
Payout accuracy	+0.20	Relative error ≤ 10% of ground-truth payout
Efficiency bonus	up to +0.10	`(MAX_TURNS - turns_used) / (MAX_TURNS - 1)`
Document penalty	−0.20 ×ばつ (missing / total)	Proportional to critical documents not collected

Mid-episode signal (every observation)

{
 "critical_docs_collected": 2,
 "critical_docs_required": 3,
 "turns_remaining": 4
}

Terminal signal (on `submit_decision`)

{
 "approval": 0.4,
 "liability": 0.3,
 "payout": 0.2,
 "efficiency": 0.071,
 "doc_penalty": -0.0
}

Reward shaping (intermediate steps)

Action	Reward
Valid document requested	+0.05
Document not in available list	−0.05
Question asked	+0.02
`submit_decision` (terminal)	`grade()` output
Max turns reached without submit	0.0

Action Space

Actions use the DamageClaimsAction model:

Field	Type	Notes
`action_type`	enum	`request_document`, `ask_question`, `submit_decision`
`document_name`	string or null	Required for `request_document`
`question`	string or null	Required for `ask_question`
`decision`	`ClaimDecision` or null	Required for `submit_decision`

ClaimDecision fields:

Field	Type	Constraint
`approved`	bool	Approve or reject claim
`payout_usd`	float	Dollar amount
`carrier_liability_pct`	float	Must sum to 100 (±1.0) with others
`warehouse_liability_pct`	float	—
`shipper_liability_pct`	float	—

Project Structure

.
├── inference.py # Agent driver: prompt loop, logging, fallback decisions
├── client.py # DamageClaimsEnv HTTP client
├── models.py # Pydantic action and observation models
├── openenv.yaml # OpenEnv spec
├── pyproject.toml
├── data/
│ └── tasks.json # Task definitions, ground truth, document library
├── assets/
│ ├── banner.png
│ └── logo.png
├── scripts/
│ └── validate-submission.sh
└── server/
 ├── app.py # FastAPI endpoints: /reset /step /state /health
 ├── damageClaims_environment.py # Core environment + grader
 ├── Dockerfile
 └── requirements.txt

Setup and Usage

Prerequisites

Python 3.10+
uv or standard pip

Two-column quickstart

Local Dev	Docker
`python -m venv .venv`	`docker build -t damageclaims-env:latest .`
`source .venv/bin/activate`	`docker run --rm -p 8000:8000 damageclaims-env:latest`
`pip install -e .`	`curl http://127.0.0.1:8000/health`
`uvicorn server.app:app --host 0.0.0.0 --port 8000`	—
`uv run python inference.py`	—

Environment variables

Copy .env.example and fill in your credentials:

cp .env.example .env

API_BASE_URL=https://api.groq.com/openai/v1
MODEL_NAME=llama-3.3-70b-versatile
HF_TOKEN=your_api_key_here

API_KEY is accepted as a fallback if HF_TOKEN is not set.

Testing

Smoke test — single task

curl -sS -X POST http://127.0.0.1:8000/reset \
 -H "Content-Type: application/json" \
 -d '{"task_id": "simple_carrier_fault"}'

Full inference run

uv run python inference.py

Expected output format:

[START] task=simple_carrier_fault env=damageClaims model=llama-3.3-70b-versatile
[STEP] step=1 action=request_document reward=0.05 done=false error=null
[STEP] step=2 action=request_document reward=0.05 done=false error=null
[STEP] step=3 action=submit_decision reward=0.53 done=true error=null
[END] success=true steps=3 score=0.525 rewards=0.05,0.05,0.53
--- simple_carrier_fault: score=0.525 ---
=== AVERAGE SCORE: 0.500 ===

OpenEnv validation

.venv/bin/openenv validate

Deployment (Hugging Face Spaces)

Create a Docker SDK Space.
Set Space secrets: API_BASE_URL, MODEL_NAME, HF_TOKEN.
Expose port 8000.
Run the submission validator after deploy:

bash scripts/validate-submission.sh https://<space_name>-<user_name>.hf.space .

The validator checks three things in order:

HF Space live — POST /reset returns HTTP 200
Docker build — docker build completes within 600 seconds
OpenEnv validate — openenv validate passes in the repo directory

Submission Checklist

Item	Status
HF Space responds to `/reset`	Implemented
`openenv.yaml` at repo root, spec-compliant	Implemented
Docker builds with health check	Implemented
`inference.py` reproduces baseline at `temperature=0.1`	Implemented
3 tasks (easy / medium / hard) with deterministic graders	Implemented
Scores clamped to `[0.0, 1.0]`	Implemented
Partial progress signal in every observation	Implemented
`reset()` / `step()` / `state()` interface satisfied	Implemented

Troubleshooting

Symptom	Fix
`openenv validate` fails	Confirm `openenv.yaml` is at repo root and `app: server.app:app` is importable
Auth error during inference	Verify `HF_TOKEN` or `API_KEY` is set in `.env`
Docker health check fails	`docker logs <container_id>` — confirm port 8000 is exposed
`KeyError` on task_id	Valid IDs: `simple_carrier_fault`, `split_liability`, `fraud_detection`
Liability validation error	Ensure the three liability percentages sum to exactly 100
Network issues in container	Check egress proxy settings; `x-deny-reason` header indicates block reason

OpenEnv Interface Compliance

Component	Location
`openenv.yaml`	Repo root
Action model (`DamageClaimsAction`)	`models.py`
Observation model (`DamageClaimsObservation`)	`models.py`
API endpoints (`/reset`, `/step`, `/state`, `/health`)	`server/app.py`
Environment class	`server/damageClaims_environment.py`

Developers

Built by Prieyan and Vishvak R.

Navigation Menu

Latest commit

History

Folders and files

Repository files navigation

DamageClaims

The Problem

Architecture

Episode Sequence

Task Suite

Scoring

Mid-episode signal (every observation)

Terminal signal (on `submit_decision`)

Reward shaping (intermediate steps)

Action Space

Project Structure

Setup and Usage

Prerequisites

Two-column quickstart

Environment variables

Testing

Smoke test — single task

Full inference run

OpenEnv validation

Deployment (Hugging Face Spaces)

Submission Checklist

Troubleshooting

OpenEnv Interface Compliance

Developers

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
assets		assets
data		data
scripts		scripts
server		server
tests		tests
.dockerignore		.dockerignore
.gitattributes		.gitattributes
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
__init__.py		__init__.py
client.py		client.py
inference.py		inference.py
models.py		models.py
openenv.yaml		openenv.yaml
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Latest commit

History

Folders and files

Repository files navigation

DamageClaims

The Problem

Architecture

Episode Sequence

Task Suite

Scoring

Mid-episode signal (every observation)

Terminal signal (on submit_decision)

Reward shaping (intermediate steps)

Action Space

Project Structure

Setup and Usage

Prerequisites

Two-column quickstart

Environment variables

Testing

Smoke test — single task

Full inference run

OpenEnv validation

Deployment (Hugging Face Spaces)

Submission Checklist

Troubleshooting

OpenEnv Interface Compliance

Developers

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Terminal signal (on `submit_decision`)