We are expanding AssetOpsBench to cover a broader range of industrial challenges. We invite researchers and practitioners to contribute new scenarios, particularly in the following areas:

• Asset Classes: Turbines, HVAC Systems, Pumps, Transformers, CNC Machines, Robotics, and so on.
• Task Domains: Prognostics and Health Management, Remaining Useful Life (RUL) estimation, or Root Cause Analysis (RCA), Diagnostic Analysis and Predictive Maintenance.

How to contribute:

1. Study the Hugging Face dataset.
2. Define your scenario following our Guideline.
3. Submit a Pull Request or open an Issue with the tag new-scenario.
4. Contact us via email if any question:
   • Dhaval Patel (pateldha@us.ibm.com)
   • Nianjun Zhou (jzhou@us.ibm.com)

• Video Overview: AssetOpsBench - AI Agents for Industrial Asset Operations & Maintenance by Reliability Odyssey.

1. Announcements
2. Introduction
3. Datasets
4. AI Agents
5. Multi-Agent Frameworks
6. System Diagram
7. Leaderboards
8. Docker Setup
9. Talks & Events
10. External Resources
11. Contributors

• 📰 AAAI-2026: SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search Authors
  Code

• 🎯 AAAI-2026 Lab: From Inception to Productization: Hands-on Lab for the Lifecycle of Multimodal Agentic AI in Industry 4.0
  Website Authors AAAI 2026 Slides

• 📰 AABA4ET/AAAI-2026: Agentic Code Generation for Heuristic Rules in Equipment Monitoring Authors

• 📰 IAAI/AAAI-2026: Diversity Meets Relevancy: Multi-Agent Knowledge Probing for Industry 4.0 Applications Authors

• 📰 IAAI/AAAI-2026: Deployed AI Agents for Industrial Asset Management: CodeReAct Framework for Event Analysis and Work Order Automation Authors

• 📰 AAAI-2026 Demo: AssetOpsBench-Live: Privacy-Aware Online Evaluation of Multi-Agent Performance in Industrial Operations
  Authors Demo Video

• 📰 NeurIPS-2025 Social — Evaluating Agentic Systems
  Talk: Building Reliable Agentic Benchmarks: Insights from AssetOpsBench Total Registered Users: 2000+ Conference
  Speaker
  Attend on Luma

• 🕓 Past Event: 2025年10月03日 – 2-Hour Workshop: AI Agents and Their Role in Industry 4.0 Applications
  Event Host

• 🏆 Accepted Papers: Parts of papers are accepted at NeurIPS 2025 , EMNLP 2025 Research Track , and EMNLP 2025 Industry Track .

• 🚀 2025年09月01日: CODS 2025 Competition launched – Access AI Agentic Challenge AssetOpsBench-Live.

• 📦 2025年06月01日: AssetOpsBench v1.0 released with 141 industrial Scenarios.

✨ Stay tuned for new tracks, competitions, and community events.

AssetOpsBench is a unified framework for developing, orchestrating, and evaluating domain-specific AI agents in industrial asset operations and maintenance.

It provides:

• 4 domain-specific agents
• 2 multi-agent orchestration frameworks

Designed for maintenance engineers, reliability specialists, and facility planners, it allows reproducible evaluation of multi-step workflows in simulated industrial environments.

AssetOpsBench scenarios span multiple domains:

Some tasks focus on a single domain, others are multi-step end-to-end workflows.
Explore all scenarios HF-Dataset.

• IoT Agent: get_sites, get_history, get_assets, get_sensors
• FMSR Agent: get_sensors, get_failure_modes, get_failure_sensor_mapping
• TSFM Agent: forecasting, timeseries_anomaly_detection
• WO Agent: generate_work_order

• MetaAgent : reAct-based single-agent-as-tool orchestration
• AgentHive : plan-and-execute sequential workflow

The mcp/ directory contains MCP servers and a plan-execute runner built on the Model Context Protocol. See INSTRUCTIONS.md for setup, usage, and testing.

• Evaluated with 7 Large Language Models
• Trajectories scored using LLM Judge (Llama-4-Maverick-17B)
• 6-dimensional criteria measure reasoning, execution, and data handling

Example: MetaAgent leaderboard

meta_agent_leaderboard

• Please Refer to the
• Pre-built Docker Images: assetopsbench-basic (minimal) & assetopsbench-extra (full)
• Conda environment: assetopsbench
• Full setup guide

cd /path/to/AssetOpsBench
chmod +x benchmark/entrypoint.sh
docker-compose -f benchmark/docker-compose.yml build
docker-compose -f benchmark/docker-compose.yml up

• 📄 Paper: AssetOpsBench: Benchmarking AI Agents for Industrial Asset Operations
• 🤗 HuggingFace: Scenario & Model Hub
• 📢 Blog: Insights, Tutorials, and Updates
• 🎥 Recorded Talks: Link coming soon.

Star History Chart

Thanks goes to these wonderful people ✨