Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence

Awesome PRs welcome arXiv:2608.21156 GitHub stars

A curated collection of research papers, benchmarks, and open-source projects on Graph Engineering in the era of LLM Agents. This repository accompanies the survey Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence and will be continuously updated.

Graph Engineering studies how explicit, dynamic, and evolving graph structures can organize tasks, coordinate heterogeneous agents, maintain runtime state, and support system evolution. The collection follows the survey's progression from Model Intelligence, through Individual Intelligence, to System Intelligence.

🤗 Contributions are welcome. If you find a missing resource or a relevant new work, please open an issue or submit a pull request.

📃 Please cite our paper if you find this survey or repository helpful.

@article{feng2026graph,
 title={Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence},
 author={Feng, Yuyuan and Xiang, Zhishang and Yang, Chaobin and Ma, Qichao and Chen, Zerui and Zhang, Yujing and Huang, Ke and Wu, Chuanjie and Liu, Zhaoxu and Wang, Yili and others},
 journal={arXiv preprint arXiv:2608.21156},
 year={2026}
}

🎉 News

[2026-08] We release our survey Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence and the accompanying resource collection.

Overview

Graph Engineering provides a structured path from standalone model capability to coordinated system-level intelligence:

Model Intelligence builds and adapts foundation-model capabilities through parameterized training, prompt engineering, and context engineering.
Individual Intelligence equips a single agent with tools, memory, skills, runtime orchestration, and persistent interaction loops.
System Intelligence organizes tasks, agents, runtime state, and system evolution through explicit graph structures, with ontology engineering providing a shared semantic layer.

The evolution from foundation models to Graph Engineering and System Intelligence

The evolution of engineering paradigms from Foundation Models to Graph and Ontology Engineering.

Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence

📚 Related Survey Papers

(arXiv 2024) Graph Retrieval-Augmented Generation: A Survey [Paper]
(arXiv 2026) Agent Harness Engineering: A Survey [Paper]
(TechRxiv 2026) A Systematic Survey of Self-Evolving Agents: From Model-Centric to Environment-Driven Co-Evolution [Paper] [Google Scholar]
(arXiv 2025) Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities [Paper]
(arXiv 2025) Graph-Augmented Large Language Model Agents: Current Progress and Future Prospects [Paper]
(arXiv 2026) Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval [Paper]
(arXiv 2024) Understanding the Planning of LLM Agents: A Survey [Paper]
(Paper 2025) A Survey on Agent Workflow—Status and Future [Paper]
(arXiv 2026) From Agent Loops to Structured Graphs: A Scheduler-Theoretic Framework for LLM Agent Execution [Paper]
(Vicinagearth 2024) A Survey on LLM-Based Multi-Agent Systems: Workflow, Infrastructure, and Challenges [Paper]
(Frontiers of Computer Science 2026) Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systems [Paper]
(arXiv 2026) Graph-Based Agent Memory: Taxonomy, Techniques, and Applications [Paper]
(TMLR 2026) A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence [Paper]
(OpenReview Archive 2026) Self-Improving Agents in the Era of Experience: A Survey of Self- to Meta-Evolution [Paper]
(arXiv 2025) Multi-Agent Collaboration Mechanisms: A Survey of LLMs [Paper]
(arXiv 2026) Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems [Paper]

📜 Research Papers

A comprehensive taxonomy of Graph Engineering in the era of LLM agents

A comprehensive taxonomy spanning Model, Individual, and System Intelligence.

Model Intelligence

From Model Intelligence to Individual Intelligence

From Model Intelligence to Individual Intelligence through Prompt, Context, Harness, and Loop Engineering.

Pre-Training

(NeurIPS 2020) GPT-3 — Language Models are Few-Shot Learners [Paper]
(arXiv 2021) Gopher — Scaling Language Models: Methods, Analysis & Insights from Training Gopher [Paper]
(JMLR 2023) PaLM — PaLM: Scaling Language Modeling with Pathways [Paper]
(arXiv 2023) LLaMA — LLaMA: Open and Efficient Foundation Language Models [Paper]
(arXiv 2020) Scaling Laws — Scaling Laws for Neural Language Models [Paper]
(NeurIPS 2022) Chinchilla — Training Compute-Optimal Large Language Models [Paper]
(JMLR 2022) Switch Transformer — Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity [Paper]
(arXiv 2024) Mixtral — Mixtral of Experts [Paper]
(Paper 2024) DeepSeekMoE — DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models [Paper]
(arXiv 2024) Llama 3 — The Llama 3 Herd of Models [Paper]
(arXiv 2024) DeepSeek-V3 — DeepSeek-V3 Technical Report [Paper]
(Paper 2022) Deduplication — Deduplicating Training Data Makes Language Models Better [Paper]
(NeurIPS 2024) FineWeb — The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale [Paper]
(NeurIPS 2024) DataComp-LM — DataComp-LM: In Search of the Next Generation of Training Sets for Language Models [Paper]
(arXiv 2024) Qwen2.5 — Qwen2.5 Technical Report [Paper]
(arXiv 2025) Qwen3 — Qwen3 Technical Report [Paper]
(arXiv 2025) Kimi K2 — Kimi K2: Open Agentic Intelligence [Paper]

Post-Training

(ICLR 2022) FLAN — Finetuned Language Models Are Zero-Shot Learners [Paper]
(ICLR 2022) T0 — Multitask Prompted Training Enables Zero-Shot Task Generalization [Paper]
(NeurIPS 2022) InstructGPT — Training Language Models to Follow Instructions with Human Feedback [Paper]
(ICML 2023) Flan Collection — The Flan Collection: Designing Data and Methods for Effective Instruction Tuning [Paper]
(arXiv 2022) Constitutional AI — Constitutional AI: Harmlessness from AI Feedback [Paper]
(arXiv 2023) RLAIF — RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback [Paper]
(NeurIPS 2023) DPO — Direct Preference Optimization: Your Language Model is Secretly a Reward Model [Paper]
(arXiv 2024) Tulu 3 — Tulu 3: Pushing Frontiers in Open Language Model Post-Training [Paper]
(arXiv 2024) DeepSeekMath — DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models [Paper]
(Nature 2025) DeepSeek-R1 — DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning [Paper]
(arXiv 2025) DAPO — DAPO: An Open-Source LLM Reinforcement Learning System at Scale [Paper]
(ICLR 2025) WebRL — WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning [Paper]
(arXiv 2025) Search-R1 — Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning [Paper]
(arXiv 2025) ReTool — ReTool: Reinforcement Learning for Strategic Tool Use in LLMs [Paper]
(arXiv 2025) ToolRL — ToolRL: Reward is All Tool Learning Needs [Paper]
(arXiv 2025) RAGEN — RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning [Paper]
(arXiv 2025) Agent-R1 — Agent-R1: Training Powerful LLM Agents with End-to-End Reinforcement Learning [Paper]
(arXiv 2025) Agent Lightning — Agent Lightning: Train ANY AI Agents with Reinforcement Learning [Paper]
(arXiv 2026) DynaWeb — DynaWeb: Model-Based Reinforcement Learning of Web Agents [Paper]

Prompt Engineering

(arXiv 2021) Prompt Programming — Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm [Paper]
(Paper 2022) Demonstrations — Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? [Paper]
(Paper 2022) In-Context Examples — What Makes Good In-Context Examples for GPT-3? [Paper]
(NeurIPS 2022) Chain-of-Thought — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models [Paper]
(ICLR 2023) Self-Consistency — Self-Consistency Improves Chain of Thought Reasoning in Language Models [Paper]
(ICLR 2023) Least-to-Most — Least-to-Most Prompting Enables Complex Reasoning in Large Language Models [Paper]
(NeurIPS 2023) Tree of Thoughts — Tree of Thoughts: Deliberate Problem Solving with Large Language Models [Paper]
(NeurIPS 2023) Self-Refine — Self-Refine: Iterative Refinement with Self-Feedback [Paper]
(AAAI 2024) Graph of Thoughts — Graph of Thoughts: Solving Elaborate Problems with Large Language Models [Paper]
(Paper 2020) AutoPrompt — AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts [Paper]
(ICLR 2023) APE — Large Language Models Are Human-Level Prompt Engineers [Paper]
(ICLR 2024) OPRO — Large Language Models as Optimizers [Paper]
(ICLR 2024) Promptbreeder — Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution [Paper]
(arXiv 2024) TextGrad — TextGrad: Automatic ``Differentiation'' via Text [Paper]

Context Engineering

(Paper 2020) DPR — Dense Passage Retrieval for Open-Domain Question Answering [Paper]
(NeurIPS 2020) RAG — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks [Paper]
(Paper 2021) FiD — Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering [Paper]
(Paper 2023) HyDE — Precise Zero-Shot Dense Retrieval without Relevance Labels [Paper]
(Paper 2023) IRCoT — Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions [Paper]
(ICLR 2024) Self-RAG — Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection [Paper]
(NeurIPS 2025) CoRAG — Chain-of-Retrieval Augmented Generation [Paper]
(NeurIPS 2024) RankRAG — RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs [Paper]
(Paper 2023) LLMLingua — LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models [Paper]
(ICLR 2024) RECOMP — RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation [Paper]
(arXiv 2024) GraphRAG — From Local to Global: A Graph RAG Approach to Query-Focused Summarization [Paper]
(ICLR 2025) Provence — Provence: Efficient and Robust Context Pruning for Retrieval-Augmented Generation [Paper]
(Paper 2024) Lost in the Middle — Lost in the Middle: How Language Models Use Long Contexts [Paper]
(arXiv 2023) MemGPT — MemGPT: Towards LLMs as Operating Systems [Paper]
(arXiv 2024) HiAgent — HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model [Paper]
(ICML 2026) ACON — ACON: Optimizing Context Compression for Long-horizon LLM Agents [Paper]
(ICLR 2026) ACE — Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models [Paper]
(arXiv 2026) ContextCurator — Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning [Paper]
(arXiv 2026) AdaCoM — Learning Agent-Compatible Context Management for Long-Horizon Tasks [Paper]

Individual Intelligence

Tool Integration

(arXiv 2022) MRKL — MRKL Systems: A Modular, Neuro-Symbolic Architecture that Combines Large Language Models, External Knowledge Sources and Discrete Reasoning [Paper]
(arXiv 2022) TALM — TALM: Tool Augmented Language Models [Paper]
(ICLR 2023) ReAct — ReAct: Synergizing Reasoning and Acting in Language Models [Paper]
(NeurIPS 2023) Toolformer — Toolformer: Language Models Can Teach Themselves to Use Tools [Paper]
(Paper 2023) API-Bank — API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs [Paper]
(ICLR 2024) ToolLLM — ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs [Paper]
(NeurIPS 2024) Gorilla — Gorilla: Large Language Model Connected with Massive APIs [Paper]
(Anthropic 2024) MCP — Introducing the Model Context Protocol [Paper]
(arXiv 2024) CodeAct — Executable Code Actions Elicit Better LLM Agents [Paper]
(arXiv 2024) SWE-agent — SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering [Paper]
(arXiv 2024) OpenHands — OpenHands: An Open Platform for AI Software Developers as Generalist Agents [Paper]
(arXiv 2025) ToolMaker — LLM Agents Making Agent Tools [Paper]
(OpenAI 2025) Codex — Introducing Codex [Paper]
(Anthropic 2025) Claude Code — Claude 3.7 Sonnet and Claude Code [Paper]
(Google 2025) Gemini CLI — Gemini CLI: Your Open-Source AI Agent [Paper]
(GitHub Blog 2025) Copilot Coding Agent — GitHub Copilot: Meet the New Coding Agent [Paper]
(OpenAI Engineering 2026) Symphony — An Open-Source Spec for Codex Orchestration: Symphony [Paper]

Memory Management

(Paper 2023) Generative Agents — Generative Agents: Interactive Simulacra of Human Behavior [Paper]
(AAAI 2024) MemoryBank — MemoryBank: Enhancing Large Language Models with Long-Term Memory [Paper]
(arXiv 2023) MemGPT — MemGPT: Towards LLMs as Operating Systems [Paper]
(arXiv 2025) A-MEM — A-MEM: Agentic Memory for LLM Agents [Paper]
(arXiv 2025) Mem0 — Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory [Paper]
(arXiv 2025) Zep — Zep: A Temporal Knowledge Graph Architecture for Agent Memory [Paper]
(arXiv 2025) MemoryOS — Memory OS of AI Agent [Paper]
(arXiv 2025) Memoria — Memoria: A Scalable Agentic Memory Framework for Personalized Conversational AI [Paper]
(Paper 2026) AgeMem — Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [Paper]
(arXiv 2026) Memori — Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agents [Paper]
(arXiv 2026) LycheeMemory V2 — LycheeMemory V2: Efficient Long-Term Memory for LLM Agents via Semantic Segment-Level Consolidation [Paper]
(arXiv 2024) Agent Workflow Memory — Agent Workflow Memory [Paper]

Skill Composition

(arXiv 2023) Voyager — Voyager: An Open-Ended Embodied Agent with Large Language Models [Paper]
(ICLR 2024) CRAFT — CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets [Paper]
(Anthropic Engineering 2025) Agent Skills — Equipping Agents for the Real World with Agent Skills [Paper]
(arXiv 2025) SAGE — Reinforcement Learning for Self-Improving Agent with Skill Library [Paper]
(arXiv 2026) HASP — Harnessing LLM Agents with Skill Programs [Paper]
(arXiv 2026) SSL Skills — From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills [Paper]
(arXiv 2026) SkillComposer — SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [Paper]
(arXiv 2026) Generative Skill Composition — Generative Skill Composition for LLM Agents [Paper]
(arXiv 2026) Skill-Use — Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses? [Paper]
(arXiv 2026) HDSO — Hypothesis-Driven Skill Optimization for LLM Agents [Paper]
(arXiv 2026) Demystifying Agent Skills — Demystifying Agent Skills: Why They Work—Until They Don't [Paper]

Runtime Orchestration

(Anthropic Engineering 2025) Long-Running Harness — Effective Harnesses for Long-Running Agents [Paper]
(arXiv 2026) Externalization — Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering [Paper]
(arXiv 2026) Harness Engineering — AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents [Paper]
(arXiv 2026) Code as Agent Harness — Code as Agent Harness [Paper]
(arXiv 2026) Harness Configuration — Configuring Agentic AI Coding Tools: An Exploratory Study [Paper]
(arXiv 2026) Harness-Bench — Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows [Paper]
(arXiv 2026) Prompts to Contracts — From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents [Paper]
(arXiv 2024) ToolSandbox — ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities [Paper]
(arXiv 2023) ToolEmu — Identifying the Risks of LM Agents with an LM-Emulated Sandbox [Paper]
(arXiv 2026) RHO — Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference [Paper]
(arXiv 2025) CaMeL — Defeating Prompt Injections by Design [Paper]
(arXiv 2025) MCP Security Bench — MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents [Paper]
(OpenAI Engineering 2026) OpenAI Harness Engineering — Harness Engineering: Leveraging Codex in an Agent-First World [Paper]
(Anthropic Engineering 2026) Anthropic Harness Design — Harness Design for Long-Running Application Development [Paper]
(arXiv 2026) Meta-Harness — Meta-Harness: End-to-End Optimization of Model Harnesses [Paper]
(arXiv 2026) Agentic Harness — Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses [Paper]
(arXiv 2026) Self-Harness — Self-Harness: Harnesses That Improve Themselves [Paper]
(arXiv 2026) HarnessFix — From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws [Paper]
(arXiv 2026) HARBOR — HARBOR: Automated Harness Optimization [Paper]
(arXiv 2024) AgentDojo — AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents [Paper]
(arXiv 2026) Harness Updating — Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents [Paper]
(arXiv 2026) Adaptive Auto-Harness — Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams [Paper]
(arXiv 2026) LongHorizon-Harness — LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks [Paper]
(arXiv 2026) OneDayAgent — OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents [Paper]
(arXiv 2026) Evo-Harness — Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [Paper]
(arXiv 2026) Harness Handbook — Harness Handbook: Making Evolving Agent Harnesses Readable, Navigable, and Editable [Paper]
(arXiv 2026) HarnessOpt-Bench — HarnessOpt-Bench: Evaluating LLMs at Harness Optimization [Paper]
(arXiv 2026) The Scaffold Effect — The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation [Paper]
(arXiv 2026) Harness-IF — Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents [Paper]
(arXiv 2026) Evo-Bench — Evo-Bench: Can Language Models Improve Agent Harness? [Paper]

Loop Architecture

(arXiv 2024) StateFlow — StateFlow: Enhancing LLM Task-Solving through State-Driven Workflows [Paper]
(arXiv 2024) Magentic-One — Magentic-One: A generalist multi-agent system for solving complex tasks [Paper]
(arXiv 2024) AIOS — AIOS: LLM Agent Operating System [Paper]
(arXiv 2024) AgentBoard — AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents [Paper]
(arXiv 2023) AdaPlanner — AdaPlanner: Adaptive Planning from Feedback with Language Models [Paper]
(arXiv 2026) When Agents Do Not Stop — When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents [Paper]
(arXiv 2026) Stop Hand-Holding Your Coding Agent — Stop Hand-Holding Your Coding Agent: Engineering the Loops That Replace Step-by-Step Prompting [Paper]
(arXiv 2026) ResearchLoop — ResearchLoop: An Evidence-Gated Control Plane for AI-Assisted Research [Paper]
(arXiv 2026) Proof-or-Stop — Proof-or-Stop: Don't Trust the Agent, Trust the Evidence – Loop Engineering for Verifiable Evidence-Gated Lifecycle Control [Paper]

Interaction Paradigm

(arXiv 2026) Beyond Message Passing — Beyond Message Passing: A Semantic View of Agent Communication Protocols [Paper]
(Paper 2025) Internet of Agents — Internet of Agents: Fundamentals, Applications, and Challenges [Paper]
(NeurIPS 2025) LACP — LLM Agent Communication Protocol (LACP) Requires Urgent Standardization: A Telecom-Inspired Protocol Is Necessary [Paper]
(Paper 2026) Beyond the Protocol — Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem [Paper]
(arXiv 2026) AgentRx — AgentRx: Diagnosing AI Agent Failures from Execution Trajectories [Paper]
(arXiv 2026) Supervising Ralph Wiggum — Supervising Ralph Wiggum: Exploring a Metacognitive Co-Regulation Agentic AI Loop for Engineering Design [Paper]
(ZTE Communications 2025) From Function Calls to MCPs — From Function Calls to MCPs for Securing AI Agent Systems: Architecture, Challenges and Countermeasures [Paper]
(arXiv 2026) Sovereign Agentic Loops — Sovereign Agentic Loops: Decoupling AI Reasoning from Execution in Real-World Systems [Paper]
(arXiv 2026) The Log is the Agent — The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems [Paper]

Environment Feedback

(arXiv 2026) EurekAgent — EurekAgent: Agent Environment Engineering is All You Need for Autonomous Scientific Discovery [Paper]
(arXiv 2023) LEVER — LEVER: Learning to Verify Language-to-Code Generation with Execution [Paper]
(arXiv 2023) CRITIC — CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [Paper]
(arXiv 2024) Executable Code Actions — Executable Code Actions Elicit Better LLM Agents [Paper]
(arXiv 2024) ToolSandbox — ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities [Paper]
(NeurIPS 2024) OSWorld — OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments [Paper]
(arXiv 2026) DeltaBox — DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback [Paper]

System Intelligence

Graph Engineering overview

Graph Engineering organizes tasks, coordinates agents, and manages runtime state.

Task Organization

(NeurIPS 2023) HuggingGPT — HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends in Hugging Face [Paper]
(arXiv 2023) ReWOO — ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models [Paper]
(ICML 2024) LLMCompiler — An LLM Compiler for Parallel Function Calling [Paper]
(arXiv 2025) Plan-over-Graph — Plan-over-Graph: Towards Parallelable LLM Agent Schedule [Paper]
(Neural Networks 2025) TDAG — TDAG: A Multi-Agent Framework Based on Dynamic Task Decomposition and Agent Generation [Paper]
(ICLR 2025) Flow — Flow: Modularized Agentic Workflow Automation [Paper]
(arXiv 2025) VFlow — VFlow: Discovering Optimal Agentic Workflows for Verilog Generation [Paper]
(ICML 2024) GPTSwarm — GPTSwarm: Language Agents as Optimizable Graphs [Paper]
(ICLR 2025) ADAS — Automated Design of Agentic Systems [Paper]
(arXiv 2024) AutoFlow — AutoFlow: Automated Workflow Generation for Large Language Model Agents [Paper]
(ICLR 2025) AFlow — AFlow: Automating Agentic Workflow Generation [Paper]
(arXiv 2025) A2Flow — A2Flow: Automating Agentic Workflow Generation via Self-Adaptive Abstraction Operators [Paper]
(arXiv 2025) MermaidFlow — MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming [Paper]
(Paper 2025) DynTaskMAS — DynTaskMAS: A dynamic task graph-driven framework for asynchronous and parallel llm-based multi-agent systems [Paper]
(NeurIPS 2025) DyFlow — DyFlow: Dynamic Workflow Framework for Agentic Reasoning [Paper]
(arXiv 2025) EvoFlow — EvoFlow: Evolving Diverse Agentic Workflows On The Fly [Paper]
(arXiv 2025) QualityFlow — QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks [Paper]
(arXiv 2026) FlowSteer — FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems [Paper]
(arXiv 2026) AgenticLab — AgenticLab: A Real-World Robot Agent Platform that Can See, Think, and Act [Paper]
(arXiv 2025) ScalingAgent — Towards a science of scaling agent systems [Paper]

Agent Coordination

Agent coordination through capability, team, and communication graphs

Agent Coordination through capability mapping, team organization, and communication structures.

(NeurIPS 2023) CAMEL — CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society [Paper]
(COLM 2024) DyLAN — A dynamic LLM-powered agent network for task-oriented agent collaboration [Paper]
(ICLR 2025) Agent-Oriented Planning — Agent-oriented planning in multi-agent systems [Paper]
(ACL 2025) MasRouter — MasRouter: Learning to route LLMs for multi-agent systems [Paper]
(IJCAI 2024) AutoAgents — AutoAgents: A framework for automatic agent generation [Paper]
(NAACL 2025) EvoAgent — EvoAgent: Towards automatic multi-agent generation via evolutionary algorithms [Paper]
(ICLR 2026) Collaborative Gym — Collaborative gym: A framework for enabling and evaluating human-agent collaboration [Paper]
(ICML 2026) AOrchestra — AOrchestra: Automating sub-agent creation for agentic orchestration [Paper]
(ECAI 2025) Captain Agent — Adaptive graph pruning for multi-agent communication [Paper]
(ICML 2026) MaAS — Multi-agent architecture search via agentic supernet [Paper]
(arXiv 2026) SkillGraph — SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topology [Paper]
(ICLR 2024) MetaGPT — MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework [Paper]
(ACL 2024) ChatDev — ChatDev: Communicative Agents for Software Development [Paper]
(arXiv 2024) Magentic-One — Magentic-One: A generalist multi-agent system for solving complex tasks [Paper]
(ICLR 2024) AgentVerse — AgentVerse: Facilitating multi-agent collaboration and exploring emergent behaviors [Paper]
(NeurIPS 2025) Puppeteer — Multi-agent collaboration via evolving orchestration [Paper]
(NeurIPS 2025) AgentNet — AgentNet: Decentralized evolutionary coordination for llm-based multi-agent systems [Paper]
(ICLR 2025) MacNet — Scaling large language model-based multi-agent collaboration [Paper]
(EMNLP 2025) SwarmAgentic — Swarmagentic: Towards fully automated agentic system generation via swarm intelligence [Paper]
(ICLR 2025) Mixture-of-Agents — Mixture-of-agents enhances large language model capabilities [Paper]
(ICML 2025) G-Designer — G-Designer: Architecting multi-agent communication topologies via graph neural networks [Paper]
(EMNLP 2025) AMAS — AMAS: Adaptively determining communication topology for llm-based multi-agent system [Paper]
(ICLR 2025) AgentPrune — Cut the crap: An economical communication pipeline for llm-based multi-agent systems [Paper]
(ACL 2025) AgentDropout — AgentDropout: Dynamic agent elimination for token-efficient and high-performance llm-based multi-agent collaboration [Paper]
(arXiv 2026) DyTopo — DyTopo: Dynamic topology routing for multi-agent reasoning via semantic matching [Paper]
(AAAI 2026) Adaptive Theory of Mind — Adaptive theory of mind for LLM-based multi-agent coordination [Paper]
(AAAI 2026) Assemble Your Crew — Assemble your crew: Automatic multi-agent communication topology design via autoregressive graph generation [Paper]
(AAAI 2026) Learning to Generate and Extract — Learning to Generate and Extract: A Multi-Agent Collaboration Framework For Zero-shot Document-level Event Arguments Extraction [Paper]
(arXiv 2026) Agent-World — Agent-world: Scaling real-world environment synthesis for evolving general agent intelligence [Paper]
(ICLR 2026) Graph-of-Agents — Graph-of-agents: A graph-based framework for multi-agent LLM collaboration [Paper]
(ICLR 2026) Emergent Coordination — Emergent coordination in multi-agent language models [Paper]
(ICLR 2026) AgentPO — AgentPO: Enhancing Multi-Agent Collaboration via Reinforcement Learning [Paper]
(ICLR 2026) Multi-Agent Design — Multi-agent design: Optimizing agents with better prompts and topologies [Paper]
(IEEE Network 2026) Agent Discovery — Agent Discovery in Internet of Agents: Challenges and Solutions [Paper]
(SIGIR 2026) LLM Agents Factory — LLM Agents Factory: Retrieval of Domain-Specific LLM Agents [Paper]
(arXiv 2026) TacoMAS — TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems [Paper]
(arXiv 2026) EvolveRouter — EvolveRouter: Co-evolving routing and prompt for multi-agent question answering [Paper]
(arXiv 2026) MoRSE — MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts [Paper]
(ICLR 2025) Internet of Agents — Internet of agents: Weaving a web of heterogeneous agents for collaborative intelligence [Paper]
(arXiv 2026) Organizational Science of Multi-Agent LLM Systems — Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm [Paper]
(arXiv 2026) Multi-Agent Teams Hold Experts Back — Multi-agent teams hold experts back [Paper]
(arXiv 2026) Dynamic Role Assignment — Dynamic role assignment for multi-agent debate [Paper]
(NAACL 2025) WorkTeam — WorkTeam: Constructing Workflows from Natural Language with Multi-Agents [Paper]
(arXiv 2026) Meta-Team — Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems [Paper]
(arXiv 2026) WebSwarm — WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search [Paper]
(arXiv 2025) OWL — Owl: Optimized workforce learning for general multi-agent assistance in real-world task automation [Paper]
(ICLR 2025) AgentSquare — Agentsquare: Automatic llm agent search in modular design space [Paper]
(arXiv 2026) MAS-on-the-Fly — MAS-on-the-Fly: Dynamic Adaptation of LLM-based Multi-Agent Systems at Test Time [Paper]
(arXiv 2026) Self-Organizing Agents — Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures [Paper]
(arXiv 2026) Swarm Skills — Swarm Skills: A Portable, Self-Evolving Multi-Agent System Specification for Coordination Engineering [Paper]
(arXiv 2025) ATLAS — Atlas: Adaptive trading with llm agents through dynamic prompt optimization and multi-agent coordination [Paper]
(EMNLP 2025) MAgICoRe — Magicore: Multi-agent, iterative, coarse-to-fine refinement for reasoning [Paper]
(EMNLP 2025) Information Propagation Effects — Understanding the information propagation effects of communication topologies in llm-based multi-agent systems [Paper]
(arXiv 2026) CARD — CARD: Towards Conditional Design of Multi-agent Topological Structures [Paper]
(arXiv 2026) QueenBee Planner — QueenBee Planner: Skill-Evolving Communication Topologies for Token-Efficient LLM Multi-Agent Systems [Paper]
(arXiv 2026) Agentic Aggregation — Agentic aggregation for parallel scaling of long-horizon agentic tasks [Paper]

State Management

Runtime state recording, fault localization, and failure recovery

Runtime State Management through state recording, fault localization, and failure recovery.

(arXiv 2024) StateFlow — StateFlow: Enhancing LLM Task-Solving through State-Driven Workflows [Paper]
(arXiv 2024) AutoGRAMS — AutoGRAMS: Autonomous Graphical Agent Modeling Software [Paper]
(arXiv 2024) Magentic-One — Magentic-One: A generalist multi-agent system for solving complex tasks [Paper]
(ICML 2026) GoS — Graph of States: Solving Abductive Tasks with Large Language Models [Paper] [Project] [Code]
(Paper 2026) LangGraph — LangGraph: Low-Level Orchestration for Stateful Agents [Paper]
(Paper 2026) Burr — Apache Burr: Stateful Application and Agent Framework [Paper]
(arXiv 2025) Aegis — Aegis: Taxonomy and Optimizations for Overcoming Agent-Environment Failures in LLM Agents [Paper]
(Paper 2026) LlamaIndex Workflows — LlamaIndex Workflows: Event-Driven Agent Workflows [Paper]
(Paper 2026) Pydantic AI — Pydantic AI: Typed Agent Framework and Graph Runtime [Paper]
(arXiv 2023) AutoGen — AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation [Paper]
(arXiv 2026) Sovereign Agentic Loops — Sovereign Agentic Loops: Decoupling AI Reasoning from Execution in Real-World Systems [Paper]
(arXiv 2023) LATS — Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models [Paper]
(arXiv 2026) PatchBoard — PatchBoard: Schema-Grounded State Mutation for Reliable and Auditable LLM Multi-Agent Collaboration [Paper]
(arXiv 2026) MemTX — MemTX: Transactional Belief Commit for Stateful Agent Memory [Paper]
(arXiv 2026) Cordon — Cordon: Semantic Transactions for Tool-Using LLM Agents [Paper]
(arXiv 2026) Atomix — Atomix: Timely, transactional tool use for reliable agentic workflows [Paper]
(arXiv 2025) SagaLLM — SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning [Paper]
(arXiv 2025) ALAS — Alas: Transactional and dynamic multi-agent llm planning [Paper]
(CAIS 2026) RAC — Robust Agent Compensation (RAC): Teaching AI Agents to Compensate [Paper]
(arXiv 2026) ProPlay — ProPlay: Procedural World Models for Self-Evolving LLM Agents [Paper]
(arXiv 2026) DART — DART: Semantic Recoverability for Structured Tool Agents [Paper]
(arXiv 2025) AgentGit — AgentGit: A version control framework for reliable and scalable LLM-powered multi-agent systems [Paper]
(arXiv 2026) Shepherd — Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Traces [Paper]
(arXiv 2026) The Log is the Agent — The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems [Paper]
(arXiv 2026) Concurrency Anomaly Prevention — Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent Large Language Model Systems [Paper]
(arXiv 2026) CausalFlow — CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures [Paper]
(arXiv 2025) ReflexGrad — ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing [Paper]
(arXiv 2026) TDAD — TDAD: Test-Driven Agentic Development - Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis [Paper]
(arXiv 2025) Who & When — Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [Paper]
(NeurIPS 2025) MAST — Why Do Multi-Agent LLM Systems Fail? [Paper]
(arXiv 2026) Living-Harness — Living-Harness Is an Interactive-Agent Evolver [Paper]
(arXiv 2026) APEX — APEX: Autonomous Policy Exploration for Self-Evolving LLM Agents [Paper]

System Evolution

(arXiv 2026) QueenBee Planner — QueenBee Planner: Skill-Evolving Communication Topologies for Token-Efficient LLM Multi-Agent Systems [Paper]
(ACL 2026) ReCreate — Recreate: Reasoning and creating domain agents driven by experience [Paper]
(arXiv 2026) SkillGraph — SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topology [Paper]
(arXiv 2026) Swarm Skills — Swarm Skills: A Portable, Self-Evolving Multi-Agent System Specification for Coordination Engineering [Paper]
(arXiv 2026) MemTX — MemTX: Transactional Belief Commit for Stateful Agent Memory [Paper]
(arXiv 2026) The Log is the Agent — The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems [Paper]
(Neural Networks 2025) TDAG — TDAG: A Multi-Agent Framework Based on Dynamic Task Decomposition and Agent Generation [Paper]
(ICLR 2025) Flow — Flow: Modularized Agentic Workflow Automation [Paper]
(ICAPS 2025) DynTaskMAS — DynTaskMAS: A dynamic task graph-driven framework for asynchronous and parallel llm-based multi-agent systems [Paper]
(NeurIPS 2025) DyFlow — DyFlow: Dynamic Workflow Framework for Agentic Reasoning [Paper]
(arXiv 2025) EvoFlow — EvoFlow: Evolving Diverse Agentic Workflows On The Fly [Paper]
(arXiv 2025) QualityFlow — QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks [Paper]
(arXiv 2026) FlowSteer — FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems [Paper]
(EMNLP 2025) SwarmAgentic — SwarmAgentic: Towards fully automated agentic system generation via swarm intelligence [Paper]
(NeurIPS 2025) AgentNet — AgentNet: Decentralized evolutionary coordination for llm-based multi-agent systems [Paper]
(arXiv 2026) Self-Organizing Agents — Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures [Paper]
(arXiv 2026) Meta-Team — Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems [Paper]
(arXiv 2026) DyTopo — DyTopo: Dynamic topology routing for multi-agent reasoning via semantic matching [Paper]
(arXiv 2026) CARD — CARD: Towards Conditional Design of Multi-agent Topological Structures [Paper]

Ontology Engineering

(arXiv 2026) OntoExtend — OntoExtend: A Framework for Requirement-driven and Scalable Ontology Extension with LLMs [Paper]
(Knowledge acquisition 1993) OntoSpecification — A translation approach to portable ontology specifications [Paper]
(Paper 2025) iCARE — iCARE: Ontology-Guided Intent Routing for Multi-Agent LLM-Based Dialogue Systems [Paper]
(arXiv 2026) OG-MAR — Toward Culturally Aligned LLMs through Ontology-Guided Multi-Agent Reasoning [Paper]
(Paper 2026) CAPAS — Agentic Information Architectures for Global Climate Governance: A Multi-Agent Decision-Support System for Cross-National Policy Analytics [Paper]
(Procedia CIRP 2026) LaMAS4PD — LaMAS4PD-A Multi-Agent LLM Approach for Ontology-Driven Structuring of Engineering Knowledge in Industry 4.0 [Paper]
(Available at SSRN 6919461 2026) Agentology — Agentology: Ontology-Driven Operational Environments for Multi-Agent Systems [Paper]
(npj Health Systems 2026) OntoCodex — OntoCodex: a multi-agent biomedical ontology enrichment framework [Paper]
(European Semantic Web Conference 2026) AgentO — AgentO: An Ontology for Modeling Agentic AI Systems [Paper]
(arXiv 2026) Ontology-to-Tools — Ontology-to-tools compilation for executable semantic constraint enforcement in LLM agents [Paper]
(Semantic Web 2026) Ontology SLR — Large language models for ontology engineering: a systematic literature review [Paper]
(arXiv 2026) Palantir Ontology — The Ontology System [Paper]

🏆 Benchmarks, Datasets, and Environments

Representative evaluation resources from the survey:

Model Intelligence

Individual Intelligence

System Intelligence

💻 Open-Source Libraries

Reusable projects grouped by their primary engineering target:

Model Intelligence

Individual Intelligence

System Intelligence

Graph Engineering

Ontology Engineering

Semantica

🚀 Applications

Representative systems and application domains covered by the survey:

Software Engineering and IT Operations

Scientific Discovery and Laboratory Automation

Healthcare and Clinical Decision Support

Enterprise Workflows and Digital Organizations

General-Purpose Digital Agents and Personal Automation

Social and Economic Simulation

🍀 Citation

If you find this repository useful, please cite the accompanying survey:

@article{feng2026graph,
 title={Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence},
 author={Feng, Yuyuan and Xiang, Zhishang and Yang, Chaobin and Ma, Qichao and Chen, Zerui and Zhang, Yujing and Huang, Ke and Wu, Chuanjie and Liu, Zhaoxu and Wang, Yili and others},
 journal={arXiv preprint arXiv:2608.21156},
 year={2026}
}

Acknowledgments

This list is maintained by the authors and community contributors. We thank the authors of all referenced works and open-source projects.

Name		Name	Last commit message	Last commit date
images		images
LICENSE		LICENSE
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation

Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence

🎉 News

Overview

Table of Contents

📚 Related Survey Papers

📜 Research Papers

Model Intelligence

Pre-Training

Post-Training

Prompt Engineering

Context Engineering

Individual Intelligence

Tool Integration

Memory Management

Skill Composition

Runtime Orchestration

Loop Architecture

Interaction Paradigm

Environment Feedback

System Intelligence

Task Organization

Agent Coordination

State Management

System Evolution

Ontology Engineering

🏆 Benchmarks, Datasets, and Environments

Model Intelligence

Individual Intelligence

System Intelligence

💻 Open-Source Libraries

Model Intelligence

Individual Intelligence

System Intelligence

Graph Engineering

Ontology Engineering

🚀 Applications

Software Engineering and IT Operations

Scientific Discovery and Laboratory Automation

Healthcare and Clinical Decision Support

Enterprise Workflows and Digital Organizations

General-Purpose Digital Agents and Personal Automation

Social and Economic Simulation

🍀 Citation

Acknowledgments

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors