Misinformation Propagation in Benign Multi-Agent Systems

This is the offical repository for the paper "Misinformation Propagation in Benign Multi-Agent Systems".

MINT studies how misinformation affects large language models in single-agent and multi-agent debate settings. The repository provides:

• the MINT dataset with task instances, false facts, and eight misinformation strategies
• experiment scripts for single-agent baselines, multi-agent debates (via MALLM), and agent-composition sweeps
• plotting utilities to reproduce paper figures

Create an environment:

conda create --name mint python=3.11
conda activate mint
pip install torch transformers datasets numpy pandas matplotlib seaborn tqdm requests

Clone MALLM next to this repository (required for exp2.py, exp2_ablation.py, and exp3.py):

github/
 mint/ # this repo
 mallm/ # MALLM framework

Multi-agent experiments expect an OpenAI-compatible model endpoint (vLLM, SGLang, TGI, or similar). Slurm job scripts are provided for cluster runs.

The released benchmark is in MINT-dataset_v1.1/:

Each instance includes a false_fact, misinformation_by_strategy (clickbait, hoax, rumor, satire, propaganda, framing, conspiracy, other), and irrelevant_true_information as a control.

To regenerate or extend datasets:

python download_datasets.py --use_config_samples --datasets winogrande ethics_commonsense complex_web_questions

Baseline vs. misinformed single-agent prompting (local HuggingFace or OpenAI-compatible API):

python exp1.py --model_name meta-llama/Llama-3.3-70B-Instruct --inference openai --endpoint_url http://127.0.0.1:8080/v1

exp1a.py runs the same setup with irrelevant true information instead of misinformation.

3-agent MALLM debates across datasets and misinformation strategies:

python exp2.py --endpoint_url http://127.0.0.1:8080/v1 --model_name meta-llama/Llama-3.3-70B-Instruct

exp2_ablation.py runs the same setup without misinformation.

5-agent debates on WinoGrande, sweeping the number of misinformed agents (0–5):

python exp3.py --endpoint_url http://127.0.0.1:8080/v1 --model_name meta-llama/Llama-3.3-70B-Instruct

Quick smoke test (no model server):

python exp2.py --mock --debug

Results are written to out/<model_name>/. Use --continue to resume unfinished runs.

Generate plots from saved results:

python exp1_figures.py
python exp2_figures.py
python exp3_figures.py
python exp1_exp2_comparison.py

If you use this repository, please cite the paper and the MALLM framework:

@misc{becker2026,
 author={Becker, Jonas and Wahle, Jan Philip and Ruas, Terry and Gipp, Bela},
 title={Misinformation Propagation in Benign Multi-Agent Systems},
 year={2026},
 month={06}
}

@inproceedings{becker-etal-2025-mallm,
 title = "{MALLM}: Multi-Agent Large Language Models Framework",
 author = "Becker, Jonas and Kaesberg, Lars Benedikt and Bauer, Niklas and Wahle, Jan Philip and Ruas, Terry and Gipp, Bela",
 booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
 year = "2025",
 url = "https://aclanthology.org/2025.emnlp-demos.29/"
}