Current safety evaluation for large models lacks comprehensive standardized protocols and dedicated assessment tools. DeepSafe is the first all-in-one framework integrating 25+ safety datasets and the specialized ProGuard evaluation model, supporting full-modal LLM/VLM assessment.

DeepSafe is part of DeepSight and works best with 🔍 DeepScan (LLM/MLLM diagnosis toolkit). See the unified evaluation–diagnosis workflow on the AI45 HomePage.

• 2026年07月15日: DeepSafe-Sci is now available on the deepsafe-sci. It provides dedicated evaluation workflows for SciHazard, Safe-Scientist, and SOSBench, covering harmful scientific assistance, safe handling of risky research requests, and over-refusal in scientific settings.
• 🔥🔥🔥 2026年02月06日: DeepSafe's latest leaderboards and in-depth analyses are out now! Our evaluation comprehensively covers mainstream LLMs and MLLMs, including GPT, Claude, Gemini, DeepSeek, Qwen, Llama, and Mistral, providing a deep dive into the current state of AI safety. Access full results: Leaderboard | Frontier Risk Report | Content Risk Report

DeepSafe features a modular, configuration-driven elastic architecture, enabling a full-link automated closed loop from inference and generation to judgment and deep evaluation reporting. It provides a deeply evaluable, reproducible, and highly scalable evolving security infrastructure for AI Safety research, aiming to drive safety assessment from superficial testing to in-depth analysis and accelerate the construction of Trustworthy AI. 🚀

• High Extensibility: Powered by a Registry mechanism, new components (datasets, metrics, etc.) can be integrated with minimal code. It supports one-click assembly through YAML and allows evaluation pipelines to be decoupled and reused.
• Streamlined Usability: Adopts a "Config-as-Execution" paradigm. Simply provide a single YAML file, and the framework automatically completes the full cycle and generates standardized reports.
• Comprehensive Coverage: Provides granular outputs—including evaluation scores, detailed response logs, error sampling, and human-readable Markdown reports—facilitating in-depth analysis and reproduction.

• Proactive Risk Identification: Introduces a pioneering proactive detection paradigm capable of reasoning about and describing unknown risks, transcending the rigid constraints of predefined classification systems.
• Eradicating Modality Bias: Implements a hierarchical multimodal safety taxonomy trained on a balanced dataset of 87,000 samples, ensuring fair and precise risk assessment across both text and visual modalities.

DeepSafe supports major open-source models and commercial APIs, allowing flexible switching between evaluation backends.

DeepSafe provides a standardized workflow consisting of four stages: Configure -> Inference -> Evaluation -> Visualization.

# Recommended to use a virtual environment
pip install -r requirements.txt
# huggingface-cli is required for downloading datasets
pip install -U huggingface_hub
# Verify if the environment is set up correctly
python smoke_test.py

Most datasets supported by DeepSafe are hosted on Hugging Face. You can use huggingface-cli to download them.

Download Example (e.g., Do-Not-Answer):

# Download dataset to a local directory
huggingface-cli download --repo-type dataset --resume-download LibrAI/do-not-answer --local-dir data/do-not-answer --local-dir-use-symlinks False

After downloading, please update the dataset.path field in the corresponding YAML configuration file (e.g., configs/eval_tasks/do_not_answer_v01.yaml) to point to your local path.

The Salad-Bench dataset needs to be manually downloaded from the official repository. You can use the following command:

# Download the dataset
huggingface-cli download --repo-type dataset --resume-download OpenSafetyLab/Salad-Data --local-dir Salad-Data --local-dir-use-symlinks False

Then move or copy base_set.json to your specified local directory. Fill in your local path in the dataset.path field of the configuration file configs/eval_tasks/salad_judge_local.yaml:

dataset:
 type: SaladDataset
 path: /your/path/Salad-Data/base_set.json

mdjudge (MD-Judge-v0.1) is the safety evaluator officially recommended by Salad-Bench. The model weights can be downloaded via HuggingFace:

# Download safetensors weight files and config files
huggingface-cli download --resume-download OpenSafetyLab/MD-Judge-v0.1 --local-dir MD-Judge-v0.1 --local-dir-use-symlinks False

Place the MD-Judge-v0.1 folder in any specified local directory. Then, in the evaluator.judge_model_cfg.model_name field, replace it with the local path, for example:

evaluator:
 judge_model_cfg:
 type: VLLMLocalModel
 model_name: /your/local/dir/MD-Judge-v0.1
 # Other configurations remain unchanged

Note: For the first-time use, ensure the evaluation machine can load the weight files, and the local configuration path matches the actual download path; otherwise, the evaluation will fail to find the model.

dataset:
 type: SaladDataset
 path: /your/path/to/Salad-Data/base_set_sample_100.json
evaluator:
 type: ScorerBasedEvaluator
 batch_size: 32
 template_name: md_judge_v0_1
 judge_model_cfg:
 type: VLLMLocalModel
 model_name: /your/path/to/MD-Judge-v0.1
 tensor_parallel_size: 1
 gpu_memory_utilization: 0.4
 trust_remote_code: true
 temperature: 0.0
 max_tokens: 64

If there are other dependencies or execution errors, please verify the config paths and model formats.

DeepSafe provides one-shot local scripts. Simply run:

# Execute from the project root
bash scripts/run_salad_local.sh configs/eval_tasks/salad_judge_local.yaml

• Configure: Specify target model, dataset paths, and judge model parameters in the YAML file.
• Inference: The script automatically starts vLLM (local mode) or calls APIs to generate responses, saved in predictions.jsonl.
• Evaluation: Launches ProGuard or other judge models to automate scoring and categorization.
• Visualization: Summarizes metrics and generates a human-readable report.md in the output directory.

Parameters explained using salad_judge_v01_qwen1.5-0.5b_vllm_local.yaml:

Integrate your own dataset in three simple steps:

Ensure your data file follows this JSONL structure: {"id": "001", "prompt": "User Input", "reference": "Ground Truth", "category": "Label"}

• Dataset (uni_eval/datasets/): Inherit BaseDataset and override load() to read your JSONL.
• Metric (uni_eval/metrics/): Inherit BaseMetric and override compute() to calculate scores.
• Evaluator: (Optional) Inherit BaseEvaluator for custom multi-stage judging logic.

Register your modules in the respective __init__.py files, create your YAML config, and you are ready!

DeepSafe/
├── uni_eval/ # Core Evaluation Framework
│ ├── datasets/ # Dataset Loader Implementations
│ ├── models/ # Model Adapters (API/HF/vLLM)
│ ├── evaluators/ # Evaluation Logic Controllers
│ ├── metrics/ # Evaluation Metric Implementations
│ ├── runners/ # Task Execution Management
│ ├── summarizers/ # Result Summarization and Reporting
│ ├── cli/ # CLI Parsing Tools
│ └── registry.py # Registry Center
├── configs/ # YAML Configuration Files
├── scripts/ # Launch Scripts (Shell)
├── tools/ # Utility Scripts
└── results/ # Evaluation Outputs (JSON/Markdown)

Contributions are welcome! Please submit an Issue or Pull Request to help us build DeepSafe. 🌟

• Email: shaojing@pjlab.org.cn