Red Teaming python-framework for testing chatbots and GenAI systems
License: CC BY-NC-SA 4.0 PyPI - Python Version PyPI Docker Downloads Downloads GitHub Repo stars Chat
pip install llamator==3.4.0
Documentation Link: https://LLAMATOR-Core.github.io/llamator
- π RAG bot testing via REST API
- π§ββοΈ Gandalf web bot testing via Selenium
- π¬ Telegram bot testing via Telethon
- π± WhatsApp bot testing via Selenium
- π LangChain client testing with custom attack
- π· Vision Model testing via LangChain
- π Running LLAMATOR in Docker
- π All LangChain clients
- π§ OpenAI-like API
- βοΈ Custom Class (Telegram, WhatsApp, Selenium, etc.)
- οΈπ‘ Support for custom attacks and datasets from the user
- π Large selection of attacks on LLMs, RAGs, Agents, VLMs in English and Russian
- π‘ Custom configuration of chat clients
- π History of attack requests and responses in Excel and CSV format
- π Test report document in DOCX format
- π LLM01: Prompt Injection and Jailbreaks
- π΅οΈββοΈ LLM07: System Prompt Leakage
- π€₯ LLM09: Misinformation
- πΈ LLM10: Unbounded Consumption
- π¬ Telegram Chat β LLAMATOR | AI Red Team Community
- π€ Supported by AI Talent Hub and ITMO University
βοΈ Email: team@llamator.org
β‘οΈComing Soonβ‘οΈ
Β© Roman Neronov, Timur Nizamov, Nikita Ivanov
This project is licensed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license. See the LICENSE file for details.