llm-md

Apache License 2.0 Version status-badge

Overview

LLM-MD is a markdown-based domain-specific language for defining, sharing, and version-controlling conversations between AI agents in plain text files. It lets you define AI workflows just like you manage code - as text files in your favorite editor, under version control, with full visibility and control.

The core idea is simple yet powerful: AI workflows should be as manageable as your source code. No proprietary formats, no special GUIs, just markdown files that define exactly how your AI agents should interact.

## context >>>
model = "urn:llm:openai:gpt-4"
system = "You are a helpful programming assistant."
### user >>> assistant
How can I implement a simple linked list in Python?
### assistant >>> 
Here's a simple implementation of a linked list in Python:
```python
class Node:
 def __init__(self, data):
 self.data = data
 self.next = None
class LinkedList:
 def __init__(self):
 self.head = None
 
 def append(self, data):
 new_node = Node(data)
 if not self.head:
 self.head = new_node
 return
 
 current = self.head
 while current.next:
 current = current.next
 current.next = new_node
```

Key Features

• Plain Text Workflows - Define complex agent interactions in markdown files.
• Agent Chain Patterns - Support for sequential flows (>>>), fan-out (>>=), fan-in (=>>), and iterative agent loops (!>>).
• Composable Agents - Use external files as agents, enabling modularity and reuse.
• Code Patch Management - Generate, apply, and revert code changes using a structured patch format.
• Context Management - TOML-based configuration for models, parameters, system prompts, and agent contracts.
• System Integration - Execute shell commands, incorporate their output, and generate context from your file system.
• Variable Management - Assign, manipulate, and use variables throughout conversations.
• Tools System - Built-in tools for file operations, search, data extraction, and system diagnostics with pipe composition.
• Rich Content Support - Markdown-style formatting with links, images, and code blocks.
• Command Line Interface - Simple execution with detailed control options.

Installation

Option 1: Direct Download

1. Go to the Releases page and download the latest release for your platform.
2. Extract the archive:

   tar -xzf llm-md-linux-x86_64.tar.gz
   

3. Add the binary to your PATH:

   sudo cp llm-md-dist/bin/llm-md /usr/local/bin/
   

Option 2: One-Line Installer

curl -fsSL https://llm.md/install.sh | bash

Option 3: Build from Source

If you have Racket installed:

git clone https://codeberg.org/anuna/llm-md.git
cd llm-md
raco pkg install --auto
raco exe -o llm-md cli.rkt

Option 4: Docker

docker pull anunaresearch/llm-md:latest
docker run -it --rm anunaresearch/llm-md

Basic Usage

1. Create a markdown file with your agent conversation:

### context >>>
model = "urn:llm:anthropic:claude-3-opus-20240229"
temperature = 0.7
system = "You are a helpful assistant with expertise in data science."
### user >>> assistant
What's the difference between a decision tree and random forest?
### assistant >>> 

2. Alternatively, use the interactive wizard to create a new file:

llm-md create conversation.md
llm-md create conversation.md --template coding

3. Run the file through llm-md:

llm-md evaluate conversation.md

The response will be appended to your file, maintaining a complete record of the conversation.

Command Line Options

Modern command-based usage:

llm-md parse file.md # Parse an LLM-MD file and print the AST
llm-md evaluate file.md # Process an LLM-MD file with default behavior
llm-md run file.md # Alias for 'evaluate'
llm-md prompt file.md # Extract raw prompt as JSON to stdout
llm-md create file.md # Create a new LLM-MD file with interactive wizard
llm-md create context ./src # Generate context from directory structure
llm-md apply file.md # Apply code patches from a conversation
llm-md patch file.md # Alias for 'apply'
llm-md revert file.md [target] # Revert patches from a conversation
llm-md version # Display version information
llm-md validate file.md # Check syntax without executing the file
llm-md update # Update to the latest version
llm-md help # Show help information
llm-md tools # Show available tools and operations

Legacy flag-based usage (maintained for compatibility):

llm-md -p file.md # Parse an LLM-MD file (same as 'parse file.md')
llm-md file.md # Process a file (same as 'evaluate file.md')
llm-md -v # Display version information
llm-md -u # Update to the latest version

Common options for all commands:

--no-append # Don't append to the original file
--provider anthropic # Specify LLM provider
--model claude-3-opus-20240229 # Specify model
--batch # Use provider's batch API for cheaper, non-urgent processing (OpenAI only)
-d, --verbose # Show detailed debug information
--splice FILE-OR-TEXT # Content to splice into {{@input}} tags

Batch Processing

For large-scale, non-urgent processing tasks, LLM-MD supports batch processing using provider APIs that offer significant cost savings:

# Process a file using OpenAI's batch API (50% cost reduction)
llm-md evaluate conversation.md --batch --provider openai
# The batch job will be submitted and you'll receive a batch ID
# Check status with: llm-md batch-status <batch-id>
# Results will be automatically applied when the batch completes

Benefits of batch processing:

• Cost savings: Up to 50% reduction in API costs (OpenAI)
• Efficiency: Optimized for large-scale processing
• Asynchronous: Non-blocking operation for background tasks

Limitations:

• Currently only supported for OpenAI provider
• Processing may take 24-48 hours to complete
• Best suited for non-urgent, bulk processing tasks

Syntax Guide

Context Configuration

The context block defines settings for the conversation:

### context >>>

model = "urn:llm:openai:gpt-4"
temperature = 0.7
system = """
You are a helpful assistant with expertise in Python programming.
"""
patch = true # Enable patch generation mode for the LLM
[tools]
search = true
database = true

Agent Chain Patterns

LLM-MD supports multiple patterns for agent interaction:

# Sequential Flow (A → B → C)
### user >>> assistant >>> validator

# Fan-Out Pattern (one to many)
### source >>= (collector1 >>> processor1) (collector2 >>> processor2)

# Fan-In Pattern (many to one)
### (analyzer1) (analyzer2) =>> aggregator

# Agent Loop (iterative execution)
### generator !>> critic

Commands and Variables

# Variable Assignment
{{@count = 1}}
{{@name = "Alice"}}
# Shell Integration (execute command and incorporate output)
{{@shell $ ls -la}}
# Tools System (built-in command execution and composition)
{{@file:read $ "config.json"}} # Read a file
{{@search:text $ "TODO" --type=racket}} # Search for text
{{@shell:exec $ "ls -la"}} # Execute shell command
{{files = @file:list $ "src/"}} # Store result in variable
{{@search:text $ "TODO" |> @extract:field $ "file"}} # Pipe operations
# Deferred Execution (queues command for the eval! agent)
{{eval! {{@some_command}} }}
# Control Statements
{{@return @result}}
{{@break}}
{{@continue}}

Tools System

LLM-MD includes a comprehensive tools system for executing commands and composing tool chains:

# Basic tool syntax
{{@tool:operation $ arguments --flags}}
# File operations
{{@file:read $ "data.json"}}
{{@file:write $ "output.txt" "Hello World"}}
{{@file:list $ "src/"}}
# Search and extract
{{@search:text $ "function" --type=racket}}
{{@extract:field $ data "name"}}
# Shell commands
{{@shell:exec $ "git status"}}
{{@shell:which $ "python"}}
# Tool composition with pipes
{{@file:list $ "src/" |> @extract:item $ 0 |> @file:read $}}
# Variable assignment and reuse
{{files = @file:list $ "src/"}}
{{content = @file:read $ "@{files[0]}"}}
{{@content}}
# Tool introspection
{{@tools:list $}} # List all available tools
{{@tools:describe $ "file"}} # Get tool documentation
{{@tools:help $ "syntax"}} # Get syntax help

Built-in Tools:

• file: Read, write, list, copy, delete files
• search: Text and function pattern search
• shell: Execute shell commands safely
• extract: Data extraction and manipulation
• diagnostics: System information and health checks
• tools: Tool discovery and help system

Agent as a File

LLM-MD supports using external files as modular, reusable agents:

### context >>>
reviewer = "../agents/code_reviewer.md"
expert = "https://llm.md/expert.md"
### user >>> @reviewer
Please review this Python code:
```python
def add(a, b):
 return a + b
```
### @reviewer >>> 
Here's my review of the code...

Real-World Examples

Multi-Agent Workflow: Code Generation and Review

### context >>>
model = "urn:llm:openai:gpt-4"
system = "You are a helpful coding assistant."
reviewer = "./agents/code_reviewer.md"
### user >>> assistant
Write a Python function to check if a string is a palindrome.
### assistant >>> @reviewer

```python
def is_palindrome(string):
 # Remove spaces and convert to lowercase
 string = string.replace(" ", "").lower()
 # Check if the string equals its reverse
 return string == string[::-1]
```
### @reviewer >>> user
The code is correct and efficient, but I have a few suggestions...

Automated Code Patching

File Processing with Tools

LLM-MD tools can automate file operations and data processing:

### context >>>
model = "urn:llm:anthropic:claude-3-5-sonnet-20240620"
### user >>> assistant
Analyze the TODO comments in my Racket source files:
{{todos = @search:text $ "TODO" --type=rkt}}
{{@todos}}
Now read the main configuration file:
{{config = @file:read $ "info.rkt"}}
{{@config}}
Please analyze the TODO items and suggest priorities based on the project structure.
### assistant >>>
Based on your TODO items and project configuration, here are my recommendations...

Automated Code Patching

LLM-MD can instruct an agent to generate code modifications in a special patch format. These patches can then be automatically applied or reverted.

1. The Conversation (fix_bug.md)

### context >>>
model = "urn:llm:anthropic:claude-3-5-sonnet-20240620"
system = "You are an expert python programmer. When fixing code, you must provide your changes inside a patch block."
patch = true # Instruct the LLM to use the patch format
### user >>> assistant
The following file has a bug. The greeting is wrong. Please fix it.
{{@shell $ cat ./app/main.py}}
### assistant >>>
Of course. The greeting should be "Hello, World!". Here is the patch to fix the file:
#### -- Begin Patch --
##### Update: ./app/main.py
```diff
 def main():
- print("Hallo, Welt!")
+ print("Hello, World!")
 
 if __name__ == "__main__":
 main()
```
#### -- End Patch --

2. Apply the Patch

After the LLM generates the response containing the patch, you can apply it from your terminal.

# First, run the conversation to get the agent's response
llm-md evaluate fix_bug.md
# Now, apply the generated patch
llm-md apply fix_bug.md

LLM-MD will find the patch block in the conversation file and apply the changes to ./app/main.py. You can later undo this with llm-md revert fix_bug.md.

Advanced Features

Applying & Reverting Code Patches

When an LLM is in patch = true mode, it is instructed to wrap file modifications in a specific format that llm-md can parse.

#### -- Begin Patch --
##### <Update|Create|Delete>: <path/to/file>
```diff
- optional diff content for updates
+ or full file content for creates
```
#### -- End Patch --

• apply command: Scans the last agent message for patch blocks and applies them to the specified files.
• revert command: Reverts the last set of applied patches from a given conversation file.

Agent Contract System

LLM-MD provides a flexible agent contract system that allows defining agents with semantic type indicators and comprehensive configuration options. This system enables multiple ways to define and configure agents while maintaining full composability.

Agent Definition Types

Agents can be defined using a semantic type field in contracts:

1. Inline Agents (type = "inline") Complete agent definition within the contract:

### context >>>
[contracts.agents.joker]
description = "Creates award winning jokes"
requires = ["feedback"]
provides = ["draft"]
type = "inline"
model = "urn:llm:anthropic:claude-sonnet-4-20250214"
system = "You are a comedian who tells excellent jokes"
parameters = { temperature = 0.9, max_tokens = 1000 }
### user >>> @joker
Tell me a joke about programming.

2. File-based Agents (type = "file") Agent defined in external LLM-MD file:

### context >>>
[contracts.agents.reviewer]
description = "Code review specialist"
requires = ["code"]
provides = ["feedback"]
type = "file"
source = "agents/code_reviewer.md"
### user >>> @reviewer
Please review this Python function...

3. URL-based Agents (type = "url") Agent loaded from remote URL:

### context >>>
[contracts.agents.expert]
description = "Domain expert agent"
type = "url"
source = "https://example.com/agents/expert.md"

4. URN-based Agents (type = "urn") Agent defined by model URN:

### context >>>
[contracts.agents.simple]
description = "Simple model-based agent"
type = "urn"
source = "urn:llm:openai:gpt-4"

Type Inference

When no explicit type is specified, the system automatically infers the type:

• model field present → inline
• source with http:// or https:// → url
• source with urn: → urn
• source with file path → file
• Otherwise → default (legacy behavior)

Configuration Flexibility

All agent types support arbitrary configuration fields, allowing for complete control over agent behavior:

### context >>>
[contracts.agents.specialized]
type = "inline"
model = "urn:llm:anthropic:claude-3-opus-20240229"
system = "You are a specialized assistant"
parameters = { 
 temperature = 0.1, 
 max_tokens = 2000,
 top_p = 0.9 
}
custom_field = "any value"
provider_specific_option = true

Agent Loops (!>>)

The !>> operator creates a loop where a chain of agents is executed iteratively. This is useful for building autonomous systems where agents refine work, validate outputs, or perform multi-step tasks until a goal is reached.

### context >>>
model = "urn:llm:anthropic:claude-3-5-haiku-20241022"
[contracts.variables]
draft = { type = "string", scope = "shared", init = "", description = "The short story" }
feedback = { type = "string", scope = "shared", init = "" }
[contracts.agents.writer]
provides = ["draft"]
requires = ["feedback"]
type = "file"
source = "agents/writer.md"
[contracts.agents.critic]
provides = ["feedback"]
requires = ["draft"]
type = "inline"
model = "urn:llm:anthropic:claude-3-5-haiku-20241022"
system = "You are a literary critic who provides constructive feedback"
[contracts.protocols]
max-iterations = 4
### @writer !>> @critic
Write a short story about a robot who discovers music. The critic will provide feedback. Iterate until the story is complete.

In this loop:

• The writer agent runs first, receiving the initial user prompt and any existing feedback (initially empty), and must produce a value for the draft variable.
• The critic agent runs next, receiving the draft and providing feedback.
• The loop repeats for up to 4 iterations, with each agent improving the work based on the other's input.

Shell Command Integration

Execute shell commands and use their output in conversations:

### user >>> assistant
What's in this directory?
{{@shell $ ls -la}}
### assistant >>> 
Based on the directory listing, I can see you have...

Technical Details

LLM-MD is implemented in Racket and uses:

• Parser Module: Lexical analysis and parsing for LLM-MD syntax.
• Interpreter Module: Execution of LLM-MD files, including agent chains, loops, commands, and interaction with LLM providers.
• Tools Module: Comprehensive tools system with registry, pipe execution, variable scoping, and built-in operations.
• Patch Module: Handles the application and reversion of code patches using a diffing engine.
• LLMs Module: Management of provider configurations and model specifications.

The parser builds an Abstract Syntax Tree (AST) from your markdown file, which the interpreter then executes to manage conversation flow between agents.

Contributing

Contributions to llm-md are welcome! This is an alpha release, so the API may change significantly.

License

Apache License Version 2.0

BNF grammar

<llm-md-file> ::= <messages>
<messages> ::= <message> 
 | <message> <messages>
<message> ::= <context-message>
 | <user-message>
 | <agent-message>
<context-message> ::= "### context >>>" NEWLINE <toml-section>
<user-message> ::= "### user" [<agent-chain>] NEWLINE <user-message-content>
<agent-message> ::= "###" <agent-name> [<agent-chain>] NEWLINE <agent-message-content>
<agent-name> ::= TEXT /* Any sequence of alphabetic characters */
<toml-section> ::= <toml-lines>
<toml-lines> ::= <toml-line>
 | <toml-line> NEWLINE <toml-lines>
<toml-line> ::= <toml-tokens>
<toml-tokens> ::= <toml-token>
 | <toml-token> <toml-tokens>
<toml-token> ::= TEXT | URI | "@" VARIABLE-NAME | NEWLINE | "=" 
 | "\"" | "(" | ")" | "[" | "](" | ":" | ">>>"
 | ">>=" | "=>>" | "!>>" | "{{" | "}}" | "!!" | "??"
 | ";;" | "```" | "_" | "$"
<agent-chain> ::= <agent-elements>
 | <agent-chain> <operation> <agent-elements>
<agent-elements> ::= <agent-element>
 | "(" <agent-chain> ")"
<agent-element> ::= <agent-with-label>
 | "@" VARIABLE-NAME
 | ε
<agent-with-label> ::= TEXT
 | TEXT ":" TEXT
<operation> ::= ">>=" /* Fan-Out */
 | ">>>" /* Sequential Flow */
 | "=>>" /* Fan-In */
 | "!>>" /* Agent Loop */
/* User message content supports full LLM-MD syntax */
<user-message-content> ::= <content-items>
<content-items> ::= ε
 | <content-item> <content-items>
<content-item> ::= TEXT
 | "\"" TEXT "\""
 | <link>
 | <image>
 | <llm-md-command>
 | <comment>
 | "```" <escaped-content> "```"
 | NEWLINE
/* Agent message content is treated as plain text */
<agent-message-content> ::= <plain-text-items>
<plain-text-items> ::= ε
 | <plain-text-item> <plain-text-items>
<plain-text-item> ::= TEXT /* All special syntax is treated as plain text */
 | NEWLINE
<escaped-content> ::= <escaped-lines>
<escaped-lines> ::= <escaped-line>
 | <escaped-line> NEWLINE
 | <escaped-line> NEWLINE <escaped-lines>
<escaped-line> ::= <escaped-tokens>
<escaped-tokens> ::= <escaped-token>
 | <escaped-token> <escaped-tokens>
<escaped-token> ::= TEXT | URI | VARIABLE-NAME | "=" | "\"" | "(" | ")"
 | "[" | "](" | ":" | ">>>" | ">>=" | "=>>" | "{{"
 | "}}" | "!!" | "??" | ";;" | "_" | "$"
<link> ::= "[" TEXT "](" URI ")"
 | "[" TEXT "](" URI TEXT ")"
 | "[" "](" URI ")" /* Empty link text */
<image> ::= "![" TEXT "](" URI ")"
 | "![" TEXT "](" URI TEXT ")"
 | "![" "](" URI ")" /* Empty alt text */
<llm-md-command> ::= "{{" <force-modifier>? <command-content> "}}"
<force-modifier> ::= "!!"
<command-content> ::= <control-statement>
 | <assignment>
 | <variable-operation>
 | <shell-command>
 | <tool-command>
 | <pipe-expression>
 | "??"
 | <comment>
 | "@" VARIABLE-NAME
 | TEXT
<assignment> ::= "@" VARIABLE-NAME "=" <expression>
<variable-operation> ::= "@" VARIABLE-NAME <expression>
<expression> ::= TEXT
 | "\"" TEXT "\""
 | "@" VARIABLE-NAME
 | "??"
 | <llm-md-command>
<shell-command> ::= "$" <remaining-text>
 | "@" VARIABLE-NAME "$" <remaining-text>
 | TEXT "$" <remaining-text>
<tool-command> ::= "@" TEXT ":" TEXT "$" <tool-arguments>
<pipe-expression> ::= <tool-command> <pipe-operator> <tool-command>
 | <pipe-expression> <pipe-operator> <tool-command>
<pipe-operator> ::= "|>"
<tool-arguments> ::= <tool-argument>
 | <tool-argument> <tool-arguments>
 | ε
<tool-argument> ::= TEXT
 | "\"" TEXT "\""
 | "@{" VARIABLE-NAME "}"
 | "--" TEXT "=" TEXT
<remaining-text> ::= <shell-token>
 | <shell-token> <remaining-text>
<shell-token> ::= TEXT | URI | VARIABLE-NAME | "=" | NEWLINE | "\""
 | "(" | ")" | "[" | "](" | ":" | ">>>" | ">>="
 | "=>>" | "{{" | "!!" | "??" | ";;" | "```" | "_" | "$"
<control-statement> ::= "@return" <expression>
 | "@break"
 | "@continue"
 | "@input"
<comment> ::= ";;" TEXT NEWLINE