Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Update examples/How_to_count_tokens_with_tiktoken.ipynb #2108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ChenyangLi4288 wants to merge 3 commits into openai:main
base: main
Choose a base branch
Loading
from ChenyangLi4288:main
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions authors.yaml
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
# You can optionally customize how your information shows up cookbook.openai.com over here.
# If your information is not present here, it will be pulled from your GitHub profile.

ChenyangLi4288:
name: "Chenyang Li"
website: "https://www.linkedin.com/in/chenyang-li-ba04a9249/"
avatar: "https://avatars.githubusercontent.com/u/113459182?v=4"

jonlim-openai:
name: "Jonathan Lim"
website: "https://www.linkedin.com/in/jonlmr"
Expand Down
198 changes: 198 additions & 0 deletions examples/How_to_count_tokens_with_tiktoken.ipynb
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -783,6 +783,204 @@
" print(f'{response.usage.prompt_tokens} prompt tokens counted by the OpenAI API.')\n",
" print()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. Counting tokens for chat completions with structured responses\n",
"\n",
"Structured responses allow you to define a JSON schema that the model must follow when generating responses. This feature adds additional tokens to account for the schema definition and validation instructions.\n",
"\n",
"Below is an example function for counting tokens for messages that include structured response schemas, passed to `gpt-3.5-turbo`, `gpt-4`, `gpt-4o` and `gpt-4o-mini`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def num_tokens_for_structured_response(response_format, messages, model):\n",
" \"\"\"\n",
" Return the number of tokens used by a list of messages with structured response format.\n",
" \n",
" Args:\n",
" response_format: The response format schema (e.g., {\"type\": \"json_object\"} or structured schema)\n",
" messages: List of messages\n",
" model: Model name\n",
" \n",
" Returns:\n",
" Total token count including schema overhead\n",
" \"\"\"\n",
" try:\n",
" encoding = tiktoken.encoding_for_model(model)\n",
" except KeyError:\n",
" print(\"Warning: model not found. Using o200k_base encoding.\")\n",
" encoding = tiktoken.get_encoding(\"o200k_base\")\n",
" \n",
" # Base message token count\n",
" messages_token_count = num_tokens_from_messages(messages, model)\n",
" \n",
" # Calculate tokens for response format\n",
" response_format_tokens = 0\n",
" \n",
" if response_format is None:\n",
" return messages_token_count\n",
" \n",
" # Simple JSON object format\n",
" if response_format == {\"type\": \"json_object\"}:\n",
" # The \"json_object\" format adds minimal overhead\n",
" response_format_tokens = 4 # Approximate tokens for JSON object instruction\n",
" else:\n",
" # Structured schema format - more complex\n",
" # This includes schema definition and validation instructions\n",
" schema_overhead = 0\n",
" \n",
" # Base overhead for structured response instructions\n",
" if model in [\"gpt-4o\", \"gpt-4o-mini\"]:\n",
" schema_overhead = 12 # Base overhead for o1 models\n",
" elif model in [\"gpt-3.5-turbo\", \"gpt-4\"]:\n",
" schema_overhead = 15 # Base overhead for older models\n",
" else:\n",
" schema_overhead = 12 # Default\n",
" \n",
" # Add tokens for schema content\n",
" schema_str = str(response_format)\n",
" schema_tokens = len(encoding.encode(schema_str))\n",
" \n",
" response_format_tokens = schema_overhead + schema_tokens\n",
" \n",
" total_tokens = messages_token_count + response_format_tokens\n",
" return total_tokens\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Example 1: Simple JSON object format\n",
"simple_json_format = {\"type\": \"json_object\"}\n",
"\n",
"# Example 2: Structured schema format\n",
"structured_schema = {\n",
" \"type\": \"json_schema\",\n",
" \"json_schema\": {\n",
" \"name\": \"extract_person_info\",\n",
" \"strict\": True,\n",
" \"schema\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"name\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"The person's full name\"\n",
" },\n",
" \"age\": {\n",
" \"type\": \"integer\",\n",
" \"description\": \"The person's age in years\"\n",
" },\n",
" \"email\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"The person's email address\"\n",
" },\n",
" \"occupation\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"The person's job title or occupation\"\n",
" }\n",
" },\n",
" \"required\": [\"name\", \"age\", \"email\"],\n",
" \"additionalProperties\": False\n",
" }\n",
" }\n",
"}\n",
"\n",
"example_messages = [\n",
" {\n",
" \"role\": \"system\",\n",
" \"content\": \"You are a helpful assistant that extracts information from text and formats it as structured data.\"\n",
" },\n",
" {\n",
" \"role\": \"user\", \n",
" \"content\": \"Please extract information about John Smith. He is 35 years old, works as a software engineer, and his email is john.smith@example.com.\"\n",
" }\n",
"]\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Compare token counts for different response formats\n",
"for model in [\"gpt-4o\", \"gpt-4o-mini\"]:\n",
" print(f\"\\n{model}:\")\n",
" \n",
" # No structured response\n",
" no_format_tokens = num_tokens_for_structured_response(None, example_messages, model)\n",
" print(f\" No structured response: {no_format_tokens} tokens\")\n",
" \n",
" # Simple JSON object format\n",
" simple_json_tokens = num_tokens_for_structured_response(simple_json_format, example_messages, model)\n",
" print(f\" Simple JSON object format: {simple_json_tokens} tokens (+{simple_json_tokens - no_format_tokens})\")\n",
" \n",
" # Structured schema format\n",
" structured_tokens = num_tokens_for_structured_response(structured_schema, example_messages, model)\n",
" print(f\" Structured schema format: {structured_tokens} tokens (+{structured_tokens - no_format_tokens})\")\n",
" \n",
" # Verify with actual API calls\n",
" print(\" API verification:\")\n",
" \n",
" # No format\n",
" response_no_format = client.chat.completions.create(\n",
" model=model,\n",
" messages=example_messages,\n",
" temperature=0,\n",
" max_tokens=1\n",
" )\n",
" print(f\" No format - API: {response_no_format.usage.prompt_tokens} tokens\")\n",
" \n",
" # Simple JSON format\n",
" response_simple = client.chat.completions.create(\n",
" model=model,\n",
" messages=example_messages,\n",
" response_format=simple_json_format,\n",
" temperature=0,\n",
" max_tokens=1\n",
" )\n",
" print(f\" Simple JSON - API: {response_simple.usage.prompt_tokens} tokens\")\n",
" \n",
" # Structured schema format\n",
" response_structured = client.chat.completions.create(\n",
" model=model,\n",
" messages=example_messages,\n",
" response_format=structured_schema,\n",
" temperature=0,\n",
" max_tokens=1\n",
" )\n",
" print(f\" Structured schema - API: {response_structured.usage.prompt_tokens} tokens\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Key observations about structured response token counting:\n",
"\n",
"1. **Simple JSON object format** (`{\"type\": \"json_object\"}`) adds minimal overhead (typically 4-6 tokens)\n",
"2. **Structured schema format** adds more significant overhead due to the schema definition and validation instructions\n",
"3. **Schema complexity matters** - more detailed schemas with more properties, descriptions, and constraints will consume more tokens\n",
"4. **Model differences** - Different models may have slightly different overhead for structured responses\n",
"\n",
"The token counting function above provides estimates that should be close to the actual API usage, but as with other token counting methods, consider these estimates rather than exact guarantees.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
Expand Down
12 changes: 12 additions & 0 deletions registry.yaml
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,18 @@
# should build pages for, and indicates metadata such as tags, creation date and
# authors for each page.

- title: Add More Examples for Structured Responses
path: examples/How_to_count_tokens_with_tiktoken.ipynb
date: 2025年09月02日
authors:
- ChenyangLi4288
tags:
- tiktoken
- token-counting
- structured-responses
- api-usage
- cost-estimation

- title: Automating Code Quality and Security Fixes with Codex CLI on GitLab
path: examples/codex/secure_quality_gitlab.md
date: 2025年08月29日
Expand Down

AltStyle によって変換されたページ (->オリジナル) /