Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

v1.0.1 candidate: Cpp runtime cant run nanogpt #15285

Open
Assignees
Labels
module: examplesIssues related to demos under examples/ module: kernelsIssues related to kernel libraries and utilities, and code under kernels/
@barakugav

Description

πŸ› Describe the bug

The issue: The Cpp runtime can't run nanogpt, and crashes with error op_index.cpp:94] Index -1 out of range .

Some context: I use executorch through Rust bindings to executorch, and I have an example running the nanogpt model. During a bump of the Cpp library from 0.7.0 to 1.0.0 the example stopped working. The reason I mention this (apart from the publicity πŸ™ƒ ) is that my runner does not use the cpp code at executorch/examples/llm_manual, and because the error is reproduced using both runners, and even when using an old export python script or .pte file, we can conclude that the bug is in the runtime library. Given that, I think this issue deserve some attention as it most likely affects other models and not just nanogpt.

Thanks in advance for the support! ❀️

How to reproduce

git clone --depth 1 --branch v1.0.0 https://github.com/pytorch/executorch.git
cd executorch
git submodule update --init --recursive --rebase; git submodule sync --recursive
uv venv --python=3.12; source .venv/bin/activate # or any other virtualenv tool
python -m ensurepip
source ./install_requirements.sh --use-pt-pinned-commit
source ./install_executorch.sh --use-pt-pinned-commit
# at the time of writing, the issue also reproduces with torch nightly without pinned commit
cd examples/llm_manual
curl https://raw.githubusercontent.com/karpathy/nanoGPT/master/model.py -O
# curl https://huggingface.co/openai-community/gpt2/resolve/main/vocab.json -O
# had some weird redirect, had to resolved the link in a browser to get the actual vocab
curl "https://huggingface.co/api/resolve-cache/models/openai-community/gpt2/607a30d783dfa663caf39e06633721c8d4cfcd7e/vocab.json?%2Fopenai-community%2Fgpt2%2Fresolve%2Fmain%2Fvocab.json=&etag=%221f1d9aaca301414e7f6c9396df506798ff4eb9a6%22" -O
python export_nanogpt.py
# CMakeLists.txt expect executorch to be at third-party/executorch
mkdir third-party
git clone --depth 1 --branch v1.0.0 https://github.com/pytorch/executorch.git third-party/executorch
pushd third-party/executorch
git submodule update --init --recursive --rebase; git submodule sync --recursive
popd
# Fix CMakeLists.txt to enable NamedDataMap extension
sed -i '' '/option(EXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR "" ON)/a\
option(EXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP "" ON)\
' CMakeLists.txt
# edit manually if sed doesn't work on your platform
mkdir build; cd build; cmake ..; cmake --build . -j16; cd ..
./build/nanogpt_runner
# enter "hello" as a prompt

Error:

E 00:00:07.545846 executorch:op_index.cpp:94] Index -1 out of range for tensor with size 1 at dimension 1
E 00:00:07.546512 executorch:op_index.cpp:98] Check failed (is_valid_index): Some index values are not within bounds of input tensor at indexed dim
E 00:00:07.546513 executorch:op_index.cpp:131] Check failed (check_fast_path_args(ctx, in, indices, dim, out)):
E 00:00:07.546518 executorch:method.cpp:1355] KernelCall failed at instruction 0:608 in operator aten::index.Tensor_out: 0x12
E 00:00:07.546520 executorch:method.cpp:1361] arg 0 with type id 1
E 00:00:07.546521 executorch:method.cpp:1361] arg 1 with type id 11
E 00:00:07.546522 executorch:method.cpp:1361] arg 2 with type id 1
E 00:00:07.546523 executorch:method.cpp:1361] arg 3 with type id 1
F 00:00:07.546530 executorch:result.h:170] In function CheckOk(), assert failed: hasValue_
hellozsh: abort ./build/nanogpt_runner

Versions

Collecting environment information...
PyTorch version: 2.9.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 14.8.1 (arm64)
GCC version: Could not collect
Clang version: 20.1.5
CMake version: version 3.31.6
Libc version: N/A
Python version: 3.12.11 (main, Oct 7 2025, 15:53:48) [Clang 20.1.4 ] (64-bit runtime)
Python platform: macOS-14.8.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M3 Pro
Versions of relevant libraries:
[pip3] executorch==1.0.0a0+8c84780
[pip3] numpy==2.3.4
[pip3] pytorch_tokenizers==0.1.0
[pip3] torch==2.9.0
[pip3] torchao==0.14.0+git02941240f
[pip3] torchaudio==2.9.0
[pip3] torchdata==0.11.0+cpu
[pip3] torchsr==1.0.4
[pip3] torchtune==0.7.0+cpu
[pip3] torchvision==0.24.0
[conda] Could not collect

cc @mergennachin @iseeyuan @lucylq @helunwencser @tarun292 @kimishpatel @jackzhxng @larryliu0820 @manuelcandales

Metadata

Metadata

Labels

module: examplesIssues related to demos under examples/ module: kernelsIssues related to kernel libraries and utilities, and code under kernels/

Type

No type

Projects

Status

To triage

Milestone

No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      AltStyle γ«γ‚ˆγ£γ¦ε€‰ζ›γ•γ‚ŒγŸγƒšγƒΌγ‚Έ (->γ‚ͺγƒͺγ‚ΈγƒŠγƒ«) /