-
Notifications
You must be signed in to change notification settings - Fork 699
Description
π Describe the bug
The issue: The Cpp runtime can't run nanogpt, and crashes with error op_index.cpp:94] Index -1 out of range .
Some context: I use executorch through Rust bindings to executorch, and I have an example running the nanogpt model. During a bump of the Cpp library from 0.7.0 to 1.0.0 the example stopped working. The reason I mention this (apart from the publicity π ) is that my runner does not use the cpp code at executorch/examples/llm_manual, and because the error is reproduced using both runners, and even when using an old export python script or .pte file, we can conclude that the bug is in the runtime library. Given that, I think this issue deserve some attention as it most likely affects other models and not just nanogpt.
Thanks in advance for the support! β€οΈ
How to reproduce
git clone --depth 1 --branch v1.0.0 https://github.com/pytorch/executorch.git cd executorch git submodule update --init --recursive --rebase; git submodule sync --recursive uv venv --python=3.12; source .venv/bin/activate # or any other virtualenv tool python -m ensurepip source ./install_requirements.sh --use-pt-pinned-commit source ./install_executorch.sh --use-pt-pinned-commit # at the time of writing, the issue also reproduces with torch nightly without pinned commit cd examples/llm_manual curl https://raw.githubusercontent.com/karpathy/nanoGPT/master/model.py -O # curl https://huggingface.co/openai-community/gpt2/resolve/main/vocab.json -O # had some weird redirect, had to resolved the link in a browser to get the actual vocab curl "https://huggingface.co/api/resolve-cache/models/openai-community/gpt2/607a30d783dfa663caf39e06633721c8d4cfcd7e/vocab.json?%2Fopenai-community%2Fgpt2%2Fresolve%2Fmain%2Fvocab.json=&etag=%221f1d9aaca301414e7f6c9396df506798ff4eb9a6%22" -O python export_nanogpt.py # CMakeLists.txt expect executorch to be at third-party/executorch mkdir third-party git clone --depth 1 --branch v1.0.0 https://github.com/pytorch/executorch.git third-party/executorch pushd third-party/executorch git submodule update --init --recursive --rebase; git submodule sync --recursive popd # Fix CMakeLists.txt to enable NamedDataMap extension sed -i '' '/option(EXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR "" ON)/a\ option(EXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP "" ON)\ ' CMakeLists.txt # edit manually if sed doesn't work on your platform mkdir build; cd build; cmake ..; cmake --build . -j16; cd .. ./build/nanogpt_runner # enter "hello" as a prompt
Error:
E 00:00:07.545846 executorch:op_index.cpp:94] Index -1 out of range for tensor with size 1 at dimension 1 E 00:00:07.546512 executorch:op_index.cpp:98] Check failed (is_valid_index): Some index values are not within bounds of input tensor at indexed dim E 00:00:07.546513 executorch:op_index.cpp:131] Check failed (check_fast_path_args(ctx, in, indices, dim, out)): E 00:00:07.546518 executorch:method.cpp:1355] KernelCall failed at instruction 0:608 in operator aten::index.Tensor_out: 0x12 E 00:00:07.546520 executorch:method.cpp:1361] arg 0 with type id 1 E 00:00:07.546521 executorch:method.cpp:1361] arg 1 with type id 11 E 00:00:07.546522 executorch:method.cpp:1361] arg 2 with type id 1 E 00:00:07.546523 executorch:method.cpp:1361] arg 3 with type id 1 F 00:00:07.546530 executorch:result.h:170] In function CheckOk(), assert failed: hasValue_ hellozsh: abort ./build/nanogpt_runner
Versions
Collecting environment information...
PyTorch version: 2.9.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 14.8.1 (arm64)
GCC version: Could not collect
Clang version: 20.1.5
CMake version: version 3.31.6
Libc version: N/A
Python version: 3.12.11 (main, Oct 7 2025, 15:53:48) [Clang 20.1.4 ] (64-bit runtime)
Python platform: macOS-14.8.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M3 Pro
Versions of relevant libraries:
[pip3] executorch==1.0.0a0+8c84780
[pip3] numpy==2.3.4
[pip3] pytorch_tokenizers==0.1.0
[pip3] torch==2.9.0
[pip3] torchao==0.14.0+git02941240f
[pip3] torchaudio==2.9.0
[pip3] torchdata==0.11.0+cpu
[pip3] torchsr==1.0.4
[pip3] torchtune==0.7.0+cpu
[pip3] torchvision==0.24.0
[conda] Could not collectcc @mergennachin @iseeyuan @lucylq @helunwencser @tarun292 @kimishpatel @jackzhxng @larryliu0820 @manuelcandales
Metadata
Metadata
Assignees
Labels
Type
Projects
Status