-
Notifications
You must be signed in to change notification settings - Fork 584
Open
@sschnug
Description
With current dev (commit 684a044 (HEAD -> dev, origin/dev)) following the Readme.md using the bazel-based path:
Gemma2: Looks ok
https://www.kaggle.com/models/google/gemma-2/gemmaCpp/2.0-9b-it-sfp
bazel-bin/gemma --tokenizer tokenizer.spm --weights 2.0-2b-it-sfp.sbs --prompt_file /mnt/c/Users/XXX/Desktop/llm_prompt.txt
1180131840 blob bytes (36.83%) of bf16
2024275968 blob bytes (63.17%) of sfp
normalizer.cc(52) LOG(INFO) precompiled_charsmap is empty. use identity normalization.
[ Reading prompt ] ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[ Timing info ] Prefill: 24927 ms for 3829 prompt tokens (153.60 tokens / sec); Time to first token: 25039 ms
...
Gemma3: Assertion error
https://www.kaggle.com/models/google/gemma-3/gemmaCpp/3.0-1b-it-sfp
bazel-bin/gemma --tokenizer tokenizer.spm.GEMMA3 --weights 1b-it-sfp.sbs.GEMMA3 --prompt_file /mnt/c/Users/XXX/Desktop/llm_prompt.txt
604248320 blob bytes (46.41%) of bf16
697761792 blob bytes (53.59%) of sfp
normalizer.cc(52) LOG(INFO) precompiled_charsmap is empty. use identity normalization.
Abort at weights.cc:454: Assert offset == range.End():
Metadata
Metadata
Assignees
Labels
No labels