883 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
0
votes
0
answers
45
views
Onnx cannot be read with Microsoft.ML on Windows 10 (19045.5854)
Everything I describe here works perfectly fine on my computer that has Windows 11 (Version 10.0.26200). However, on a computer that has Windows 10 (10.0.19045), it does not work. This is a client's ...
0
votes
0
answers
35
views
Issue with converting mobile_sam.pt to onnx format (decode part)
I want to use the mobile_sam.pt model in web browser, so I need the onnx format of that.
I tried these method, but always getting the same error.
segment-anything -
samexporter
The error above ...
0
votes
0
answers
34
views
ONNX Script rewriter: how to match patterns with multiple outputs?
I am trying to implement this rewrite rule from the TASO paper with ONNX Script rewriter. However, I cannot figure out how to implement a pattern with multiple outputs X and Y.
The ONNX Script does ...
0
votes
2
answers
190
views
Why does TFLite INT8 quantization decompose BatchMatMul (from Einsum) into many FullyConnected layers?
I’m debugging a model conversion using onnx2tf and post-training quantization issue involving Einsum, BatchMatMul, and FullyConnected layers across different model formats.
Pipeline:
ONNX → TF ...
Tooling
1
vote
1
replies
71
views
Is using TRT in Tensorflow 2.18+ possible?
I am aware that TensorFlow has announced that they will no longer support TRT. A while back the announced "Starting with TensorFlow 2.18, support for TensorRT will be dropped. TensorFlow 2.17 ...
1
vote
0
answers
111
views
How to include a .aar file in a Kivy/Buildozer Android project
I would like to add a .aar file (onnxruntime.aar) to my Kivy project. My project structure is:
├── buildozer.spec
├── dress.jpg
├── libs
│ └── onnxruntime.aar
├── log.txt
├── main.py
├── __pycache__
...
3
votes
1
answer
98
views
Matching PyTorch and ONNX outputs layer-wise for debugging inference drift
I want to debug layer-by-layer to see where the ONNX model starts deviating from the PyTorch model outputs.
I can extract intermediate outputs in PyTorch using forward hooks, like:
def get_activation(...
1
vote
1
answer
132
views
Transformer model outputs degrade after ONNX export — what could be causing this?
I’ve exported a fine-tuned BERT-based QA model to ONNX for faster inference, but I’m noticing that the predictions from the ONNX model are consistently less accurate than those from the original ...
0
votes
0
answers
68
views
ONNX Runtime Helsinki-NLP in Java
has anyone managed to translate something using Helsinki-NLP and ONNX Runtime in Java? Using a Python script, I generated these files:
├── encoder_model.onnx
├── decoder_model.onnx
├── ...
0
votes
0
answers
42
views
Required support for Detectron2 to ONNX conversion
Currently I am working on Object Detection model. I have trained a model, want to convert to ONNX format for real time inference and deploy in the field but I am unable to determine how to convert to ...
1
vote
0
answers
158
views
Struggling to run a ONNX model in rust
I'm trying to run a ONNX model, using the rust crate ort
The model I'm trying to use is braai/RMBG-1.4
Below is the code I'm using. Apologies in advance for the variable names
let mut model = ...
2
votes
1
answer
134
views
How to integrate a lightweight image-to-text model into a React Native app?
I am trying to integrate an image-to-text model into a React Native mobile app.
My requirements:
The model should support image + text input → text output.
It should be lightweight enough to run on ...
1
vote
0
answers
177
views
Convert SentencePiece tokenizer to ONNX
I'm developing (Python) an FAQ system based on embeddings to perform similarity queries between a user's question and the FAQ knowledge base. The FAQ needs to run on Android smartphones.
I'm using the ...
0
votes
1
answer
234
views
In old version (v1.18.1) of onnx web runtime, how do I specify custom location for the `.js` files (like `ort-wasm-threaded.js`)?
I have a website that runs an onnx model (pre-optimized to .ort format) on the client via ONNX's web runtime.
I must also stick to an old version of onnx (before 1.19.0), because the support for non-...
0
votes
0
answers
68
views
Export a PyTorch custom hash table lookup OP to ONNX
I have implemented a PyTorch OP which accepts a torch.int64 tensor and outputs another torch.int64 tensor by looking up a hash table with predefined key-value pairs.
The torch part is implemented like ...