🚀 The feature, motivation and pitch

IBM is at the forefront of the most efficient LLMs on edge devices, for example Granite-4.0-H-Micro.

Could Granite models be supported for responsible, high throughput and quality with extremely low memory consumption?

Alternatives

Qwen3-Next achieves many similar breakthroughs, but no small model yet.
https://qwen3-next.com/

Additional context

https://www.storagereview.com/news/ibm-granite-4-0-efficient-open-weight-llms-target-enterprise-grade-performance-cost-and-governance

RFC (Optional)

No response