A passionate AI Infra researcher focused on high-performance computing and model-and-system co-design.

Core developer of:

BitNet — official inference framework for 1-bit LLMs.

• 6.25x faster than full-precision and 2.32x faster than low-bit baselines
• lossless inference for BitNet b1.58 via the TL / I2_S ternary mpGEMM kernels
• measured on edge CPUs: Intel i7-13700H, Apple M2 Ultra

BitNet inference timeline

TeraMoE — cross-node expert-parallel MoE training library.

• 1.30x speedup over DeepEP + TE in communication-bound sparse-expert regimes, 1.24x under 3.0x expert load imbalance
• 28% less activation memory than Megatron
• one cooperative persistent kernel overlapping dispatch, expert compute and combine
• measured on SM100 GPUs, EP16–64 over RDMA

TeraMoE SM role timeline

• Bitnet.cpp: Efficient Edge Inference for Ternary LLMs — ACL 2025 Main, first author
• 1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs — first author

llama.cpp · Paddle · TensorRT-LLM

trophy