(简体中文|English)
paraformer is a chinese asr from funasr. this project provided a port of paraformer model with ggml
- Define model structure
- Initialize model
- Load model parameters and vocabulary
- fbank + lfr + cmvn feature extraction
- Build calculation forward graph
- encoder
- predictor
- decoder
- bias encoder
- seaco decoder
you need download model from modelscope and convert weight with script
git clone https://github.com/lovemefan/paraformer.cpp cd git paraformer.cpp git submodule sync && git submodule update --init --recursive mkdir build && cd build cmake ../src/csrc && make -j 8 # download model weight from modelscope git clone https://www.modelscope.cn/damo/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch.git resource/model # convert weight and vocab into gguf format python src/python/convert-pt-to-gguf.py \ --model /Users/cenglingfan/Code/cpp-project/paraformer.cpp/resource/model/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch \ --outfile /Users/cenglingfan/Code/cpp-project/paraformer.cpp/resource/model/seaco-paraformer-ggml-model-fp16.bin
-
we borrowed and imitated most c++ code from whisper.cpp
-
we referenced model structure and forward detail from FunASR
-
we borrowed fbank feature extract algorithm code from kaldi-native-fbank and lrf + cmvn algorithm code fromfunasr