基于大模型的高质量情感虚拟人系统

1. 一些测试结果

1.1 测试卡通人像

1.2 合成人物测试

1.3 不同表情测试

1.4 不同的声音测试

1.5 不同语言测试

1.6 不同的动作测试

2. 环境准备

2.1 准备EAT环境和GPT-SOVITS环境

git clone https://github.com/lililuya/Graduation-Project.git
cd env

# if use conda, modify the prefix of environment.yml or delete it to use the default location
conda env create -f environment.yml

# if use pip, delete some local package index.
pip install -r requirements.txt

2.2 ModelScope和GPT-SOVISTS的环境问题,以ModelScope的为准

pip install funasr==1.0.22
pip install modelscope==1.13.3

2.3 tensorrt安装

2.4 一些问题

pip install -U numba

3.权重文件

4.运行

4.1 本地运行

python whole_pipeline_GPTSOVITS_asr_en_gradio_multivoice.py

4.2 使用Gradio自带内网穿透

# Modify launch=True

4.3界面

引用文献

@InProceedings{Gan_2023_ICCV,
 author = {Gan, Yuan and Yang, Zongxin and Yue, Xihang and Sun, Lingyun and Yang, Yi},
 title = {Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation},
 booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
 month = {October},
 year = {2023},
 pages = {22634-22645}
}
@InProceedings{wang2021gfpgan,
 author = {Xintao Wang and Yu Li and Honglun Zhang and Ying Shan},
 title = {Towards Real-World Blind Face Restoration with Generative Facial Prior},
 booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
 year = {2021}
}
@inproceedings{gao22b_interspeech,
 author={Zhifu Gao and ShiLiang Zhang and Ian McLoughlin and Zhijie Yan},
 title={Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition},
 year=2022,
 booktitle={Proc. Interspeech 2022},
 pages={2063--2067},
 doi={10.21437/Interspeech.2022-9996}
}
@inproceedings{du2022glm,
 title={GLM: General Language Model Pretraining with Autoregressive Blank Infilling},
 author={Du, Zhengxiao and Qian, Yujie and Liu, Xiao and Ding, Ming and Qiu, Jiezhong and Yang, Zhilin and Tang, Jie},
 booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
 pages={320--335},
 year={2022}
}

相关仓库

目前存在的问题

声明