Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Releases: codefuse-ai/MFTCoder

MFTCoder v0.4.3: Bugfix

11 Jun 02:22
@chencyudel chencyudel
cc55b06
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

Bugfix: Remove default tensor board writer which may cause permission problem

P.S. If you have problem like "permission denied" of "/home/admin", please try the new fixed release v0.4.3

Assets 2
Loading
13inccc reacted with rocket emoji
1 person reacted

MFTCoder v0.4.2: Support more open source models; Support QLoRA + Deepspeed ZeRO3 / FSDP

04 Jun 04:06
@chencyudel chencyudel
d0b8457
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

Support more open source models like Qwen2, Qwen2-moe, Starcoder2, etc.
Support QLoRA + Deepspeed ZeRO3 / FSDP, which is efficient for very large models.

Loading

MFTCoder v0.3.0: Support more open source models, support Self-Paced Loss, support FSDP

19 Jan 11:16
@chencyudel chencyudel
e5243da
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

Updates:

  1. Mainly for MFTCoder-accelerate.
  2. It now supports more open source models like Mistral, Mixtral(MoE), DeepSeek-coder, chatglm3.
  3. It supports FSDP as an option.
  4. It also supports Self-paced Loss as a solution for convergence balance in Multitask Fine-tuning.
Loading

v0.1.0 release: Multi Task Fintuning Framework for Multiple base modles

27 Dec 08:20
@chencyudel chencyudel
7946e4f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Choose a tag to compare

  1. We released MFTCoder which supports finetuning Code Llama, Llama, Llama2, StarCoder, ChatGLM2, CodeGeeX2, Qwen, and GPT-NeoX models with LoRA/QLoRA.
  2. mft_peft_hf is based on the HuggingFace Accelerate and deepspeed framework.
    mft_atorch is based on the ATorch frameworks, which is a fast distributed training framework of LLM.
Loading

AltStyle によって変換されたページ (->オリジナル) /