-
Notifications
You must be signed in to change notification settings - Fork 3.1k
-
1. LR Scheduler 升级更新
Huggingface的 https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_utils.py
class SchedulerType(Enum): LINEAR = "linear" COSINE = "cosine" COSINE_WITH_RESTARTS = "cosine_with_restarts" POLYNOMIAL = "polynomial" CONSTANT = "constant" CONSTANT_WITH_WARMUP = "constant_with_warmup"
而当前的PaddleNLP的Trainer中只支持下面几个,建议更新。
class SchedulerType(ExplicitEnum): LINEAR = "linear" COSINE = "cosine" CONSTANT = "constant" CONSTANT_WITH_WARMUP = "constant_with_warmup"
2. 统一使用Trainer里面的LR Scheduler(与HF对齐)
https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/transformers/optimization.py 有旧的LR Scheduler,建议废弃,未来的新代码需要升级成最新的 Trainer 里面的 Scheduler,这样在使用体验上对习惯HF的用户来说更友好。
image
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 1 comment
-
这两个应该需要补上。
COSINE_WITH_RESTARTS = "cosine_with_restarts" POLYNOMIAL = "polynomial"
这两个可以考虑合并到 Trainer 里面一起支持
"CosineAnnealingWithWarmupDecay", "LinearAnnealingWithWarmupDecay",
Beta Was this translation helpful? Give feedback.
All reactions
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment