tfm.optimization.PowerDecayWithOffset
Stay organized with collections
Save and categorize content based on your preferences.
Power learning rate decay with offset.
View aliases
Main aliases
tfm.optimization.PowerDecayWithOffset(
initial_learning_rate: float,
power: float = 1.0,
offset: int = 0,
pre_offset_learning_rate: float = 1000000.0,
name: str = 'PowerDecayWithOffset'
)
Learning rate equals to pre_offset_learning_rate if step < offset.
Otherwise, learning rate equals to lr * (step - offset)^power.
Args | |
|---|---|
initial_learning_rate
|
The initial learning rate. |
power
|
The order of the polynomial. |
offset
|
The offset when computing the power decay. |
pre_offset_learning_rate
|
The maximum learning rate we'll use. |
name
|
Optional, name of learning rate schedule. |
Methods
from_config
@classmethodfrom_config( config )
Instantiates a LearningRateSchedule from its config.
| Args | |
|---|---|
config
|
Output of get_config().
|
| Returns | |
|---|---|
A LearningRateSchedule instance.
|
get_config
get_config()
Get the configuration of the learning rate schedule.
__call__
__call__(
step
)
Call self as a function.