tfm.nlp.encoders.XLNetEncoderConfig
Stay organized with collections
Save and categorize content based on your preferences.
XLNet encoder configuration.
Inherits From: Config, ParamsDict
tfm.nlp.encoders.XLNetEncoderConfig(
default_params: dataclasses.InitVar[Optional[Mapping[str, Any]]] = None,
restrictions: dataclasses.InitVar[Optional[List[str]]] = None,
vocab_size: int = 32000,
num_layers: int = 24,
hidden_size: int = 1024,
num_attention_heads: int = 16,
head_size: int = 64,
inner_size: int = 4096,
inner_activation: str = 'gelu',
dropout_rate: float = 0.1,
attention_dropout_rate: float = 0.1,
attention_type: str = 'bi',
bi_data: bool = False,
tie_attention_biases: bool = False,
memory_length: int = 0,
same_length: bool = False,
clamp_length: int = -1,
reuse_length: int = 0,
use_cls_mask: bool = False,
embedding_width: int = 1024,
initializer_range: float = 0.02,
two_stream: bool = False
)
Attributes | |
|---|---|
BUILDER
|
|
default_params
|
Dataclass field |
restrictions
|
Dataclass field |
vocab_size
|
Dataclass field |
num_layers
|
Dataclass field |
hidden_size
|
Dataclass field |
num_attention_heads
|
Dataclass field |
head_size
|
Dataclass field |
inner_size
|
Dataclass field |
inner_activation
|
Dataclass field |
dropout_rate
|
Dataclass field |
attention_dropout_rate
|
Dataclass field |
attention_type
|
Dataclass field |
bi_data
|
Dataclass field |
tie_attention_biases
|
Dataclass field |
memory_length
|
Dataclass field |
same_length
|
Dataclass field |
clamp_length
|
Dataclass field |
reuse_length
|
Dataclass field |
use_cls_mask
|
Dataclass field |
embedding_width
|
Dataclass field |
initializer_range
|
Dataclass field |
two_stream
|
Dataclass field |
Methods
as_dict
as_dict()
Returns a dict representation of params_dict.ParamsDict.
For the nested params_dict.ParamsDict, a nested dict will be returned.
from_args
@classmethodfrom_args( *args, **kwargs )
Builds a config from the given list of arguments.
from_json
@classmethodfrom_json( file_path: str )
Wrapper for from_yaml.
from_yaml
@classmethodfrom_yaml( file_path: str )
get
get(
key, value=None
)
Accesses through built-in dictionary get method.
lock
lock()
Makes the ParamsDict immutable.
override
override(
override_params, is_strict=True
)
Override the ParamsDict with a set of given params.
| Args | |
|---|---|
override_params
|
a dict or a ParamsDict specifying the parameters to be overridden. |
is_strict
|
a boolean specifying whether override is strict or not. If
True, keys in override_params must be present in the ParamsDict. If
False, keys in override_params can be different from what is currently
defined in the ParamsDict. In this case, the ParamsDict will be extended
to include the new keys.
|
replace
replace(
**kwargs
)
Overrides/returns a unlocked copy with the current config unchanged.
validate
validate()
Validate the parameters consistency based on the restrictions.
This method validates the internal consistency using the pre-defined list of restrictions. A restriction is defined as a string which specifies a binary operation. The supported binary operations are {'==', '!=', '<', '<=', '>', '>='}. Note that the meaning of these operators are consistent with the underlying Python immplementation. Users should make sure the define restrictions on their type make sense.
For example, for a ParamsDict like the following
a:
a1: 1
a2: 2
b:
bb:
bb1: 10
bb2: 20
ccc:
a1: 1
a3: 3
one can define two restrictions like this ['a.a1 == b.ccc.a1', 'a.a2 <= b.bb.bb2']
| What it enforces are | |
|---|---|
|
| Raises | |
|---|---|
KeyError
|
if any of the following happens (1) any of parameters in any of restrictions is not defined in ParamsDict, (2) any inconsistency violating the restriction is found. |
ValueError
|
if the restriction defined in the string is not supported. |
__contains__
__contains__(
key
)
Implements the membership test operator.
__eq__
__eq__(
other
)
Class Variables | |
|---|---|
| IMMUTABLE_TYPES |
(<class 'str'>,
<class 'int'>,
<class 'float'>,
<class 'bool'>,
<class 'NoneType'>)
|
| RESERVED_ATTR |
['_locked', '_restrictions']
|
| SEQUENCE_TYPES |
(<class 'list'>, <class 'tuple'>)
|
| attention_dropout_rate |
0.1
|
| attention_type |
'bi'
|
| bi_data |
False
|
| clamp_length |
-1
|
| default_params |
None
|
| dropout_rate |
0.1
|
| embedding_width |
1024
|
| head_size |
64
|
| hidden_size |
1024
|
| initializer_range |
0.02
|
| inner_activation |
'gelu'
|
| inner_size |
4096
|
| memory_length |
0
|
| num_attention_heads |
16
|
| num_layers |
24
|
| restrictions |
None
|
| reuse_length |
0
|
| same_length |
False
|
| tie_attention_biases |
False
|
| two_stream |
False
|
| use_cls_mask |
False
|
| vocab_size |
32000
|