Hello,

I have a question regarding the autoformer-tiny model mentioned in the README.md of Autoformer.

image

When I downloaded the file, it was named supernet-tiny.pth, leading me to believe that it is a supernet trained with the following configurations: head_num: 4, layer_num: 14, and embed_dim: 240(256). However, after examining the weight matrix of the file, it doesn't seem to match these specifications.

Could you please clarify if the autoformer-tiny is indeed a supernet? If not, can you provide more details about the specific structure options used to train this model?

https://github.com/microsoft/Cream/blob/main/AutoFormer/experiments/subnet/AutoFormer-T.yaml
image
Or, is it a subnet sampled from the supernet with the above configuration?

Thank you for your assistance.