Currently, there's a free_params_immediately option, which is quite useful when we're not going to use the model any time soon. However, it would be nice to be able to leave the weights that won't be used in RAM, for fast reading, until we need them again. This way, the total VRAM usage when generating the VAE (or anything else) is reduced, eliminating the need to reload the entire model from the hard drive.