-
Notifications
You must be signed in to change notification settings - Fork 111
Train a 2D VAE with dual channel images #422
Hi, i have a question about the visual autoencoder, what if the input images have 2 channels or even 3.
I get a tensor shape mismatch error in the the following line :
p_loss = perceptual_loss(reconstruction.float(), images.float())
- The Shape of the reconstruction.float() : torch.Size([1, 2, 256, 256])
- The shape of images.float() : torch.Size([1, 2, 256, 256])
here are the parameters i used for the AutoencoderKL :
autoencoderkl = AutoencoderKL(
spatial_dims=2,
in_channels=2,
out_channels=2,
num_channels=(128, 128, 256),
latent_channels=3, num_res_blocks=2,
attention_levels=(False, False, False),
with_encoder_nonlocal_attn=False,
with_decoder_nonlocal_attn=False
)
discriminator = PatchDiscriminator(
spatial_dims=2,
num_layers_d=3,
num_channels=64,
in_channels=2,
out_channels=2
)
Thank very much in advance !!
here is the error log : Error_log.pdf
All reactions
Replies: 1 comment 6 replies
Hi,
Can you verify you don't get this error for images/reconstructions with a single channel?
All reactions
Ah yes - the percepual loss expects grayscale ( 1 channel) or RGB (3 channels). It won't deal with 2 channels. What do the two channels represent? If they are seperate grayscale images you could split them out and run the perceptual loss on each channel separately.
All reactions
Thank you very much for your help! Each channel represents the same image at different wave lengths. Each highlights different features and structures in the scanned body part. Would actually creating a third channel, maybe a duplicate of one of the channels also solve the problem ?
All reactions
Yes that could work, too
All reactions
-
👍 1
Hi mark, thank you very much for your help, I have another question. do channels have an impact on the training and the output of the other channels or are each channel handled separately in the training process?
All reactions
The input channels will interact during training, as convolutions work across channels.