9 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
0
votes
0
answers
28
views
Question on the loss quality of diffusion model(RaMVID), base model of journal:Diffusion Models For Video Prediction and Infilling
I trained a model on a chlorophyll dataset, one channel for its value. When I start to train it, for the first hundred steps, the loss of forward backward drops quickly from 0.5 to 0.46.
Since then, ...
0
votes
0
answers
71
views
What could cause model trainings to have big random loss spikes?
I am currently training a DeepLabv3 variant with EfficientNet as a backbone for semantic segmentation, on tf 2.16.1 with an RTX 4090. On some training instances, at a single random epoch (never the ...
1
vote
0
answers
48
views
Addressing Saddle Points in Keras Model Training
My keras model seems to have to hit a saddle point in it's training. Of course this is just an assumption; I'm not really sure. In any case, the loss stops at .0025 and nothing I have tried has worked ...
0
votes
2
answers
3k
views
What happens if optimal training loss is too high
I am training a Transformer. In many of my setups I obtain validation and training loss that look like this:
Then, I understand that I should stop training at around epoch 1. But then the training ...
1
vote
0
answers
115
views
How to understand a periodicity in the training loss using a pre-trained model of PyTorch?
I'm using a pre-trained model from Pytorch ( Resnet 18,34,50) in order to classify images. During the training, a weird periodicity appears in the training as you can see in the image below. Did ...
-1
votes
1
answer
1k
views
training loss during LSTM training is higher than validation loss
I am training an LSTM to predict a time series. I have tried an encoder-decoder, without any dropout. I divided my data n 70% training and 30% validation. The total points in the training set and ...
1
vote
1
answer
59
views
Implemented model network but neither training error nor val error decreasing
Since I'm novice to Pytorch, this question might be a very trivial one, but I'd like to ask for your help about how to solve this one.
I've implemented one network from a paper and used all hyper ...
0
votes
0
answers
2k
views
Validation loss is constant and training loss decreasing
I have a model training and I got this plot. It is over audio (about 70K of around 5-10s) and no augmentation is being done. I have tried the following to avoid overfitting:
Reduce complexity of the ...
1
vote
0
answers
233
views
About model comparison in terms of training/validation loss
I'm comparing two models, and want to clarify the weird results.
Model 1 achieves lower training loss than model 2, but get higher validation loss.
Because over-fitting and under-fitting are ...