I will just freeze the weights in the PipNet model itself, but I'd like to get gradients (to adjust aspects of the input image) from a loss created by the Landmarks vs some target Landmarks.

It looks like the code 'flows through' from input image to Landmark predictions - does this sound right? Or is there some strange discretisation step that would make the gradients nonsense?