An issue I encounter often with ncdu, is that I'm deep in some directory tree, deleting a huge file. Then, when I go back up, I suddenly find myself at or near the bottom of the list, so I have to then scroll all the way back up to investigate the next biggest subdirectory.
What I suggest is to maintain a stack of directory listings as the user traverses the tree, and instead of recalculating where each item should be when going upwards again, to just pop off this stack. This means my cursor doesn't jump around wildly when I go around the tree deleting files, but if I go all the way back and then down into the tree again, I'll see the new ordering.