So, selecting an item would shift items in the list slightly, this is because the border changes and the items are sized by their border and not content. But we need that or we'll overflow.
So this can be fixed by always having a 1px border and just hiding it.
Only changing the color of the border to something visible when needed.

Example of the border shift:
firefox_wo8tHSTvfp

^{_{Oh no, 1 pixel, absolute horror}}