create-react-app has a really neat custom error overlay. Not sure if it is usable outside of CRA - I actually don't mind using it as-is if it's possible, but we likely have to port it (already have a placeholder package here).

• Reports both webpack build errors & runtime errors
• Stack frames for runtime errors
• Jump to source files (we already have launch-editor middleware in the dev server)