@pamelafox this is ready and working but I wanna experiment with their approach here
https://github.com/microsoft/ai-chat-protocol/blob/main/samples/frontend/js/react/src/Chat.tsx
instead of setTimeout.
If I have time today I will look into it.

I separated the functions to run and run_stream as I can't yield and return something from the same function.
I had to do lots of refactoring to not repeat the code in the two functions.

@john0isaac I believe we need setTimeout for the desired effect. Without setTimeout, React flushes too much at once. I've recommended to Gerardo (the author of that) to show setTimeout for streaming. But let me know if you see otherwise, that was my experience.

@john0isaac I believe we need setTimeout for the desired effect. Without setTimeout, React flushes too much at once. I've recommended to Gerardo (the author of that) to show setTimeout for streaming. But let me know if you see otherwise, that was my experience.

It works without it but it in the frontend it looks like it's dumping the text bulk by bulk not streaming it.
I think the delay in updating the answers makes it better, readable, and it's smoother.

I won't change it. setTimeout is technically not needed but I will leave it for aesthetics.

Done, what's left is if you have any suggestions for the SQLAlchmey error, as I said it's fixed but I'm open to alternative approaches.

@john0isaac I did some digging and found this thread:
fastapi/fastapi#11321

I refactored so that there's a "prepare_context" method as well as a "answer"/"answer_stream", so we call "prepare_context" first, and then pass answer_stream to StreamingResponse.
I think that ends up being a pretty nice interface, actually.

LGTM