0

In laravel, I have a query that searches thousands of rows in a database, and I trying to compile that into a CSV file. In attempt to reduce memory usage, I get 500 rows at a time and output the CSV.

 $callback = function () use ($query) {
 $file = fopen('php://output', 'w');
 $query->chunk(500, function ($rows) use ($file) {
 foreach ($rows as $key => $row) {
 fputcsv($file, array_map(...$rows...));
 }
 log_info("Memory used " . memory_get_usage());
 });
 fclose($file);
 };
 $headers = [ ... ];
 return response()->stream($callback, 200, $headers);

The actual query is a bit more complex, and involves getting related models which also need to be rehydrated. When I run this, it begins generating the CSV file and, after a while, runs out of memory. This is in my log

(59): Memory used 17208328;
(59): Memory used 25105328;
(59): Memory used 30601328;
...
(59): Memory used 127380496;
(59): Memory used 129352584;
(59): Memory used 131207672;
[2025年11月23日 23:50:15] qa.ERROR: Allowed memory size of 134217728 bytes exhausted (tried to allocate 16384 bytes) 

What I Tried

I tried putting the following inside the chunk loop, hoping that it would free memory. It had no effect on the memory consumption.

 unset($rows);
 flush();
 gc_collect_cycles();
Shadow
34.5k10 gold badges67 silver badges76 bronze badges
asked 10 hours ago
3
  • Since you are writing to the output it's possible you have buffered output on. Try running ob_end_clean() before you start streaming the response to see if that makes a difference Commented 10 hours ago
  • I added ob_end_clean() but no luck. I would think that flush() would release that memory Commented 8 hours ago
  • How much rows are in the CSV? Since you are writing to php://output, this writes to the output buffer (keyword being buffer), so if the output exceeds 128MB, it will trip the memory limit (but 128MB is a lot of text). Commented 2 hours ago

0

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.