The best approach for processing a large data on php

Question

I am searching for the best approach for process a large amount data using PHP.

I want to create a xml file from a database, when I execute my code the system will stop with a "PHP timeout" error, I just adjusted php timeout value (not a valid solution).

Now I try to implement it with AJAX, is there any better way to do it easily?

Answer 1

If you are running some sort of processing that will unavoidably take a long time then instead of doing it synchronously within the user's request it is usually better to do the processing asynchronously.

The user request would add the processing request to a job queue, then return immediately with a message that it will be processed shortly. You then have a background process which processes the data then informs the user (i.e. via an email or UI pop-up) that processing has completed, at which point they can then choose to view the results.

• Midhun: Hi, Yes, Job queue is a best option. Thanks for the info

Answer 2

Is the problem the sheer volume of data or that it takes a long time to assemble it?

If it's the former (too much data) then ask yourself why you need to process so much all in one go. If there's a User on the receiving end of [all] this data, you can pretty much guarantee that they will not want to read it all.

If if's the latter (taking too long) then you need to tune the SQL queries that make it up.