Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Change ordering LIFO to FIFO on delayed jobs #416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mariocamaraneto wants to merge 1 commit into yiisoft:master
base: master
Choose a base branch
Loading
from mariocamaraneto:lifo-to-fifo-on-dayled

Conversation

@mariocamaraneto
Copy link

@mariocamaraneto mariocamaraneto commented Apr 30, 2021
edited
Loading

Push in left and pop from right

Without this we can have Live Lock because some Jobs added first never wil be attendend if new jobs won't stop arriving

Q A
Is bugfix? yes
New feature? no
Breaks BC? no
Tests pass? yes

Push in left and pop from right
@mariocamaraneto mariocamaraneto changed the title (削除) Change ordering LIFO to FIFO. (削除ここまで) (追記) Change ordering LIFO to FIFO on delayed worekr (追記ここまで) Apr 30, 2021
@mariocamaraneto mariocamaraneto changed the title (削除) Change ordering LIFO to FIFO on delayed worekr (削除ここまで) (追記) Change ordering LIFO to FIFO on delayed worker (追記ここまで) Apr 30, 2021
@mariocamaraneto mariocamaraneto changed the title (削除) Change ordering LIFO to FIFO on delayed worker (削除ここまで) (追記) Change ordering LIFO to FIFO on delayed jobs (追記ここまで) Apr 30, 2021
Copy link

rob006 commented May 1, 2021
edited
Loading

Without this we can have Live Lock because some Jobs added first never wil be attendend if new jobs won't stop arriving

This is not true. Expired/delayed job will be executed after execution of all new jobs that were pushed in the meantime. So while order of execution may not match order of push (it was never a guarantee in queue), this job will eventually be executed.

Copy link
Author

The delayed job added to the queue can change the order that they were pushed. Eventually the can be executed, but this may delay ther execution significantly.

In servers with a high amount of delayed jobs being added to the queue, the jobs can be delayed for hours. I saw this happening in our servers, where new jobs were being processed almost immedietly, while old jobs were queued for hours.

I attached a file with a simulation, where in the queue with delayed jobs we can see that job p3 can be stuck in waiting "forever" if new delayed jobs keeps being pushed. Like in "live lock".

In queues without delay, all new jobs are pushed to the left and reserved in right. This is exactly I coded in this PR.

Queue simulator with delay.pdf

rob006 reacted with thumbs up emoji

Copy link
Author

I made a test to simulate this situation, but I don't like the solution because use many SLEEP and decrease the decreases the speed of tests.
You can see here: https://github.com/mariocamaraneto/yii2-queue/pull/1/files

@samdark samdark added this to the 2.3.1 milestone May 4, 2021
Copy link
Member

samdark commented May 4, 2021

A test is necessary. Also please add a line for CHANGELOG.

@samdark samdark modified the milestones: 2.3.1, 2.3.2 May 5, 2021
@samdark samdark removed this from the 2.3.2 milestone Oct 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /