Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Symfony. Async event dispatching #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
makasim merged 19 commits into master from async-events
May 20, 2017
Merged

Symfony. Async event dispatching #86

makasim merged 19 commits into master from async-events
May 20, 2017

Conversation

Copy link
Member

@makasim makasim commented May 16, 2017
edited
Loading

Install the bundle and then enable async_events (If you do not enable it, events will be processed as usual).

# app/config/config.yml
enqueue:
 async_events: true
# if you'd like to send onTerminate use spool_producer (it makes response time lesser) :
enqueue:
 async_events: 
 enabled: true
 spool_producer: true

To add async listener you have to add async: true attribute to the tag, like this:

# app/config/config.yml
app.test_async_listener:
 class: 'AppBundle\Listener\TestAsyncListener'
 tags:
 - { name: 'kernel.event_listener', async: true, event: 'test_async', method: 'onEvent' }

and of course you must have a consumer running:

$ ./bin/console enqueue:consume --setup-broker -vvv 

The listener is not executed in the same process but instead, a message is sent to MQ and dispatched in a consumer.

TODO:

  • (削除) Support event_subscribers (削除ここまで) Symfony async events. Support event subscribers. #94
  • Add default transformers (php serializer)
  • Send a transformer that was used to convert event to message. So it could be used on consumer side.
  • Docs
  • Tests
  • (削除) Add a asyncTransformer attribute support to event_listener tag. (削除ここまで) That was not implmeneted, isntead there is ability to set eventName on transformer tag as regexp.
  • Send messages onTerminate.

Koc reacted with hooray emoji
Copy link
Contributor

@makasim this looks really good, actually! Thanks for the heads up.

Copy link
Contributor

I'm looking if I could reuse mechanics from this for my Doctrine event converter: https://gist.github.com/dkarlovi/4a3c9f0ec01d98f8d75bce46d0df38f6 (new version).

BTW this is my Symfony user token thing: https://gist.github.com/dkarlovi/4971003ab1d42ec2871c8bd33589f007

Copy link
Member Author

makasim commented May 17, 2017

BTW this is my Symfony user token thing

Looks good. You can send a PR with such an extension If you willing to contribute.

I'll ask you for tests and a few small corrections.

Copy link
Contributor

Sure, I'll try to get it for the weekend, thanks.

Copy link
Member Author

makasim commented May 17, 2017
edited
Loading

Koc and dzubchik reacted with heart emoji

<tr>
<td>{{ loop.index }}</td>
<td>{{ sentMessage.topic }}</td>
<td><pre>{{ collector.prettyPrintMessage(sentMessage.body)|raw }}</pre></td>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixes #90

Copy link
Contributor

@makasim do I understand the code correctly: you're pushing the events at the end of the request?

I've just stumbled upon a race condition with my code where in 5% of the time my consumer gets the event before the producer does a Doctrine flush() (and can't find the entity to process). The order at which this is dispatched (regarding other listeners) is important here, should take into account, it should be very late in the process, probably almost the last thing.

Copy link
Member Author

makasim commented May 17, 2017
edited
Loading

@dkarlovi The entity must already be in the database If you do post a message on postXXX Doctrine's event.

Copy link
Contributor

(this isn't related to this PR directly, sorry)

I'm using ApiPlatform to build an API app. They ship with a listener which does a flush() for you. Doctrine will trigger the event listeners when you do a persist(), NOT when you do a flush() (which makes sense, I guess). Problem is, my handlers get invoked before flush() which I assumed cannot happen because lazy = true in Enqueue config (my assumption was lazy=true does the same thing Swiftmailer does).

Timeline:

  1. Doctrine::persist()
  2. Doctrine::postPersist() listeners (which, in my case, dispatches a Enqueue message)
  3. Doctrine::flush() (which actually stores to database)

In my case, 2) & 3) are competing which will be done first and I'm consistently getting a failure (which means, RabbitMQ "won").

Might be a good idea to create an in-memory message spool and flush it at request end, much like Swiftmailer Bundle does?

Copy link
Member Author

makasim commented May 17, 2017

@dkarlovi shouldn't you subscribe on the right event which is triggered after the models are persisted to db? Like postFlush event.

postFlush is called at the end of EntityManager#flush(). EntityManager#flush() can NOT be called safely inside its listeners.

Copy link
Contributor

@makasim you might be right, will look into it, if it provides the same information postXXX() listeners get, I don't see why not.

Missed the onTerminate item on the TODO, awesome. 👍 You're really building great piece of tech here, thanks a lot for your effort.

Copy link
Contributor

@makasim will I be able to use this onTerminate message dispatcher when just sending messages regularly, without having an event listener?

Copy link
Member Author

makasim commented May 19, 2017

@makasim will I be able to use this onTerminate message dispatcher when just sending messages regularly, without having an event listener?

Sure, use enqueue.spool_producer and you are done.

dkarlovi reacted with heart emoji

@makasim makasim merged commit ef84aa2 into master May 20, 2017
@makasim makasim deleted the async-events branch May 20, 2017 20:05
Copy link
Member Author

makasim commented Sep 25, 2017

ASKozienko pushed a commit that referenced this pull request Nov 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants

AltStyle によって変換されたページ (->オリジナル) /