Integrating PHP With RabbitMQ ÁlvaroVidela | The NetCircle Zendcon 2010 Tuesday, November 2, 2010
Who? Tuesday, November 2, 2010
About Me • Development Manager at TheNetCircle.com • Writing "RabbitMQ in Action" for Manning • Blog: http://videlalvaro.github.com/ • Twitter: @old_sound Tuesday, November 2, 2010
Why Do I need RabbitMQ? Tuesday, November 2, 2010
The User Tuesday, November 2, 2010
I don’t want to wait till your app resizes my image! Tuesday, November 2, 2010
The Product Owner Tuesday, November 2, 2010
Can we also notify the user friends when she uploads a new image? Tuesday, November 2, 2010
Can we also notify the user friends when she uploads a new image? I forgot to mention we need it for tomorrow... Tuesday, November 2, 2010
The Sysadmin Tuesday, November 2, 2010
Dumb!You’re delivering full size images! The bandwidth bill has tripled! Tuesday, November 2, 2010
Dumb!You’re delivering full size images! The bandwidth bill has tripled! We need this fixed for yesterday! Tuesday, November 2, 2010
The Developer in the other team Tuesday, November 2, 2010
I need to call your PHP stuff but from Python Tuesday, November 2, 2010
I need to call your PHP stuff but from Python And also Java starting next week Tuesday, November 2, 2010
You Tuesday, November 2, 2010
FML! Tuesday, November 2, 2010
Is there a solution? Tuesday, November 2, 2010
RabbitMQ & AMQP Tuesday, November 2, 2010
AMQP Tuesday, November 2, 2010
AMQP • Advanced Message Queuing Protocol • Suits Interoperability • Completely Open Protocol • Binary Protocol • AMQP Model • AMQP Wire Format Tuesday, November 2, 2010
AMQP Model • Exchanges • Message Queues • Bindings • Rules for binding them Tuesday, November 2, 2010
AMQP Wire Protocol • Functional Layer • Transport Layer Tuesday, November 2, 2010
Message Flow http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html Tuesday, November 2, 2010
Exchange Types • Fanout • Direct • Topic Tuesday, November 2, 2010
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Fanout_Exchange.html Tuesday, November 2, 2010
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Direct_Exchange.html Tuesday, November 2, 2010
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Topic_Exchange.html Tuesday, November 2, 2010
Usage Scenarios Tuesday, November 2, 2010
Usage Scenarios • Batch Processing Tuesday, November 2, 2010
Usage Scenarios • Batch Processing • Image Uploading Tuesday, November 2, 2010
Usage Scenarios • Batch Processing • Image Uploading • Distributed Logging Tuesday, November 2, 2010
Scenario Batch Processing Tuesday, November 2, 2010
Requirements Tuesday, November 2, 2010
Requirements • Generate XML Tuesday, November 2, 2010
Requirements • Generate XML • Distribution Over a Cluster Tuesday, November 2, 2010
Requirements • Generate XML • Distribution Over a Cluster • Elasticity - Add/Remove new workers Tuesday, November 2, 2010
Requirements • Generate XML • Distribution Over a Cluster • Elasticity - Add/Remove new workers • No Code Changes Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $msg = new AMQPMessage($video_info, array('content_type' => 'text/plain', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'video-desc-ex'); $channel->close(); $conn->close(); Tuesday, November 2, 2010
Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $msg = new AMQPMessage($video_info, array('content_type' => 'text/plain', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'video-desc-ex'); $channel->close(); $conn->close(); Tuesday, November 2, 2010
Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $msg = new AMQPMessage($video_info, array('content_type' => 'text/plain', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'video-desc-ex'); $channel->close(); $conn->close(); Tuesday, November 2, 2010
Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $msg = new AMQPMessage($video_info, array('content_type' => 'text/plain', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'video-desc-ex'); $channel->close(); $conn->close(); Tuesday, November 2, 2010
Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $msg = new AMQPMessage($video_info, array('content_type' => 'text/plain', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'video-desc-ex'); $channel->close(); $conn->close(); Tuesday, November 2, 2010
Publisher Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $msg = new AMQPMessage($video_info, array('content_type' => 'text/plain', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'video-desc-ex'); $channel->close(); $conn->close(); Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $conn = new AMQPConnection(HOST, PORT, USER, PASS, VHOST); $channel = $conn->channel(); $channel->exchange_declare('video-desc-ex', 'direct', false, true, false); $channel->queue_declare('video-desc-queue', false, true, false, false); $channel->queue_bind('video-desc-queue', 'video-desc-ex'); $channel->basic_consume('video-desc-queue', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Scenario Upload Pictures Tuesday, November 2, 2010
Requirements Tuesday, November 2, 2010
Requirements • Upload Picture Tuesday, November 2, 2010
Requirements • Upload Picture • Reward User Tuesday, November 2, 2010
Requirements • Upload Picture • Reward User • Notify User Friends Tuesday, November 2, 2010
Requirements • Upload Picture • Reward User • Notify User Friends • Resize Picture Tuesday, November 2, 2010
Requirements • Upload Picture • Reward User • Notify User Friends • Resize Picture • No Code Changes Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $metadata = json_encode(array( 'image_id' => $image_id, 'user_id' => $user_id, ‘image_path' => $image_path)); $msg = new AMQPMessage($metadata, array('content_type' => 'application/json', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'upload-pictures'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $metadata = json_encode(array( 'image_id' => $image_id, 'user_id' => $user_id, ‘image_path' => $image_path)); $msg = new AMQPMessage($metadata, array('content_type' => 'application/json', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'upload-pictures'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $metadata = json_encode(array( 'image_id' => $image_id, 'user_id' => $user_id, ‘image_path' => $image_path)); $msg = new AMQPMessage($metadata, array('content_type' => 'application/json', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'upload-pictures'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $metadata = json_encode(array( 'image_id' => $image_id, 'user_id' => $user_id, ‘image_path' => $image_path)); $msg = new AMQPMessage($metadata, array('content_type' => 'application/json', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'upload-pictures'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $metadata = json_encode(array( 'image_id' => $image_id, 'user_id' => $user_id, ‘image_path' => $image_path)); $msg = new AMQPMessage($metadata, array('content_type' => 'application/json', 'delivery_mode' => 2)); $channel->basic_publish($msg, 'upload-pictures'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $channel->queue_declare('resize-picture', false, true, false, false); $channel->queue_bind('resize-picture', 'upload-pictures'); $channel->basic_consume('resize-picture', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $channel->queue_declare('resize-picture', false, true, false, false); $channel->queue_bind('resize-picture', 'upload-pictures'); $channel->basic_consume('resize-picture', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $channel->queue_declare('resize-picture', false, true, false, false); $channel->queue_bind('resize-picture', 'upload-pictures'); $channel->basic_consume('resize-picture', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $channel->queue_declare('resize-picture', false, true, false, false); $channel->queue_bind('resize-picture', 'upload-pictures'); $channel->basic_consume('resize-picture', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $channel->queue_declare('resize-picture', false, true, false, false); $channel->queue_bind('resize-picture', 'upload-pictures'); $channel->basic_consume('resize-picture', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('upload-pictures', 'fanout', false, true, false); $channel->queue_declare('resize-picture', false, true, false, false); $channel->queue_bind('resize-picture', 'upload-pictures'); $channel->basic_consume('resize-picture', $consumer_tag, false, false, false, false, $consumer); while(count($channel->callbacks)) { $channel->wait(); } Tuesday, November 2, 2010
Consumer Code $consumer = function($msg){ $meta = json_decode($msg->body, true); resize_picture($meta['image_id'], $meta['image_path']); $msg->delivery_info['channel']-> basic_ack($msg->delivery_info['delivery_tag']); }; Tuesday, November 2, 2010
Consumer Code $consumer = function($msg){ $meta = json_decode($msg->body, true); resize_picture($meta['image_id'], $meta['image_path']); $msg->delivery_info['channel']-> basic_ack($msg->delivery_info['delivery_tag']); }; Tuesday, November 2, 2010
Consumer Code $consumer = function($msg){ $meta = json_decode($msg->body, true); resize_picture($meta['image_id'], $meta['image_path']); $msg->delivery_info['channel']-> basic_ack($msg->delivery_info['delivery_tag']); }; Tuesday, November 2, 2010
Consumer Code $consumer = function($msg){ $meta = json_decode($msg->body, true); resize_picture($meta['image_id'], $meta['image_path']); $msg->delivery_info['channel']-> basic_ack($msg->delivery_info['delivery_tag']); }; Tuesday, November 2, 2010
Consumer Code $consumer = function($msg){ $meta = json_decode($msg->body, true); resize_picture($meta['image_id'], $meta['image_path']); $msg->delivery_info['channel']-> basic_ack($msg->delivery_info['delivery_tag']); }; Tuesday, November 2, 2010
Scenario Distributed Logging Tuesday, November 2, 2010
Requirements Tuesday, November 2, 2010
Requirements • Several Web Servers Tuesday, November 2, 2010
Requirements • Several Web Servers • Logic Separated by Module/Action Tuesday, November 2, 2010
Requirements • Several Web Servers • Logic Separated by Module/Action • Several Log Levels: Tuesday, November 2, 2010
Requirements • Several Web Servers • Logic Separated by Module/Action • Several Log Levels: • Info,Warning, Error Tuesday, November 2, 2010
Requirements • Several Web Servers • Logic Separated by Module/Action • Several Log Levels: • Info,Warning, Error • Add/Remove log listeners at will Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Design Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('logs', 'topic', false, true, false); $msg = new AMQPMessage('some log message', array('content_type' => 'text/plain')); $channel->basic_publish($msg, 'logs', server1.user.profile.info'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('logs', 'topic', false, true, false); $msg = new AMQPMessage('some log message', array('content_type' => 'text/plain')); $channel->basic_publish($msg, 'logs', server1.user.profile.info'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('logs', 'topic', false, true, false); $msg = new AMQPMessage('some log message', array('content_type' => 'text/plain')); $channel->basic_publish($msg, 'logs', server1.user.profile.info'); Tuesday, November 2, 2010
Publisher Code $channel->exchange_declare('logs', 'topic', false, true, false); $msg = new AMQPMessage('some log message', array('content_type' => 'text/plain')); $channel->basic_publish($msg, 'logs', server1.user.profile.info'); Tuesday, November 2, 2010
Consumer Code Get messages sent by host: server1 Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('server1-logs', false, true, false, false); $channel->queue_bind('server1-logs', 'logs', 'server1.#'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('server1-logs', false, true, false, false); $channel->queue_bind('server1-logs', 'logs', 'server1.#'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('server1-logs', false, true, false, false); $channel->queue_bind('server1-logs', 'logs', 'server1.#'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('server1-logs', false, true, false, false); $channel->queue_bind('server1-logs', 'logs', 'server1.#'); Tuesday, November 2, 2010
Consumer Code Get all error messages Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('error-logs', false, true, false, false); $channel->queue_bind('error-logs', 'logs', '#.error'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('error-logs', false, true, false, false); $channel->queue_bind('error-logs', 'logs', '#.error'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('error-logs', false, true, false, false); $channel->queue_bind('error-logs', 'logs', '#.error'); Tuesday, November 2, 2010
Consumer Code $channel->exchange_declare('logs', 'topic', false, true, false); $channel->queue_declare('error-logs', false, true, false, false); $channel->queue_bind('error-logs', 'logs', '#.error'); Tuesday, November 2, 2010
PHP Integration Tuesday, November 2, 2010
php-amqplib • http://github.com/tnc/php-amqplib Tuesday, November 2, 2010
php-amqplib • http://github.com/tnc/php-amqplib • Fork of http://code.google.com/p/php- amqplib/ Tuesday, November 2, 2010
php-amqplib • http://github.com/tnc/php-amqplib • Fork of http://code.google.com/p/php- amqplib/ • PHP 5.3 Compatible Tuesday, November 2, 2010
php-amqplib • http://github.com/tnc/php-amqplib • Fork of http://code.google.com/p/php- amqplib/ • PHP 5.3 Compatible • Improved performance Tuesday, November 2, 2010
php-amqplib • http://github.com/tnc/php-amqplib • Fork of http://code.google.com/p/php- amqplib/ • PHP 5.3 Compatible • Improved performance • Used in production one+ year Tuesday, November 2, 2010
php-amqplib $ git clone http://github.com/tnc/php-amqplib.git <?php require_once('./php-amqplib/amqp.inc'); $conn = new AMQPConnection(‘localhost’, 5672, guest, guest); $channel = $conn->channel(); ?> Tuesday, November 2, 2010
Why RabbitMQ? Tuesday, November 2, 2010
RabbitMQ • Enterprise Messaging System • Open Source MPL • Written in Erlang/OTP • Commercial Support Tuesday, November 2, 2010
Features • Reliable and High Scalable • Easy To install • Easy To Cluster • Runs on:Windows, Solaris, Linux, OSX • AMQP 0.8 - 0.9.1 Tuesday, November 2, 2010
Client Libraries • Java • .NET/C# • Erlang • Ruby, Python, PHP, Perl,AS3, Lisp, Scala, Clojure, Haskell Tuesday, November 2, 2010
Docs/Support • http://www.rabbitmq.com/documentation.html • http://dev.rabbitmq.com/wiki/ • #rabbitmq at irc.freenode.net • http://www.rabbitmq.com/email-archive.html Tuesday, November 2, 2010
One Setup for HA Tuesday, November 2, 2010
Conclusion Tuesday, November 2, 2010
Conclusion • Flexibility Tuesday, November 2, 2010
Conclusion • Flexibility • Scalability Tuesday, November 2, 2010
Conclusion • Flexibility • Scalability • Interoperability Tuesday, November 2, 2010
Conclusion • Flexibility • Scalability • Interoperability • Reduce Ops Tuesday, November 2, 2010
Questions? Tuesday, November 2, 2010
Thanks! Álvaro Videla http://twitter.com/old_sound http://github.com/videlalvaro http://github.com/tnc http://www.slideshare.net/old_sound Tuesday, November 2, 2010