Skip Navigation LinksHome > Categories > Code from a Category

Ruby & PHP. Starling and Zend_Queue.

User Name: serfcompany
Name: Serf
Contact Me:
Home Page:
php,mysql,javascript,html,css. Preferable working with Zend Framework. Good know javascript. I worked with various, javascript frameworks such as(jquery, YUI3, extjs, sencha touch). [More]
Viewed Times: 3888
Add Date: 07/18/2012
In this article I will discuss the topical issue for many. How to interconnect applications in different languages. For example, Ruby and PHP.

In the Twitter integration problem with the queue decided to help Starling.
Actually now there is a tendency that, for each task, pick your language.
Homogeneous systems is becoming smaller. In consequence of this there is a need for standards to integrate motley software in a single system.

Statement of the Problem

Suppose we have a situation where, during a HTTP request needs to be done long-term operation. For example, when a user adds a comment to the article, send email notification to the author. In the classical scheme of interaction (Figure 7-1), we have to wait for the completion of this operation to notify the user. It turns out that this operation blocks execution of Web applications, and hence there is a loss of productivity. Yes, and the user has to wait extra time. When the number of users is small - it's no problem, but the projects Highload this question is of great importance. We decided on the challenge - to make all the resource-intensive tasks in the background processes. Consider the case with the addition of hosting and domain web panel. The objective is to execute a script in Ruby, which will manipulate the DNS zones and restart the DNS service.
Now define a scheme of work. Client will turn ZendFramework application that will add tasks to the queue (send messages to the queue). Server (vorkerom) is Ruby, who will receive messages from the queue and process them.
Why choose such a combination? ZF customer historically, but the server in Ruby is chosen, because really do not want to give PHP with root, I think many admins will understand me. The choice of queue has been made in favor of Starling, as it is based on Memcached protocol and uses (well, almost :-)) and implementation in Ruby is very simple.

What is the message queue?

Theme was inspired by the message queue on "division of labor: Organizing a multitasking, distributed system in Zend Framework using the Job Queue" on the last of Alexander Gotgelfa ZFCONF. He worked with the server queues Gearman, and told him a few, but very serious bugs (such as the problem of free memory). In this connection, I started looking at other thoron and stopped at MemcacheQ and Starling, who is advised users Habré. And because Ruby is easier on the implementation of the second, he was chosen.

What is Starling?

Starling - songbird of the family, widespread in large parts of Eurasia, and successfully introduced in South Africa, North America, Australia and New Zealand. In the south and west of Europe leads a sedentary life, and in the northern and eastern parts of it is migratory in winter migrating south. On the surface (size, yellow bill and dark plumage) is slightly reminiscent of blackbirds, but unlike them, walking on the earth, and not jumping. :-)

And so is the message queue written in Ruby. It is used on Twitter! To install the server queues you should already be installed Ruby. Starling himself put very simply. The next line is executed only once!

gem sources-a Put the gem (you can do any number of times). sudo gem install starling

If you put Ruby to all users, then execute instead of sudo rvmsudo Next should start the server and can connect servers and clients.

Now you can connect customers to the ruby and php. Clients will call those who send the job to the queue, and servers - those who receive and serves them (to serve). Now you can connect customers to the ruby and php.

The general format of the data. Ruby marshalizatsiya in PHP

The first thing with which I have encountered is the format for storing messages in the queue. The choice was a few options.

*[b] Marshalizatsiya (marshalling) objects. So does the ruby and the default qualify this way of working Starling,
* [t] Serialization (serializing). So stores objects Zend Framework, namely Zend_Queue,
* [t] JSON. De-facto standard for communication in heterogeneous environments,
* [t] XML. The standard de jure, but has a larger memory consumption for storing messages as tag is the nature,
where [m] - Text Format, [b]-binary format.

I will say so in the beginning, I was tempted to never touch the Starling, and learn to work with in PHP marshalizovannymi Ruby objects. But nothing like I could not find. Use XML does not want to work with JSON had to be nice to rewrite the server queue. It was then that my eyes and got hem php_serialize.

Ruby + Starling

He does exactly what I expected from him, but it does serialize () function of the Ruby objects and vice versa. So the scheme is as follows: php -> serialize -> starling -> unserialize -> ruby. To do this, though still have some things written in the Starling. Here is a simple code vorkera queue.

As you can see, instead of Marshal.load MemCache :: get and use PHP :: unserialize and skipping one level of inheritance (MemCache) in order not to break the usual MemCache client I introduced it to the class of Starling. Otlchino now take for the second part of our system.

PHP + Starling (ZendExtra_Queue_Adapter_Starling)

In Zend Framework already have a good class of queuing - Zend_Queue, but unfortunately it does not support Starling. To create our support for the introduction of an adapter that otnasleduem from the nearest relative (Zend_Queue_Adapter_MemcacheQ).

In the Zend-Q (this is read Zend_Queue) is very well implemented to work with adapters storage. Each of them is very different, and the pattern of "strategy" would not come here, but the authors have found out and added a feature getCapabilities (), which returns an array of opportunities the store.

In our store it is possible to obtain the number of messages in the queue for service, but it must be realized. In the write / rewrite / finished the portion of the adapter functions as a result was born ZendExtra_Queue_Adapter_Starling

Here is an example of initializing the class and work with it: StarlingZF.php

Or the same thing but at the level of PHP: StarlingPHP.php

As you can see it's not that difficult. Now you can even write a utility to monitor (if you do not fit existing) that will monitor the queue.

Advantages and disadvantages of Starling

Let's see what we have, and that it gives us.

*Speed. Data is stored in Memcache, which is renowned for its performance.
* Scalability. The queue can be distributed across multiple servers.
* And standardization the unification. A standard Memcached-protocol.
But there are drawbacks.
* Reliability. Since the All data is stored in memory, then restart the server problems - lost.

But in some cases it may not be critical.
For example, in the case of host mark the completion of the task is placed at the end of it, and if necessary, the client side will be able to repeat the request. In case of sending email-is the outer part Postfix, to put messages to send, and Starling is a buffer queue for the mail server.
In the case of resize pictures, resize to the task can be assigned on subsequent calls to verify the condition kkartinke task.
Here's what they write about it by the developers.

The system is reliable, fast and uses a standard protocol memcached.

On assurances of developers, it is generally the most stable link in Twitter. When other elements are muted, Starling has always continued to work. So it's not so bad, it is important to understand that there are no ideal solutions, and any of them compromise on either side, and to determine with priorities. If the priority - Highload, I think, starling - the perfect solution!

Post a Comment

Name: (Optional)
Email: (Optional, you can get an email if somebody replys your comments)*
Email me if somebody respons my comment below:
Enter Text
as Below:
(case insensitive, if hard to read, click the "get a new one" button)
* Your email address will not be shared with any third parties for any reason.
** Maximum 1000 charactors.