File Synchronization Between Several Application Servers in the Cluster

As traffic of your project increases you need to improve your app performance to keep up with it. A solution to this is to migrate your application on multiple web servers. But it is often difficult to run such applications as WordPress, Drupal, Joomla, Liferay, Redmine and others that are similar, across multiple web servers. The main problem is that these applications by default save all uploaded files only to one server and do not synchronize these files between server nodes in the cluster. In other words, only the server which was processing the file upload request will contain newly added content. The rest of the servers will fall out-of-sync.

Therefore, here is an instruction solving the issue of keeping uploaded files in sync or, in other words, providing solution for the file synchronization problem.

For now this is just a workaround which helps to automate content synchronization on multiple app servers. In the future, our team will integrate it into the core system.

The current solution is a combination of bookmarklet and server side script. Basically, server side script is used with a help of such a utility as lsyncd together with cron.

Lsyncd is a light-weight, live mirror solution used to synchronize app servers. Being wisely coupled with inotify, lsyncd initiates file sync only if it has detected any actual changes in the system. In such a way, the load on CPU is decreased and you do not burn up many resources on processing the synchronization.

With a help of our bookmarklet, lsyncd is automatically installed on each app server available in the environment. Before its installation, you will be asked to choose the folders you want to synchronize. As a result, when a change is made in the chosen folders on any of the app servers, those changes will be synced to each server node automatically.

Scheme


So now let’s see how to turn this into reality, step-by-step.

In our example we will use the WordPress application deployed to the environment with two servers. This instruction is also fully appropriate for other PHP, Java or Ruby applications such as Drupal, Joomla, Liferay, Redmine and others.

Install Application

1. You can install your WordPress application manually or using our JPS widget for one click installation. The instruction and the widget can be found in the WordPress document.

2. Now let’s create a cluster, to increase the number of app servers.

wordpress cluster


By using instructions in our documentation, you can easily add extra app servers, enable high availability or even configure a clustered solution:


In our case, we have simply added one more app server to the environment. And, you can use a more complex scenario by setting the clustered solution with your WordPress application.

3. Click Open in browser for your environment.

open wordpress


4. Complete the
WordPress installation by providing the required information.

wordpress installation

Test File Upload Without Synchronization

Let’s check the file upload processing by the servers, without synchronization enabled.

1. Go to the admin panel and make your custom changes: upload images or files, edit the themes etc. In our case we uploaded one image to the Media Library.

upload image wordpress


2. Return to the Jelastic dashboard and click
Config next to your server node.

configure nginxphp


3. Navigate to the
webroot > ROOT > wp-content folder.

At one of the app servers you will see a newly added uploads folder that contains your newly added files.

wp-content folder
Note that the folder with uploaded files depends on the app that you are running.
Here is the list of such folders for some widely used applications:
WordPress: webroot/ROOT/wp-content
Drupal: webroot/ROOT/sites
Joomla: webroot/ROOT/images
Liferay: webroot/home/liferay/data

4. If you navigate to the same folder (wp-content, in our case) of the second node you’ll see that there is no uploads directory there at all.

node configuration


As a result, you will see that the added image is located only at the server that processed the file upload.

Apply File Synchronization

Now let’s proceed to solving the problem of file synchronization in your cluster.

1. Firstly, drag this bookmarklet to your browser bookmarks bar:


file-sync

2. Navigate to your Jelastic dashboard and initiate the script by clicking on File Synchronization in your bookmarks bar.

synchronization bookmarklet


3. In the opened window, choose the environment with your application and
check the folder (or several folders) you want to sync. Click Install.

sync folders

Note: do not select the whole webroot folder because this will disable the synchronization

4. Wait a short time (about 2 mins) for the settings to be applied. Initially, you will see that on both server nodes lsyncd folder, install.sh and replication.tar files appear in the webroot directory.

nodes sychronization


5. In a moment, check the
webroot > ROOT > wp-content folder on all app servers.

As you can see, the previously uploaded image is located in the uploads folder on each node.

uploads folder


It’s that easy. As a result, the synchronization will be automatically performed every time you add or change anything in your WordPress (or any other application).

Change Topology, Keeping Synchronization

If you change the environment topology by adding more app servers, you need to initiate the script again in order to keep the synchronization process on all nodes.

1. Click Change environment topology for the environment with your application.

change topology


2. Add extra servers and
Apply the changes.

add nginx nodes


3. Perform 2-4 steps of
Apply File Synchronization block in order to run internal process for setting synchronization.

Note: After additional running of the script, the previous configurations will be reset and the newly chosen settings applied.

Checking Logs

The synchronization process can be monitored via logs.

1. Navigate to the webroot/lsyncd/var/log folder.

2. The status of sync can be viewed in the lsyncd.log and lsyncd.status files.

checking logs


We trust that these instructions will be helpful to you.