Hello everyone, welcome to my blog. Today, I would like to shed some light on the Varnish cache and its implementation in Magento. This is the first part of the Varnish cache implementation. In this part, I will try to give a simple overview of Varnish cache, reverse proxy, Varnish management process, and storage backends. So let’s jump into it.
The varnish is a “web accelerator” that acts as an HTTP reverse proxy. Basically, a reverse proxy is a type of proxy server which serves the request on the behalf of the actual server. Below is the diagrammatic representation of the request served by the server and reverse proxy.
So, what is the role of Varnish in this case? Simply Varnish acts as a mediator between the server and the requester. Varnish caches all the requests served by the server to the client for the first time and if the requestor is requesting the same content on the second time, Varnish comes into a picture. Varnish saves previous requests and response cached data to the requestor. In this way, we are able to reduce the load on the server by reducing the multiple requests to the webserver and database. Let’s take an overview of the Varnish cache in more detail.
Varnish Overview
As we saw from the above image, Varnish sits between the requestor and the server. Varnish sends a request to the server if the data is not cached or any other issue happens. The varnish is used to cached all the data like CSS, images, HTML which means less web server does less work and which generally means a happier server and a fast website.
If we are accessing any data from the RAM is much faster than any other media, like through disk or from over the network. Varnish uses RAM to save cached data. Varnish can save a large number of files into the RAM for faster access which significantly improves the performance. There is a VCL language that can be used to set up the rules for Varnish, these can be of which files we need to cache, or which file we don’t need to cache, which cache management we should use or how much time we need to keep cached data.
The varnish is RAM hungry so we need to give as much RAM as possible. The varnish is not very CPU intensive unless we are getting a large number of requests. We will see the Varnish configuration in the next part of Varnish in Magento 2.
Process Architecture
Varnish runs with two processes
- Management process
- Child & thread process
Management Process
The management process applies the configuration changes given in the VCL and compiles it to convert in C language code. Management process also handles the child process and if there is a delay in response from the child process or child process exits unexpectedly, the management process is responsible for terminating child processes and starting it again. The management process also handles logging which is done by Syslog.
Child Process
The child process consists of multiple threads which are used to perform some of the following activities but not limited are
- Accept new connections
- Expired thread, evict old content from the cache.
In order to reduce contention between all the threads, Varnish uses workspaces that allow each thread to use or modify memory independently of other threads, which avoids locking. The most important workspace is the session workspace which handles and modifies session data.
VCL Compilation
VCL stands for “Varnish configuration language” which is used for configuring cache policies of Varnish. VCL is interpreted by the management process into C and then compiled by a normal C compiler. This compilation process is done outside the child processes so it will not affect the current configuration of Varnish. The compiled VCL file is kept around until we restart the Varnish, or until the issue in vcl.discard from the management interface.
Storage Backends
Varnish supports three storage options, allocating space for the cache.
- File
- Malloc
- Persistent
Varnish mentioned that Persistent is still in experimental mode, so there are two options left file and Malloc.
Malloc Storage
Using the Malloc method, Varnish requests the entire size of the cache with malloc() memory allocation library call. The entire cache is divided into memory and disk by the operating system, and, the operating system swaps the data when it’s required. In this process, if we don’t limit the RAM usage, Varnish will use the entire RAM(available) for storing cache. But at this point, performance may be affected because there is no available space(memory) for other applications to utilize.
We should use the Malloc method only when we have sufficient RAM to utilize. Let’s say if we have a small website having CSS, images of size 512MB and we are using a server having 2G of RAM so in this case, we can configure Varnish to utilize 512MB of RAM for storing the cache.
Using a Malloc method, Varnish saves objects into the RAM despite the size, and Varnish swaps these objects between disk and RAM based on their usage. Most access objects will be kept in memory, whereas the least access objects will be kept in the disk. There is one thing that we should note here that the malloc method does not retain cached data after a restart. So, if we restart our server it will take some time to warm up the Varnish cache again.
File Storage
In the file storage method, Varnish saves the cache on the filesystem, It then tells the OS via mmap() to map the entire file into memory if possible. The file storage method does not retain the data if we restart the server. We can save the cache on any location of the filesystem. We can modify this location using VARNISH_STORAGE_FILE parameter in the configuration file. It is recommended to use the Malloc method when we are dealing with a small scale website, but, when we need to deal with large amounts of data like 20GB or more we should use File storage as our storage method.
That’s all for now, in the next part we will see how to configure Magento 2 with Varnish cache. Please let me know your thoughts about this blog down in the comment section. Goodbye for now and happy coding 🙂
[post-views]
