How-to Optimize Website Speed with Varnish Cache
Guidelines to speed up your website with Varnish Cache
The page load time is one of the most important factors for today’s online business.
Walmart found a 2% increase in conversions for every 1-second of improvement of website speed. Amazon found 1% increase of revenue for every 100 milliseconds improvement to their website speed. Akamai found that:
Not sure yet? Then think again; a 1-second delay in page load time yields:
Still not sure? Okay, then there's another one:
By using the techniques described below in this article, you can save your hosting costs. You can have a website that works on a cheap $15 hosting to serve as many numbers of users as several hundred dollars hosting.
In this article, we'll describe how you can achieve this by using the Varnish Cache.
Varnish serves data from virtual memory, a response is returned without needing to access the application and database servers. Each Varnish server can handle thousands of requests per second, much faster than a ‘usual’ website's framework alone.
The secret of acceleration is to install Varnish before your application server (Apache, Nginx, IIS). In this case, each HTTP request from browser will first go to the Varnish server. If the current request was not found in the cache, Varnish will request your application server, cache the response and send the response back to the browser. All subsequent requests for similar URLs will be served from the Varnish cache thus unloading your application server.
Another advantage of this approach is that Varnish will improve the availability of your website. For example, if a PHP fatal error breaks your website, the pages still will be served from the Varnish cache and end-users will not realize that anything went wrong.
Of course, each approach has its downside. If you change the content for some of your pages, your users will not be able to immediately see these changes as they will receive the old version of the page from the Varnish cache. But this problem can be easily fixed by forcing Varnish to invalidate specific resources by sending so called 'PURGE' requests to it, to notify that there is a fresh content available for the specified URLs. Varnish will delete the cached content for these URLs and request them again from your server. You can also reduce the time for storing a copy of page in the cache by specifying the time-to-live (ttl) ratio for example to 2 minutes. This will force Varnish to update the data in its cache more frequently.
Another downside relates to security reasons, Varnish can serve only anonymous users and will remove all session-specific cookies from HTTP headers (this can be managed via VCL). Requests from registered users won’t be cached and will go directly to your application server.
Let's say our production website is hosted on 140.45.129.179 IP address and we deployed our Varnish server on 165.227.10.154. One approach is to force browser to visit 165.227.10.154 instead of 140.45.129.179 by changing the domain IP address in the hosts file (%WINDIR%\System32\drivers\etc\hosts). But this requires you to restart your computer every time you change the hosts file.
If you have the HTTP Debugger application installed on your computer, you can do this without changing the hosts' file and without restarting the computer. To do this, just add TCP / IP Redirection Rule to HTTP Debugger as shown on the image:
The positive side of this solution is that in this case Varnish can be configured on any port and not only to 80, and you can quickly enable / disable redirection without restarting the computer.
Below is the same response from a web server without and with Varnish cache enabled. As you can see on the right image, the page was served from the cache and Varnish added/removed some HTTP headers (this can be managed via VCL).
For our test, we chose the free package that allows generating 10,000 requests in 15 seconds. The testing website is a simple ASP.NET website hosted on Microsoft Azure (IIS) for about $ 100/m and does not use MS SQL. The Varnish Cache is hosted on DigitalOcean droplet for $10/m.
Below are test results for without and with using the Varnish Cache. Quite impressive, isn’t it?
Copyright Notice: Please don't copy or translate this article without prior written permission from the HTTPDebugger.com
Walmart found a 2% increase in conversions for every 1-second of improvement of website speed. Amazon found 1% increase of revenue for every 100 milliseconds improvement to their website speed. Akamai found that:
- 47% of people expect a web page to load in two seconds or less.
- 40% will abandon a web page if it takes more than three seconds to load.
- 52% of online shoppers say quick page loads are important for their loyalty to a website.
Not sure yet? Then think again; a 1-second delay in page load time yields:
- 11% fewer page views.
- 16% decrease in customer satisfaction.
- 7% loss in conversions.
Still not sure? Okay, then there's another one:
By using the techniques described below in this article, you can save your hosting costs. You can have a website that works on a cheap $15 hosting to serve as many numbers of users as several hundred dollars hosting.
In this article, we'll describe how you can achieve this by using the Varnish Cache.
What is Varnish Cache?
According to the official documentation, Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. It can speed up delivery of your website with a factor of 300 - 1000x, depending on your architecture. Varnish comes with its own configuration language called VCL. Varnish is used in such large CDN providers as Fastly and KeyCDN.Varnish serves data from virtual memory, a response is returned without needing to access the application and database servers. Each Varnish server can handle thousands of requests per second, much faster than a ‘usual’ website's framework alone.
How does it work?
The secret of acceleration is to install Varnish before your application server (Apache, Nginx, IIS). In this case, each HTTP request from browser will first go to the Varnish server. If the current request was not found in the cache, Varnish will request your application server, cache the response and send the response back to the browser. All subsequent requests for similar URLs will be served from the Varnish cache thus unloading your application server.
Another advantage of this approach is that Varnish will improve the availability of your website. For example, if a PHP fatal error breaks your website, the pages still will be served from the Varnish cache and end-users will not realize that anything went wrong.
Of course, each approach has its downside. If you change the content for some of your pages, your users will not be able to immediately see these changes as they will receive the old version of the page from the Varnish cache. But this problem can be easily fixed by forcing Varnish to invalidate specific resources by sending so called 'PURGE' requests to it, to notify that there is a fresh content available for the specified URLs. Varnish will delete the cached content for these URLs and request them again from your server. You can also reduce the time for storing a copy of page in the cache by specifying the time-to-live (ttl) ratio for example to 2 minutes. This will force Varnish to update the data in its cache more frequently.
Another downside relates to security reasons, Varnish can serve only anonymous users and will remove all session-specific cookies from HTTP headers (this can be managed via VCL). Requests from registered users won’t be cached and will go directly to your application server.
Installing Varnish Cache
There are a lot of good articles about how to install and configure Varnish and we will not elaborate on this in our article. For example you can check this article (you can skip all steps not related to the installation of Varnish).Testing Varnish Cache
Using Varnish requires changing the DNS settings for your domain name provider. This is OK for production server, but how to test it on your development environment?Let's say our production website is hosted on 140.45.129.179 IP address and we deployed our Varnish server on 165.227.10.154. One approach is to force browser to visit 165.227.10.154 instead of 140.45.129.179 by changing the domain IP address in the hosts file (%WINDIR%\System32\drivers\etc\hosts). But this requires you to restart your computer every time you change the hosts file.
If you have the HTTP Debugger application installed on your computer, you can do this without changing the hosts' file and without restarting the computer. To do this, just add TCP / IP Redirection Rule to HTTP Debugger as shown on the image:
The positive side of this solution is that in this case Varnish can be configured on any port and not only to 80, and you can quickly enable / disable redirection without restarting the computer.
Below is the same response from a web server without and with Varnish cache enabled. As you can see on the right image, the page was served from the cache and Varnish added/removed some HTTP headers (this can be managed via VCL).
Load testing
Now let's check how in fact the Varnish can speed up the website by doing a load testing using the loader.io service.For our test, we chose the free package that allows generating 10,000 requests in 15 seconds. The testing website is a simple ASP.NET website hosted on Microsoft Azure (IIS) for about $ 100/m and does not use MS SQL. The Varnish Cache is hosted on DigitalOcean droplet for $10/m.
Below are test results for without and with using the Varnish Cache. Quite impressive, isn’t it?
Conclusion
As we see, using Varnish you can really boost your website speed for dozens of times, increase its reliability and save on hosting costs at the same time. It sounds fantastic, but as we've seen it really works.Useful Links
- https://varnish-cache.org
- https://loader.io
- https://www.httpdebugger.com
- https://www.crazyegg.com/blog/speed-up-your-website
- https://www.digitalocean.com/community/tutorials/how-to-configure-varnish-cache-4-0-with-ssl-termination-on-ubuntu-14-04
- https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl
Copyright Notice: Please don't copy or translate this article without prior written permission from the HTTPDebugger.com
HTTP Debugger is a proxy-less HTTP analyzer for developers that provides the ability to capture and analyze HTTP headers,
cookies, POST params, HTTP content and CORS headers from any browser or desktop application.
Awesome UI and very easy to use. Not a proxy, no network issues!
Download FREE 7-Day Trial
HTTP Debugger
Debug HTTP API calls to a back-end and between back-ends.
Very easy to use.
Not a proxy, no network issues!
Download Free Trial