Flickr recently finished a project that sped the html download of their pages to users by an average of 30%, by adding a proxy layer between the user request and the webservers. It seems counter-intuitive that you can actually improve the download time by adding an extra (non-caching) layer of machines. So, if the users and the Flickr webservers stayed in the same place, and we still use TCP and HTTP (and the speed of light didn't change), how could we affect the download time?
Now, when a user connects to www.flickr.com, they first connect through a proxy/cache server instead of directly connecting to our webservers. Yahoo has set up a globally distributed system of proxy/cache servers, roughly based on the Apache Traffic Server. When a browser does a DNS lookup for www.flickr.com, Yahoo's DNS system attempts to find the closest available proxy/cache server to you. For instance, from my home in Mountain View, California, my request would likely be directed to a proxy/cache in California or Washington.
$ host www.flickr.com www.flickr.com is an alias for geoycpi-uno-deluxe.gycpi.b.yahoodns.net. ... has address 18.104.22.168...
Whereas someone in the New York City area would likely be directed to a proxy cache in the mid-Atlantic.
$ host www.flickr.com www.flickr.com is an alias for geoycpi-uno-deluxe.gycpi.b.yahoodns.net. ... has address 22.214.171.124 ...
From there, the request is forwarded over an existing connection to another proxy/cache near the Flickr webservers. It's this existing connection that is the magic.
Without the proxy/cache layer, you would make a connection directly to our webservers -- most likely in Texas. From my home in Mountain View, CA, I could expect a roundtrip time of ~60ms -- the TCP three way handshake would take about ~90ms. The webserver can then start processing the request.
With the proxy/cache layer, you would make a (keep-alive) connection to the proxy layer. From my home, the roundtrip time is ~20ms -- the TCP three way handshake would take about ~30ms. A connection from that server to the Texas proxy/cache is used to transmit the request, so no new three way handshake occurs -- the request should get to Texas 30ms later. All said, the connection setup would be 50ms (20ms to the proxy + 30ms for proxy to Texas), shaving 40ms off the download time.
This effect is magnified the further you are from the Flickr servers. For instance, if you are browsing from Singapore, the roundtrip time to the Flickr Texas servers is likely to be in the 200ms range. The three way handshake would thus normally take around a third of a second. Instead, by connecting to a nearby proxy/cache, almost 200ms can be removed.
By sending the proxied connection from one proxy cache to another over Yahoo's network, we can also modify some TCP settings. For instance, for these connections, we typically turn off TCP slow start restart. Normally, for initial connections or after packet loss, TCP slow start makes the content come slowly at first, exponentially speeding up as the network proves it has the capacity to devote larger bandwidth to the transfer. By turning off TCP slow start restart, we minimize the time spent in the slowest phase of TCP's slow start.
To measure the change in download time, we set up global tests using Gomez. They have computers set up throughout the world on different ISP networks. We chose several computers in each geographic region, and had them each download the html from Flickr's home page both using the proxy layer and without it:
- Overall: 32% improvement
- USA: 9% improvement
- Asia: 44% improvement
- Europe, the Middle East and Africa: 34% improvement
There are many other aspects of web performance than network optimization, and
Flickr is always looking for ways to improve total rendering time. There are
still a few tricks we are looking at in the network side to speed -- hopefully
we'll have some more news for you soon.