Flickr recently finished a project that sped the html download of their pages to users by an average of 30%, by adding a proxy layer between the user request and the webservers. It seems counter-intuitive that you can actually improve the download time by adding an extra (non-caching) layer of machines. So, if the users and the Flickr webservers stayed in the same place, and we still use TCP and HTTP (and the speed of light didn't change), how could we affect the download time?
Now, when a user connects to www.flickr.com, they first connect through a proxy/cache server instead of directly connecting to our webservers. Yahoo has set up a globally distributed system of proxy/cache servers, roughly based on the Apache Traffic Server. When a browser does a DNS lookup for www.flickr.com, Yahoo's DNS system attempts to find the closest available proxy/cache server to you. For instance, from my home in Mountain View, California, my request would likely be directed to a proxy/cache inRead More »from Flickr’s New Dynamic Content Acceleration