<p><a href="https://www.linkedin.com/in/alan-m-carroll-b128623">Alan M. Carroll</a> and <a href="https://www.linkedin.com/in/brian-neradt-0a14924/">Brian Neradt</a>, Software Engineers, Verizon Media<br/></p><p>We’re pleased to announce <a href="https://github.com/yahoo/proxy-verifier">Proxy Verifier</a> - an open source tool for testing HTTP based proxies. Originally built as part of Verizon Media’s support for <a href="https://trafficserver.apache.org">Apache Traffic Server</a> (ATS) to improve testability and reliability, Proxy Verifier generates traffic through a proxy and verifies the behavior of the proxy. A key difference between Proxy Verifier and existing HTTP based test tools is Proxy Verifier verifies traffic to and from the proxy. This bi-directional ability was a primary motivation. In addition, handling traffic on both sides of the proxy means a Proxy Verifier setup can run in a network disconnected environment, which was an absolute requirement for this work - no other servers are required, and the risk of hitting production servers with test traffic is eliminated.<br/></p><p>After sharing the idea for Proxy Verifier with the Apache Traffic Server community, we’ve received significant external interest. We are pleased to have achieved a level of maturity with the tool’s development that we can now share it with the world by open sourcing it. As a related benefit, by open sourcing Proxy Verifier we will also be able to use it as a part of Traffic Server’s end-to-end test automation.<br/></p><p><b></b></p><p>Within Verizon Media, Proxy Verifier serves to support correctness, production simulation, and load testing. Generated and captured replay files are used for production simulation and load testing. Handbuilt replay files are used for debugging and correctness testing. Replay files are easily constructed by hand based on use cases or packet capture files, and also easily edited and extended later. Proxy Verifier is being integrated into the <a href="https://bitbucket.org/autestsuite/reusable-gold-testing-system/src/master/">AuTest framework</a> used in ATS for automated end-to-end testing.</p><p>Proxy Verifier builds two executables, the client and server, which are used to test the proxy:</p><figure data-orig-width="1378" data-orig-height="392" class="tmblr-full"><img src="https://66.media.tumblr.com/c729047bc3af58e4a30339cc8311e2b0/2ca4991e6f659b8b-e0/s540x810/37126d81e602823b3f1ece1eae089a38b44cc40f.png" alt="image" data-orig-width="1378" data-orig-height="392"/></figure><p>The client sends requests to the proxy under test, which in turn is configured to send them to the server. The server parses the request from the proxy, sends a response, which the proxy then sends to the client. This traffic is controlled by a “replay file”, which is a YAML formatted configuration file. This contains the transactions as four messages - client to proxy, proxy to server, server to proxy, and proxy to client. <br/></p><p>Transactions can be grouped into sessions, each of which represents a single connection from the client to the proxy.<br/></p><p>This set of events are depicted in the following sequence diagram:</p><figure data-orig-width="1456" data-orig-height="1242" class="tmblr-full"><img src="https://66.media.tumblr.com/e1b3928731a3afe587913b2663c15801/2ca4991e6f659b8b-ca/s540x810/2de5801691f0bc99aef3132bf2e3702aa5722037.png" alt="image" data-orig-width="1456" data-orig-height="1242"/></figure><p><b></b></p><p>Because the Proxy Verifier server needs only the replay file and no other configuration, it is easy for a developer to use it as a test HTTP server instead of setting up and configuring a full web server.</p><p>Other key features:<br/></p><p><b></b></p><ul><li>Fine-grained control of what is sent from the client and server, along with what is expected from the proxy.<br/></li><li>Specific fields in the proxy request or response can be checked against one of three criteria: the presence of a field, the absence of a field, or the presence of a field with a specific value.<br/></li><li>Transactions in the config can be run once or repeatedly a specified number of times.<br/></li><li>Sessions allow control of how much a client session is reused.<br/></li><li>Transactions can be sent at a fixed rate to help simulate production level loads. Proxy Verifier has been tested up to over 10K RPS sustained.<br/></li><li>The &ldquo;traffic_dump&rdquo; plugin for ATS can be used to capture production traffic for later testing with Proxy Verifier.<br/></li><li>Protocol support:<br/></li><ul><li>IPv4 and IPv6 support.<br/></li><li>HTTP/1.x support for both the Verifier client and server.<br/></li><li>The Verifier client supports HTTP/2 but the server currently does not. We have plans to support server-side HTTP/2 sometime before the end of Q2 2020.<br/></li><li>HTTPS with TLS 1.3 support (assuming Proxy Verifier is linked against OpenSSL 1.1.1 or higher). </li></ul></ul><p>For build and installation instructions, explore the github <a href="https://github.com/yahoo/proxy-verifier">README</a> page. Please file <a href="https://github.com/yahoo/proxy-verifier/issues">github issues</a> for bugs or enhancement requests.<br/></p><p><b>Acknowledgments</b><br/></p><p>We would like to thank several people whose work contributed to this project:<br/></p><ul><li>Syeda “Persia” Aziz, initial work and proof of concept for the replay server.</li><li>Jesse Zhang, previous generation prototype and the schema.<br/></li><li>Will Wendorf, initial verification logic.<br/></li><li>Susan Hinrichs, implemented the client side HTTP/2 support.<br/></li></ul>