<p><a href="https://www.linkedin.com/in/jamesdiss/">James Diss</a>, Software Systems Engineer, Verizon Media</p> <p>Panoptes is an open source network telemetry system that we built to replace a myriad of tools and scripts built up over time to monitor the fleet of hosts worldwide.</p> <p>The core framework is designed to be extended through the use of plugins which allows the many different devices and device types to be monitored. Within Verizon Media, Panoptes provides the data collection layer that we feed to other projects to allow for visualization of device health according to need, alerting and collation of information.</p> <p>Normally the components of Panoptes run as a distributed system that allows for horizontal scaling and sharding to different geographical and virtual locations, handling thousands of endpoints, but this can be a difficult environment to simulate. Therefore, we have created a docker image which holds the entire structure of Panoptes in a single container. <img src="https://66.media.tumblr.com/59917791544478538356563b5e7a7aa4/tumblr_inline_pp393jrcQR1wxhpzr_540.png" alt="image"/></p> <p>I hasten to add that this is not a production instance of Panoptes, and we would not recommend trying to use the docker container “as-is”. Rather, it is more of a workbench installation to examine in motion.</p> <p>The container is entirely self-contained and builds Panoptes using the freely available package with <code>pip install yahoo-panoptes</code>; it is open-source and built on the ubiquitous Ubuntu 18.04 (Bionic Beaver). A set of scripts are supplied that allow for examination of the running container, and it also runs a Grafana instance on the container to see the data being collected. A dashboard already exists and is connected to the internal data store (influxdb) collecting the metrics. <img src="https://66.media.tumblr.com/f3ee22563d2ae132d1515e89140b334d/tumblr_inline_pp397hNXQ51wxhpzr_540.png" alt="image"/></p> <p>If you would like to get started right away, you can have Panoptes running very easily by using the prebuilt docker hub image;</p> <pre><code>docker pull panoptes/panoptes_docker docker run -d \ --sysctl net.core.somaxconn=511 \ --name="panoptes_docker" \ --shm-size=2G \ -p 127.0.0.1:8080:3000/tcp \ panoptes/panoptes_docker </code></pre> <p>This pulls the docker image from docker hub, then runs the image with a couple of parameters. In order, the “sysctl” command allows redis to run, the “name” is the name that the running container will be given (<code>docker ps</code> shows the currently running containers), and the “shm-size” reserves memory for the container. The “p” parameter exposes port 3000 inside the container to port 8080 on the outside; this is to allow the Grafana library to communicate outside of the container.</p> <p>If you’re more interested in building the image yourself (which would allow you to play with the configuration files that are dropped in place during the build), clone the repo and build from source.</p> <pre><code>git clone <a href="https://github.com/yahoo/panoptes_docker.git">https://github.com/yahoo/panoptes_docker.git</a> &amp;&amp; cd panoptes_docker docker build . -t panoptes_docker </code></pre> <p>Once the image is built, run with:</p> <pre><code>docker run -d \ --sysctl net.core.somaxconn=511 \ --name="panoptes_docker" \ --shm-size=2G \ -p 127.0.0.1:8080:3000/tcp \ panoptes_docker </code></pre> <p>Here are a few useful links and references:</p> <p>Docker Resources</p> <ul><li><p>Docker desktop for Mac <br/><a href="https://docs.docker.com/docker-for-mac/install/">https://docs.docker.com/docker-for-mac/install/</a></p></li> <li><p>Docker desktop for Windows <br/><a href="https://docs.docker.com/docker-for-windows/install/">https://docs.docker.com/docker-for-windows/install/</a></p></li> <li><p>Docker Hub - prebuilt images <br/><a href="https://hub.docker.com">https://hub.docker.com</a></p></li> </ul><p>Panoptes Resources</p> <ul><li><p>Panoptes in Docker prebuilt image <br/><a href="https://hub.docker.com/r/panoptes/panoptes_docker">https://hub.docker.com/r/panoptes/panoptes_docker</a></p></li> <li><p>Panoptes in Docker GitHub repo <br/><a href="https://github.com/yahoo/panoptes_docker">https://github.com/yahoo/panoptes_docker</a></p></li> <li><p>Panoptes GitHub repo <br/><a href="https://github.com/yahoo/panoptes/">https://github.com/yahoo/panoptes/</a></p></li> </ul><p><strong>Questions, Suggestions &amp; Contributions</strong></p> <p>Your feedback and contributions are appreciated! Explore Panoptes, use and help contribute to the project, and <a href="https://join.slack.com/t/panoptescommunity/shared_invite/enQtNTgwNDEzNDg1MjgzLTllMzVmMjFhZDYwZjQ4OTQ4YWZjZWQyNDA4NzgzNGNjN2Q3NjQ2MDE5YzQ0MThkYzM1ODA3MDA4NzA4NDg0ZWU">chat with us on Slack</a>.</p>