Docker is the popular open platform that makes the task of creating, deploying, and running your applications inside containers easier to do. Containers are a form of operating system level virtualization that allow you to run an application and dependencies together in a small package. Containers themselves have been around for a number of years in various forms and the availability of the Docker framework has made this technology much more accessible and useable.

Containers start and stop almost instantly, are extremely modular, and easily scale based on the needs of your clients or organization. Due to their low performance overhead on the host system, where you might have run three or five virtual systems, you can now run significantly more containers. This enables organizations to create highly scalable and dynamic platforms that more efficiently leverage the resources available to them.

So, where do we start when trying to monitor Docker and our containers? First and foremost, we need to validate the health and availability of the host system, and assess its capability to run any dependent applications. Our baseline monitoring solutions (CPU, Hardware, Network & DISK) easily provide the information you need to make this determination. We also need to ensure that the Docker process itself is up and running. In addition to telling us the current state of a process, our Processes plugin will also provide CPU and Memory utilization details, startup arguments, time of startup, and other information specific to that process. By including a few more details, we can also start and stop the various processes and access the Docker log files directly from the Active Console.

Since we are most likely monitoring a highly scalable, dynamic application platform, we should expect that there is at least one container instance of the appropriate application running on the host system. Once we understand what containers are running on the host, we need to look at the performance of each container and how it's impacting the overall system performance.


 

By modifying the output of ‘docker stats’ to CSV format in a quick shell script, we can see containers running on the host, including their container ID, name, port information, and uptime. In the screenshot above, we’ve used the Compute Engine to count the number of rows in the dataview so we can evaluate that at least a single instance is running.

To collect performance statistics about each container, we can use a similar shell script that executes the ‘docker stats’ command with an additional flag, ‘- - no stream’, and outputs to CSV format. This returns each container’s individual CPU and Memory utilization information, NET IO and BLOCK IO Information, and the configured limits for each.

Finally, we can use our GatewaySQL plugin to combine these two views in order to provide us with all the information we need in a single view. Alternatively, if the system is host to multiple types of containers, we need to ensure that there is a single instance of each type of container running. Using GatewaySQL, we can create views according to the container name using simple SQL filters.

To monitor your applications inside a container it will typically require the installation of our Self-Announcing Netprobe. Our agents are designed to be extremely light-weight and use limited resources, so the performance implications on the container itself would be minimal. Self-Announcing Netprobes are recommended because of the dynamic nature of the platform, so when a container comes online, they will announce themselves to the Gateway(s) in your environment, and quickly appear in your view. To help reduce unnecessary performance overhead, we strongly advise that you do not monitor baseline metrics (CPU/HARDWARE/Network/Disk/DEVICE-IO) from inside the container. We have access to these details through the Docker command interface, which should be sufficient for most requirements. Once you have our agent installed, you will have access to all of our other monitoring tools, including our more advanced toolkit solutions, log file parses, etc.

If installing a Netprobe within the container isn’t an option for you, we can monitor log files from outside the container by mapping a local volume to the container itself. In addition to looking for keywords or phrases in the log file we can setup automated clear keys, view the log file size, update rate, modification times, and other details to help us know things are operating as expected. If your application has a remote API we can make calls against, you can create a script in the language of your choice that will return the results in CSV format so they are quickly consumed and evaluated by the Gateway server.

The scripts used to create the first two views in this post were written by our EMEA Pre-Sales team and can be accessed here. We hope to provide additional information on monitoring the Docker engine and your containers in future posts. For now, this should serve as a general guide as to what you can do and how you can do it using Geneos. If you have any questions about the information here or our recommendations, you can always contact our support team or your Technical Account Manager.

Tags: Geneos