Etcd is an open source distributed key value store that provides a reliable way to store data across a cluster of machines. Analyze and optimize your etcd server by configuring our plugin. Use our key indicators to ensure continuous functioning of your etcd server and make informed troubleshooting decisions.
This document details how to configure the etcd plugin and the monitoring metrics for providing in-depth visibility into the performance, availability, and usage stats of etcd servers.
Etcd performance monitoring metrics:
Key metrics we support are:
Successful/failed gets
The metrics "gets_success", "gets_fail" gives the rate of successful or failed get requests that the etcd server recieved.
Successful/failed sets
The metrics "sets_success", "sets_fail" gives the rate of successful or failed set requests that the etcd server recieved.
Successful/failed deletes
The metric "delete_success", "delete_fail" gives the rate of successful or failed delete requests that the etcd server recieved.
Successful/failed updates
The metrics "update_success", "update_fail" gives the rate of successful or failed update requests that the etcd server recieved.
Successful/failed creates
The metrics "create_success", "create_fail" gives the rate of successful or failed create requests that the etcd server recieved.
Successful/Failed compares and swaps
The metric "compare_and_swap_success", "compare_and_swap_fail" gives the rate of successful or failed compare and swap requests that the etcd server recieved.
Successful/failed compares and deletes
The metrics "compare_and_delete_success", "compare_and_delete_fail" gives the rate of successful or failed compare and delete requests that the etcd server recieved.
Expire count
Use the metric "expire_count" and get the total count of expired keys in your etcd setup.
Watchers
The metric "watchers" gives the total count of watchers in your etcd setup.
Append requests recieved
Use the metric "self_recv_appendreq_cnt" and get the total count of append requests that this node has processed.
Requests per second recieved
Use the metric "self_recv_pkg_rate" and get the total number of requests per second this node is recieving. This metric works only if the node is a follower. For a leader node the value is always 0.
Bytes per second recieved
Use the metric "self_recv_bandwidth_rate" and get the total number of bytes per second this node is recieving. This metric works only if the node is a follower. For a leader node the value is always 0.
Requests sent
Use the metric "self_appendreq_cnt" and get the total count of append requests that this node has sent.
Requests per second sent
Use the metric "self_send_pkg_rate" and get the total number of requests per second this node is sending. This metric works only if the node is a leader. For a follower node the value is always 0.
Bytes per second sent
Use the metric "sendbandwidthrate" and get the total number of bytes per second this node is sending. This metric works only if the node is a leader. For a follower node the value is always 0.
How it works?
- Log-in to your Site24x7 account. Sign up here if you don't have one
- Download and install the latest version of Site24x7 Linux agent or Site24x7 Windows agent based on what operating system you are currently using
- Install the etcd plugin
- The agent will execute the etcd plugin and push the data to the Site24x7 server.
Prerequisites:
- The plugin requires Python module "psycopg2" to fetch the statistics from the etcd database if your operating system is Linux.
Etcd plugin Linux installation:
- Go to the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/ and create a directory "etcd_store"
cd /opt/site24x7/monagent/plugins/sudo mkdir etcd_store
- Download the file ""etcd_store.py" from our GitHub repository and place it under the "etcd_store" directory
cd etcd_storesudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_store/etcd_store.py
- Similarly, in the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/, create a directory "etcd_self"
cd /opt/site24x7/monagent/plugins/sudo mkdir etcd_self
- Download the file ""etcd_self.py" from our GitHub repository and place it under the "etcd_self" directory
cd etcd_selfsudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_self/etcd_self.py
Etcd plugin Linux installation:
- Go to the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/ and create a directory "etcd_store"
cd /opt/site24x7/monagent/plugins/sudo mkdir etcd_store
- Download the file ""etcd_store.py" from our GitHub repository and place it under the "etcd_store" directory
cd etcd_storesudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_store/etcd_store.py
- Similarly, in the Site24x7 Linux agent's plugin directory - /opt/site24x7/monagent/plugins/, create a directory "etcd_self"
cd /opt/site24x7/monagent/plugins/sudo mkdir etcd_self
- Download the file ""etcd_self.py" from our GitHub repository and place it under the "etcd_self" directory
cd etcd_selfsudo wget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_self/etcd_self.py
Etcd plugin Windows installation:
- Create a directory with the name "etcd_store" and place them under the Site24x7 Windows agent's plugin directory - C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\
cd C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\mkdir etcd_store
- Download the file ""etcd_store.ps1" from our GitHub repository and place it under the "etcd_store" directory
cd etcd_storewget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_store/etcd_store.ps1
- Similarly, go to the Site24x7 Windows agent's plugin directory - C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\ and create a directory "etcd_self"
cd C:\Program Files\Site24x7\WinAgent\monitoring\Plugins\mkdir etcd_self
- Download the file ""etcd_self.ps1" from our GitHub repository and place it under the "etcd_self" directory
cd etcd_selfwget https://raw.githubusercontent.com/site24x7/plugins/master/etcd_self/etcd_self.ps1
Etcd plugin configuration:
- The default python path given in the plugin script is #!/usr/bin/python. If you wish to provide an alternate python path, replace the existing one preceded by the shebang character "#!".
- Configure host and port values for the etcd plugin
Eg :HOST = "localhost"ADMINPORT = "4848"
- Save the changes and restart the agent.
/etc/init.d/site24x7monagent restart
The below example is for a Linux installation
Monitor additional metrics:
- To monitor additional metrics, edit the "etcd_self.py" and "etcd_store.py" file for Linux installation and add the new metrics that need monitoring. For Windows installation, edit "etcd_store.ps1" and "etcd_self.ps1".
- Increment the plugin version value in the file where changes have been made, to view the newly added metrics (For e.g., change the default plugin version from PLUGIN_VERSION = "1" to "PLUGIN_VERSION = "2")
Related plugins:
- Redis plugin - Monitor performance metrics of your Redis databases
- MongoDB plugin - Monitor performance metrics of your MongoDB databases
- Memcached plugin - Analyze performance of your Memcached server
- Nagios plugin - Execute thousands of Nagios plugins in Site24x7 without the need of running a Nagios server
- Out-of-the-box plugins - Monitor your entire app stack with our extensive list of integrations
- Create custom plugins - Create custom Linux and Windows plugins and monitor custom attributes