Saving JMX Metric Data to CSV Files
Here are some Tcat admin console groovy scripts for saving JMX metric data from your Tcat server group servers to CSV files from the Tcat console JVM. You can copy and paste these into your admin shell and save them there, and below is some info about how to use the scripts.
These scripts require you to have a server group, and to configure server group ID into the top of the scripts. Once the group exists in the Tcat console, you can obtain the group ID via the REST API, for instance with this curl command:
> curl --basic -u admin:admin http://localhost:8080/console/api/serverGroups
Next, the script will monitor one or more JMX attributes. You will need to input the JMX ObjectName and attribute name strings, and whether you want to track it as a delta (a boolean), each in their own array, also shown near the top of each script:
These three arrays are comma separated, and the first element of each describes the first metric to be monitored, the second element of each describes the second metric to be monitored, and so on. Each of these three arrays has to have the same number of elements. If you're monitoring two metrics, each array has to contain two elements.
Next, set the data collection period in whole seconds. This is how often you would like the metric values to be collected. You may go as low as a 1 second period, but the lower you go, the more metric data collection load you place on your servers. Choose a period that will be okay for more than one metric, because CSV files are currently per (server JVM & period).
Configure each of the scripts as described above – make sure they are configured the same.
The Start script
Run the start script just once when you have it configured. You should see a success message in the admin shell, and your Tomcat's catalina.out log file should show that it is now collecting metric data for the metrics you listed, on each of the servers in the Tcat server group matching the ID you configured. As soon as you execute this script, it adds a group membership event listener that will make any servers added to the group also begin monitoring these metrics, and any server you remove from the group will have the metric monitors removed as well. You can run the start script as many times as you like – it will only refresh the listener and monitors on each subsequent re-run, whether or not any refresh is needed. You should schedule this start script to run on Tcat startup so that the event listener runs after Tcat console server JVM restarts.
The CSV Save Writer script
Run the CSV save writer script once you've run the start script. The CSV save writer script is what writes the metric data to CSV files. Near the top of the script, you can configure the directory where the CSV files are written. By default, this is in the Tomcat logs/ directory, but you may configure any absolute filesystem path here. Each invocation of the CSV save writer script collects the JMX metric value(s) from each server in the monitored server group, and writes one CSV file per server JVM. The period of all data in one CSV file is the same period, because dealing with mixed period data is a more complex problem. If you want to collect data of different periods, you should duplicate the scripts, one configured copy for each sampling period you would like to use. Schedule the CSV save writer script to run in the Tcat console admin shell at the same period that you configure data collection to occur inside the CSV save start script. The Tcat admin shell scheduler must run the writer script at the same period. When configuring the scheduler to run the CSV save writer script, you should check the Allow Concurrent Execution checkbox.
Currently, the CSV files grow unbounded, but you may either move, truncate, or delete them at any time, and the console will begin a new one. If you have an external log rolling system (such as logrotate on Linux), you can roll these CSV files that way. Otherwise, you can allow the CSV files to grow until you either delete it or move it to another filename.
The Stop script
Simply removing servers from the monitored Tcat server group should be enough for the Start script's event listener to stop monitoring the metrics on the server that is being removed. But, if you would like to stop monitoring the metrics on all servers in the group, run this Stop script. It will remove all monitors for the listed metrics from every server in the monitored Tcat server group. If the CSV save writer script is still running, the metric values it logs will no longer be changing values.. but the CSV save writer script will continue to log into the CSV files until you reconfigure the scheduler to stop running the CSV save writer script.