运维监控系列文章入口【运维监控】系列文章汇总索引最近在研究 AI BI智能数据分析 的落地实践。敬请期待后续专题实战系列《从零手把手教你搭建 AI 驱动的 BI 系统》将覆盖 Text2SQL、多轮对话、语义层、权限治理、生产级部署全链路代码可落地、坑点全复盘。文章目录一、prometheus二、grafana三、tomcat与jmx_exporter配置1、下载jmx_exporter2、部署jmx_exporter3、添加tomcat的配置信息4、修改tomcat的启动文件5、重启tomcat及验证6、其他四、集成prometheus与grafana监控tomcat1、配置prometheus2、导入grafana模板3、验证本示例通过jmx_exporter收集tomcat的运行情况将数据收集到prometheus中然后通过grafana的dashboard导入模板进行可视化。本示例分为四个部分即prometheus、grafana部署、tomcat与jmx_exporter配置和最后的集成。说明jmx_exporter需要与tomcat在同一台机器上部署这个好理解因为需要通过jmx代理运行tomcat服务。关于java应用的监控本系列有文章如下【运维监控】influxdb 2.0telegraf 监控tomcat 8.5运行情况【运维监控】influxdb 2.0grafana 监控java 虚拟机以及方法耗时情况【运维监控】Prometheusgrafana监控tomcat运行情况【运维监控】Prometheusgrafana监控spring boot 3运行情况一、prometheus参考【运维监控】prometheusnode exportergrafana 监控linux机器运行情况关于prometheus的部署。二、grafana参考【运维监控】prometheusnode exportergrafana 监控linux机器运行情况 关于grafana的部署。三、tomcat与jmx_exporter配置1、下载jmx_exporter用于集成到tomcat实施监控指标采集通过代理实现jmx信息的收集。下载地址https://github.com/prometheus/jmx_exporter/tree/release-1.0.1/docs下载版本jmx_prometheus_javaagent-1.0.1.jar2、部署jmx_exporter将下载的jmx_prometheus_javaagent-1.0.1.jar文件放在tomcat所在机器的任一位置即可本示例放在/usr/local/apache-tomcat-8.5.30/jmx_exporter目录下。3、添加tomcat的配置信息可自己编写也可以在下面地址下载可直接使用https://github.com/prometheus/jmx_exporter/blob/main/example_configs/tomcat.yml下载后的源码如下文件名称随意存放位置随意本示例存放位置在/usr/local/apache-tomcat-8.5.30/jmx_exporter目录下。# https://grafana.com/grafana/dashboards/8704-tomcat-dashboard/---lowercaseOutputLabelNames:truelowercaseOutputName:truewhitelistObjectNames:[java.lang:typeOperatingSystem,Catalina:*]blacklistObjectNames:[]rules:-pattern:CatalinatypeServerserverInfo: (.)name:tomcat_serverinfovalue:1labels:serverInfo:$1type:COUNTER-pattern:CatalinatypeGlobalRequestProcessor, name\(\w-\w)-(\d)\(\w):name:tomcat_$3_totallabels:port:$2protocol:$1help:Tomcat global $3type:COUNTER-pattern:Catalinaj2eeTypeServlet, WebModule//([-a-zA-Z0-9#/%?~_|!:.,;]*[-a-zA-Z0-9#/%~_|]), name([-a-zA-Z0-9/$%~_-|!.]*), J2EEApplicationnone, J2EEServernone(requestCount|processingTime|errorCount):name:tomcat_servlet_$3_totallabels:module:$1servlet:$2help:Tomcat servlet $3 totaltype:COUNTER-pattern:CatalinatypeThreadPool, name(\w-\w)-(\d)(currentThreadCount|currentThreadsBusy|keepAliveCount|connectionCount|acceptCount|acceptorThreadCount|pollerThreadCount|maxThreads|minSpareThreads):name:tomcat_threadpool_$3labels:port:$2protocol:$1help:Tomcat threadpool $3type:GAUGE-pattern:CatalinatypeManager, host([-a-zA-Z0-9#/%?~_|!:.,;]*[-a-zA-Z0-9#/%~_|]), context([-a-zA-Z0-9/$%~_-|!.]*)(processingTime|sessionCounter|rejectedSessions|expiredSessions):name:tomcat_session_$3_totallabels:context:$2host:$1help:Tomcat session $3 totaltype:COUNTER4、修改tomcat的启动文件修改/usr/local/apache-tomcat-8.5.30/bin目录下的catalina.sh文件中的JAVA_OPTS变量加入javaagent相关配置增加内容“-javaagent:/usr/local/apache-tomcat-8.5.30/jmx_exporter/jmx_prometheus_javaagent-1.0.1.jar12345:/usr/local/apache-tomcat-8.5.30/jmx_exporter/tomcat.yml”路径需要写绝对路径所以上述的jar和yml文件放在哪里都无所谓。12345是端口可以随意设置。修改后的JAVA_OPTS变量属性如下。JAVA_OPTS$JAVA_OPTS-Dorg.apache.catalina.security.SecurityListener.UMASKumask-javaagent:/usr/local/apache-tomcat-8.5.30/jmx_exporter/jmx_prometheus_javaagent-1.0.1.jar12345:/usr/local/apache-tomcat-8.5.30/jmx_exporter/tomcat.yml5、重启tomcat及验证重启tomcat后验证tomcat是否正常启动以及验证指标是否收集到即可。在浏览器中输入地址http://server4:12345/出现如下页面点击“/metrics”链接http://server4:12345/metrics出现如下页面即成功。6、其他如果是jar运行环境执行命令如下# jmx_prometheus_javaagent-1.0.1.jar和yml文件根据自己需要进行设置java-javaagent:./jmx_prometheus_javaagent-1.0.1.jar12345:tomcat.yml-jaryourJar.jar如果是war运行环境参考tomcat的部署方式四、集成prometheus与grafana监控tomcat1、配置prometheus/usr/local/bigdata/prometheus-2.54.0.linux-amd64/prometheus.yml配置文件增加关于jmx_exporter的配置即如下- job_name:tomcat_node_10.44# metrics_path defaults to /metrics# scheme defaults to http.static_configs: - targets:[server4:12345]关于本示例的完整配置文件示例如下。# my global configglobal: scrape_interval: 15s# Set the scrape interval to every 15 seconds. Default is every 1 minute.evaluation_interval: 15s# Evaluate rules every 15 seconds. The default is every 1 minute.# scrape_timeout is set to the global default (10s).# Alertmanager configurationalerting: alertmanagers: - static_configs: - targets:# - alertmanager:9093# Load rules once and periodically evaluate them according to the global evaluation_interval.rule_files:# - first_rules.yml# - second_rules.yml# A scrape configuration containing exactly one endpoint to scrape:# Here its Prometheus itself.scrape_configs:# The job name is added as a label jobjob_name to any timeseries scraped from this config.- job_name:prometheus# metrics_path defaults to /metrics# scheme defaults to http.static_configs: - targets:[server4:9090]- job_name:tomcat_node_10.44# metrics_path defaults to /metrics# scheme defaults to http.static_configs: - targets:[server4:12345]完成以上配置后重启prometheus即可。验证prometheus是否配置成功比较简单打开prometheus的web UI界面点击“Status”下的“Targets”链接出现配置的job_name点击job_name对应的“show more”按钮再点击“Endpoints”对应的链接如果有相应的监控指标数据即可。如下2个图所示。2、导入grafana模板模板地址https://grafana.com/grafana/dashboards/19532-apache-tomcat/导入模板示例图如下3、验证在grafana的dashboard中查看添加的“Apache Tomcat”dashboard如下图。以上完成了集成操作。