Need guidance with production monitoring?
Book FREE office hours and we'll help you out
Not sure how to approach production monitoring? Book FREE office hours and we'll help you out

You are viewing documentation for an outdated version. Do you wish to see documentation for the latest version?

Instrumentation/Available Instrumentation

System Metrics #

Our kamon-system-metrics module registers a number of entities with the metrics module that track the performance indicators of both the host machine and the JVM where your application is running.

This module doesn’t have any bytecode instrumentation requirement, and its only requirement to work properly is that the appropriate Sigar native library is correctly loaded. To do so, the kamon-system-metrics module makes use of the sigar-loader library. If your application uses Sigar for other purposes, it is advisable that you take a look at sigar-loader to simplify the sigar native library provisioning process.

As you might expect, you and any other module can subscribe to all the metrics that are reported by this module using the system-metric category and the entity recorder names described below.

By default the kamon-system-metrics module starts with Host and JVM metrics enabled, in the case that you want to enable/disable one of them, you can configure it this way:

kamon {
  system-metrics {
   #sigar is enabled by default
   sigar-enabled = true

   #jmx related metrics are enabled by default
   jmx-enabled = true
  }
}

Dependency Installation #


libraryDependencies += "io.kamon" %% "kamon-system-metrics" % "1.0.1"



    <dependency>
      <groupId>io.kamon</groupId>
      <artifactId>kamon-system-metrics_2.13</artifactId>
      <version>1.0.1</version>
    </dependency>


implementation 'io.kamon:kamon-system-metrics_2.13:1.0.1'

Host System Metrics #

We are using Sigar to gather all the host system metrics information and this requires us to have a few special considerations given that Sigar instances are not thread-safe and some metrics (like cpu usage metrics) do not work correctly when updated in intervals of less than a second. In the sections below, you will see histograms tracking metrics that typically should be recorded with a gauge, but that we couldn’t allow because of the need to have a tight control on timings and thread-safety.

In the case that Sigar can’t obtain some metric in the host, we will log a warning indicating the error and the metric name.

cpu #

  • user: a histogram tracking total percentage of system cpu user time.
  • system: a histogram tracking total percentage of system cpu kernel time.
  • wait: a histogram tracking total percentage of system cpu io wait time.
  • idle: a histogram tracking total percentage of system cpu idle time
  • stolen: a histogram tracking total percentage of system cpu involuntary wait time.

file-system #

  • readBytes: a histogram tracking total number of physical disk reads in bytes.
  • writesBytes: a histogram tracking total number of physical disk writes in bytes.

load-average #

  • one-minute: a histogram tracking the system load average for the last minute.
  • five-minutes: a histogram tracking the system load average for the five minutes.
  • fifteen-minutes: a histogram tracking the system load average for the fifteen minutes.

memory #

  • memory-used: a histogram tracking total used system memory in bytes.
  • memory-cache-and-buffer: a histogram tracking total memory used in cache and buffers memory in bytes.
  • memory-free: a histogram tracking total free system memory in bytes.
  • memory-total: a histogram tracking total system memory capacity in bytes.
  • swap-used: a histogram tracking total used system swap in bytes.
  • swap-free: a histogram tracking total used system swap in bytes.

network #

All network metrics represent the aggregate of all interfaces available in the host.

  • rx-bytes: a histogram tracking total number of received packets in bytes.
  • tx-bytes: a histogram tracking total number of transmitted packets in bytes.
  • rx-errors: a histogram tracking total number of packets received with errors. This includes too-long-frames errors, ring-buffer overflow errors, etc.
  • tx-errors: a histogram tracking total number of errors encountered while transmitting packets. This list includes errors due to the transmission being aborted, errors due to the carrier, etc.
  • rx-dropped: a histogram tracking total number of incoming packets dropped.
  • tx-dropped: a histogram tracking total number of outgoing packets dropped.

process-cpu #

  • process-user-cpu: a histogram tracking the total percentage of CPU spent by the application process in user space, relative to the overall CPU usage.
  • process-system-cpu: a histogram tracking the total percentage of CPU spent by the application process in system space, relative to the overall CPU usage.
  • process-cpu: a histogram tracking the total percentage of CPU spent by the application, relative to the overall CPU usage.

context-switches #

The context switches metrics are special in the sense that they are not read using the Sigar library but rather reading the information available in the /proc/$pid/status file for Linux systems.

  • context-switches-process-voluntary: Total number of voluntary context switches related to the current process (one thread explicitly yield the CPU to another).
  • context-switches-process-non-voluntary: Total number of involuntary context switches related to the current process (the system scheduler suspends an active thread, and switches control to a different thread).
  • context-switches-global: Total number of context switches across all CPUs.

JVM Metrics #

All JVM-specific metrics are gathered using JMX and all of them are using gauges to record the data. The reported JVM metrics include:

*-garbage-collector #

Depending on your specific instance configuration, the available garbage collectors will differ, but the same set of metrics are recorded regardless of the collector in place.

  • garbage-collection-count: a gauge tracking the number of garbage collections that have occurred.
  • garbage-collection-time: a gauge tracking the time spent in garbage collections, measured in milliseconds.

class-loading #

  • classes-loaded: a gauge tracking the number of classes ever loaded by the application.
  • classes-unloaded: a gauge tracking the number of classes ever unloaded by the application.
  • classes-currently-loaded: a gauge tracking the number of classes currently loaded by the application.

heap-memory #

  • heap-used: a gauge tracking the amount of heap memory currently being used in bytes.
  • heap-max: a gauge tracking the maximum amount of heap memory that can be used in bytes.
  • heap-committed: a gauge tracking the amount of memory that is committed for the JVM to use in bytes.

non-heap-memory #

  • non-heap-used: a gauge tracking the amount of non-heap memory currently being used in bytes.
  • non-heap-max: a gauge tracking the maximum amount of non-heap memory that can be used in bytes.
  • non-heap-committed: a gauge tracking the amount of non-heap memory that is committed for the JVM to use in bytes.

threads #

  • daemon-thread-count: a gauge tracking the total number of daemon threads running in the JVM.
  • peak-thread-count: a gauge tracking the peak number of threads running in the JVM since it started.
  • thread-count: a gauge tracking the total number of live threads in the JVM, including both daemon and non-daemon threads.
On this article
Kamon APM Logo
Monitor and fix issues in production without being an expert
Learn about APM
Try Kamon APM I know how Kamon APM can help with monitoring. Don't show this again.