Actor System Metrics #

Kamon is able to gather metrics from all core Akka components: actor systems, actors, dispatchers, routers, remoting channels and, one addition of ours, actor groups. All thanks to bytecode instrumentation shipping with Kamon modules.

Base Metrics #

These metrics are collected for all actor systems, without any special configuration or filtering:

Filtered Metrics #

Since your application might be creating thousands of different actors, routers or even several dispatchers, Kamon won’t simply catch them all but rather rely on filters to decide which components to track. Filters are configured under the kamon.instrumentation.akka.filters configuration path. Within this path, these filters are expected:

• actors.track: Decides which actors should be instrumented for dedicated metrics collection. This means that each actor matched by this filter will get its own set of Actor metrics.
• routers: Decides which routers should be instrumented for metrics collection.
• dispatchers: Decides which dispatcher should be instrumented for metrics collection.

Here is how this would look like in your configuration file:

kamon.instrumentation.akka.filters {

  actors.track {
    includes = [ "my-app/user/job-manager", "my-app/user/worker-*" ]
    excludes = [ "my-app/system/**", "my-app/user/worker-helper" ]
  }

  dispatchers {
    includes = [ "my-app/akka.actor.default-dispatcher" ]
  }

  routers {
    includes = [ "my-app/user/some-router" ]
  }
}

In the example filters above, for an actor system named my-app, all system actors and the user/worker-helper actor are excluded but the user/job-manager and all the user/worker-* actors should be tracked. With regards to dispatchers, only the default dispatcher in the application’s actor system and the database-dispatcher are tracked. Finally, the user/some-router router is the only one to be tracked by Kamon.

The Doomsday Wildcard #

It might be very tempting to track all actors by adding ** in the actors.track filter, which might be very useful for testing purposes but is definitely a bad practice for running in production where thousands of tracked actors could eat all your memory. To avoid this, Kamon does not allow using ** on the actors.track filter, but you can disable this safeguard by adding actors.doomsday-wildcard = on setting to your config. Think twice before doing it, since most likely all you will need is to create actor groups, use the auto-grouping feature or target specific actors.

Actor Metrics #

These metrics are recorded for all tracked actors:

Router Metrics #

When a router is instrumented, metrics are recorded both from the router itself (measuring how long it takes to process the routing logic) and from all the routees behind the router, recording the following metrics together:

Dispatcher Metrics #

The Dispatchers instrumentation relies on the kamon-executors module to capture metrics on the underlying executor service for Akka dispatchers. The following metrics will be recorded:

Besides the default tags added to all metrics be the Executors module, the Dispatchers instrumentation will add a system tag with the name of the Actor System to which the dispatcher belongs.

Actor Groups #

There are situations in which you don’t want to track how a specific actor instance is behaving but rather, how a group of similar actors are behaving. The typical use cases for actor groups include:

• Worker actors.
• Per-request actors.
• Short lived actors that perform support work for other bigger pieces of the system.

Auto-grouping #

As part of the default instrumentation, Kamon will automatically create groups that contain all actors of the same type at the same level in the Actor System tree. For example, in a system that has the following actors:

• myApp/user/parent with class ParentActor
• myApp/user/parent/child-1 with class ChildActor
• myApp/user/parent/child-2 with class ChildActor

The instrumentation will automatically create two actor groups: one for all the ParentActor instances in the first level and another for all ChildActor instances in the second level. If you want to control which groups are eligible for auto-grouping use the filters for the auto-grouping group.

Finally, take into account that auto-grouping will only pick up actors that are not being explicitly tracked in any other way (e.g. the actor is not explicitly tracked nor included in an explicit group).

Explicit Groups #

Besides auto-grouping, you can explicitly create groups be defining filters under the groups section in the configuration:

kamon.instrumentation.akka.filters.groups {
  worker-actors {
    includes = [ "my-app/user/worker-*" ]
  }
}

The name of the group will be the name of the configuration (worker-actors in the example above) and the group will include all actors matching the include/excludes patterns.

Actor Group Metrics #

Regardless of how the group was created (auto-grouping or explicitly) the following metrics are recorded for all groups:

Remoting Metrics #

If you are using Akka Remote or Akka Cluster, the instrumentation will also record these metrics for the remoting infrastructure: