Kapacitor

Kapacitor is a native data processing engine in the TICK Stack. It can process both stream and batch data from InfluxDB. It lets you plug in your own custom logic or user-defined functions to process alerts with dynamic thresholds, match metrics for patterns, compute statistical anomalies, and perform specific actions based on these alerts like dynamic load rebalancing. It also integrates with HipChat, OpsGenie, Alerta, Sensu, PagerDuty, Slack, and more.

Why We Use?

Action Oriented

Today’s modern applications require more than just dashboarding and operator alerts—they need the ability to trigger actions. Kapacitor’s alerting system follows a publish-subscribe design pattern. Alerts are published to topics and handlers subscribe to a topic. This pub/sub model and the ability for these to call User Defined Functions make Kapacitor very flexible to act as the control plane in your environment, performing tasks like auto-scaling, stock reordering, and IoT device control.

Streaming Analytics

Kapacitor is designed to process streaming data in real-time. It can be deployed across the infrastructure as both a pre-processor to downsample and perform advanced analytics before shipping the data to InfluxDB, and a post-processor allowing older high-precision data to be stored in data stores like Hadoop (for example) for further analysis. Kapacitor is very easy to use and is also very powerful. It allows the scripting to be done using lambda expressions to define transformations on data points as well as define boolean conditions that act as the filter.

Anomaly Detection

Kapacitor provides a simple plugin architecture, or interface, that allows it to integrate with any anomaly detection engine. This means that Kapacitor can integrate with machine learning libraries, pattern matching engines, rules engines, and the like. Kapacitor treats these plugins as User Defined Functions which allow the flexibility to yield the stream to an arbitrary function and resume processing on the output of that function. This makes Kapacitor a powerful control system, calling User Defined Functions that automate the complete system.

Key features

Here are some of the features that Kapacitor currently supports that make it a great choice for data processing.

• Process both streaming data and batch data.
• Query data from InfluxDB on a schedule, and receive data via the line protocol and any other method InfluxDB supports.
• Perform any transformation currently possible in InfluxQL.
• Store transformed data back in InfluxDB.
• Add custom user defined functions to detect anomalies.
• Integrate with HipChat, OpsGenie, Alerta, Sensu, PagerDuty, Slack, and more.

Kapacitor is a data processing engine. It can process both stream and batch data. This guide will walk through both workflows and teach the basics of using and running a Kapacitor daemon.

What will be needed

Do not worry about installing anything at this point. Instructions are found below.

The following applications will be required:

• InfluxDB – While Kapacitor does not require InfluxDB, it is the easiest integration to setup and so it will be used in this guide. InfluxDB >= 1.3.x will be needed.
• Telegraf – Telegraf >= 1.4.x will be required.
• Kapacitor – The latest Kapacitor binary and installation packages can be found at the downloads page.
• Terminal – The Kapacitor client application works using the CLI and so a basic terminal will be needed to issue commands.

The process

• Install InfluxDB and Telegraf.
• Start InfluxDB and send it data from Telegraf.
• Install Kapacitor.
• Start Kapacitor.
• Define and run a stream task to trigger CPU alerts.
• Define and run a batch task to trigger CPU alerts.