- 11. Sep
Fluent Bit is an open source and multi-platform log forwarder tool which aims to be a generic Swiss knife for log collection and distribution.
We, Treasure Data, as a Big Data company, provide an analytics infrastructure in the Cloud where we provide an end-to-end solution to collect, store and do analytics over the data. Fluent Bit is an integral part of this pipeline where it solves the log collection needs.
Being an open source project, it has been widely adopted to solve logging needs in Cloud Native environments where Docker and Kubernetes are key components; Fluent Bit is a natural fit.
It have been designed to be lightweight and provide high performance. All I/O operations are made in asynchronous mode (event-driven), all data internally is serialized in binary mode.
All input and output features are implemented through the plugins interface. At build enable just include what you really need. Create your own plugins easily.
Meassuring is important. Fluent Bit comes with native plugins to gather metrics from your CPU and Memory usage on Linux systems.
Event Driven: Fluent Bit as a service is fully event-driven, it only use asynchronous operations to collect and deliver data.
Routing: The data that comes through an Input plugin, is tagged. The Routing mechanism allows specifying where these records should be delivered. It also allows delivering one record to multiple destinations.
Configuration: The service uses a simple and human-readable configuration file, it allows to specify how it will behave, which features to enable and how Routing is performed. Optionally Fluent Bit can run from the command line without it.
I/O Handler: The Input/Output layer provides an abstraction that allows performing read/write operations in an asynchronous way.
Upstream Manager: When dealing with upstream backends like remote network services, handling TCP connections can be challenging. The Upstream manager simplifies the connection process and takes care of timeout/network exceptions and Keepalive states.
TLSv1.2 / Security: When delivering data to destinations, output plugins inherit full TLS capabilities in an abstracted way. Add your certificates as required.