Fluent Bit Config Generator
Generate production-ready Fluent Bit configurations with inputs, filters, outputs, parsers, and TLS.
Fluent Bit Settings
Inputs (1)
Filters (0)
Outputs (1)
Configuration Health
Configuration needs improvements. Address warnings below for production readiness.
Detailed Analysis
Best Practice Analysis
- Use Fluent Bit as a DaemonSet in Kubernetes to collect logs from every node.
- Enable the HTTP server for Prometheus metrics and health check monitoring.
- Configure DB files on tail inputs to prevent duplicate logs after restarts.
- Set Mem_Buf_Limit to 16MB+ to handle burst traffic without dropping logs.
- Enable TLS on all outputs forwarding logs over networks.
Production Readiness
- Input Sources Configured
- Output Destinations Configured
- TLS Encryption Enabled
- HTTP Metrics Server
- Health Check Enabled
- Offset Tracking (DB)
Overview
The Fluent Bit Config Generator produces production-ready Fluent Bit configurations for lightweight, high-performance log collection. Fluent Bit is an open-source log processor and forwarder part of the Fluentd ecosystem, written in C with a minimal memory footprint (~1MB) making it ideal for edge deployments, Kubernetes DaemonSets, and resource-constrained environments.
Fluent Bit uses an INI-style configuration file with [SERVICE], [INPUT], [FILTER], [OUTPUT], and [PARSER] sections. This generator abstracts the syntax complexities, letting you visually configure inputs, filters, outputs, parsers, TLS, and buffering to produce a perfectly valid configuration.
How It Works
1. Service Configuration: Defines global settings like flush interval, log level, daemon mode, and the path to parsers files. The HTTP server enables Prometheus-compatible metrics at port 2020.
2. Define Inputs: Inputs specify where logs come from. Use tail for files, forward for network forwarding, systemd for journal logs, docker for container logs, kube for Kubernetes metadata enrichment, or tcp/udp/mqtt for streaming sources.
3. Configure Filters: Filters transform and enrich log records. The kubernetes filter adds pod metadata, grep matches patterns, parser applies custom parsers, and modify adds/removes fields.
4. Set Outputs: Outputs route logs to destinations. Choose from Elasticsearch, Loki, Kafka, S3, Datadog, CloudWatch, Splunk, Prometheus, and more. Each output supports buffering, TLS, retry policies, and format options.
5. Add Parsers: Parsers define how raw log lines are structured. Built-in parsers handle JSON, regex, logfmt, combined, NGINX, Apache, Docker, and CRI formats.
Best Practices
- Always set <code>Mem_Buf_Limit</code> on inputs to prevent out-of-memory conditions — start with 16MB and scale based on throughput.
- Use the <code>kubernetes</code> filter with <code>Merge_Log On</code> and <code>Kube_Tag_Prefix</code> to properly enrich container logs with pod metadata.
- Enable the HTTP server in the SERVICE section for Prometheus metrics collection and health checking.
- Configure <code>Retry_Limit</code> on outputs to handle transient failures gracefully without losing logs.
- Use <code>DB</code> and <code>DB.lock</code> on tail inputs to track file offsets across restarts, preventing duplicate log delivery.
- Set <code>Refresh_Interval</code> to control how often Fluent Bit checks for new log lines — lower values reduce latency but increase CPU usage.
- Use TLS encryption for all network-forwarding inputs and outputs in production environments.
- Keep Fluent Bit configurations modular using <code>Include</code> directives to separate inputs, filters, and outputs into distinct files.
Common Mistakes
- Setting <code>Mem_Buf_Limit</code> too low causes Fluent Bit to drop logs when downstream outputs are slow or unavailable.
- Forgetting to set <code>DB</code> on tail inputs causes duplicate logs to be reprocessed after Fluent Bit restarts.
- Not configuring <code>Retry_Limit</code> on outputs can lead to infinite retry loops that consume resources.
- Using incorrect <code>Match</code> patterns in filters — patterns must match the tag exactly or use wildcards like <code>kube.*</code>.
- Omitting the parsers file path in the SERVICE section when using custom parsers in INPUT sections.
Security Recommendations
- Enable TLS for all outputs that forward logs over networks — configure <code>Tls On</code>, <code>Tls.verify On</code>, and provide CA certificates.
- Use <code>HTTP_User</code> and <code>HTTP_Passwd</code> for basic authentication on the built-in HTTP server.
- Restrict the HTTP server to listen on <code>127.0.0.1</code> instead of <code>0.0.0.0</code> to prevent external access to metrics.
- Run Fluent Bit as a non-root user with minimal permissions — use <code>Set_Daemon Off</code> for debugging only.
- Rotate log files and configure retention policies to prevent disk exhaustion attacks.
Production Tips
- Deploy Fluent Bit as a Kubernetes DaemonSet with resource limits (CPU: 100m, Memory: 128Mi) for predictable performance.
- Use <code>docker</code> filter with <code>Docker_Mode On</code> to automatically parse multi-line Docker logs into single records.
- Configure <code>storage.type filesystem</code> in the SERVICE section for disk-backed buffering, enabling survival across restarts.
- Monitor Fluent Bit metrics via the Prometheus endpoint at <code>/api/v1/metrics/prometheus</code> for throughput, errors, and buffer usage.
- Use <code>Tag</code> patterns with wildcards (<code>kube.*</code>, <code>var.log.*</code>) to efficiently route logs through filter and output chains.
- Set <code>storage.backlog.mem_limit</code> to control memory usage for backlog chunks when outputs are temporarily unavailable.