The Linux Foundation | 06 December 2017
Logging is a critical component on production environments that allow us to perform monitoring and data analysis. While applications runs at scale, the Logging layer also needs to scale and year over year we see new challenges that needs to be solved from different angles such as parsing, performance, log enrichment (metadata), filtering and so on.
Fluentd was born to solve Logging problems as a whole, not only for standalone applications but also for distributed architectures where each running application and system have their own way to solve logging, integration between all components and the ability to move data from one place to another in a secure and reliable way was a requirement from day one.. and it continue to be as of today. That’s why it has been adopted by thousands of companies and thanks to it growing community around the world demonstrate that Fluentd became the Industry Standard for Logging.
Fast iterations and solving the problem better than yesterday is our mantra, that’s why the global team of maintainers is proud to announce that Fluentd v1.0 has been released!, this is a big milestone for everyone around the Fluentd Ecosystem which means much more that a number, it’s a maturity resulting of years of work in the community where feedback, ideas and general testing have been the roots for it growth. So thank all you who have been involved on this process!
What’s Hot in Fluentd v1.0 ?
Fluentd v1.0 built on top of v0.14-stable, some of the biggest changes on this new series are:
- Core:
- Multiprocess Workers: take advantage of SMP systems
- Sub-second time resolution: all log records now have a granular time resolution
- Portability: Windows Support has finally arrived to Fluentd.
- New plugins API: Our biggest contributions to Fluentd ecosystem is through plugins, with more than 700 plugins available made by the community we have focused into improve the developer experience.
- Data Management: new internal buffers can optionally enable compressions to save disk space.
- Networking:
- New Fluentd Forward Protocol v1: includes authentication using shared keys and authorizations through username/password.
- Native Transport-Layer-Security (TLS) support
These changes are not the only ones, there many improvements around performance, portability and flexibility for data management.
Monitoring and Data Streaming
Fluentd is more than a project, it’s a full ecosystem and integration with third party components is fundamental, that’s why as part of our Fluentd v1.0 release we are proud to announce also the continuous investment in integration in Prometheus (monitoring) and Apache Kafka (data streaming) within many others.
Prometheus makes easier monitoring and having Fluentd provide native support for it has been in high demand in the last time and we are happy to announce that fluent-plugin-prometheus is now officially part of Fluentd Ecosystem hosted on CNCF Fluent organization on Github.
Data streaming is such important as monitoring, and companies around are looking for ways to integrate more components where Logging can be a critical compoment of the data pipeline. Fluentd is getting better and better at it: Fluentd and Apache Kafka can talk each other smoothly and securely.
What’s Next
Next year we will continue working towards performance improvements and connectors that makes easier to hook Fluentd on any place. As well from an ecosystem perspective, Fluent Bit, our lightweight log processor will keep growing in terms of capabilities for cloud native environments such as like load balancing, persistent queues and monitoring within others.
2018 will be an exciting journey, don’t hesitate to be part of it!
Connect at CloudNativeCon + KubeCon
If you are attending KubeCon + CloudNativeCon North America this week, please join us December 6th for a Fluentd Salon, our community will be presenting the following lightning talks:
- “Moving Data at Scale” by Sadayuki Furuhashi (Fluentd Creator)
- “Fluentd and Kafka” by Masahiro Nakagawa (Fluentd Maintainer)
- “Fluentd and Prometheus” by Yuki Ito (Fluentd Maintainer)
- “Fluent Bit” by Eduardo Silva (Fluent Bit Maintainer)
- “Plugging into fluent-bit: how to use plugin templates to customize fluent-bit to serve your needs” by Yeni Capote, Samsung SDS
we will also discuss Fluentd v1.0, roadmap and tools around the Fluent ecosystem plus an open space for discussions. Additionally, do not miss the additional sessions on Fluentd:
Closing Thoughts
Reaching v1.0 for us it’s a new begginning, we will continue working together with our end-users, community and companies around to make Fluentd better for 2018. If you are around CloudNativeCon+KubeCon don’t hesitate to contact us in our sessions or during the break ours, come and join us!, you can also help to solve the problem better than yesterday !
Let’s keep improving the Logging Standard…