Repository navigation
Auto flush on multiline codec not working #156
Description
Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.
For all general issues, please provide the following details for fast resolution:
- Version: 7.6.2
- Operating System: Ubuntu 18.04.3 LTS
- Config File (if you have sensitive info, please remove it):
input {
tcp {
port => 10030
codec => multiline {
pattern => "new_line"
negate => true
what => "previous"
auto_flush_interval => 1
}
}
}
filter {
}
output {
stdout {}
}
- Steps to Reproduce:
- Write any line to tcp port. For example via
nc localhost 10030and line "some_test_line" - Nothing appear in stdout - auto flush is not working
- Write any line to tcp port. For example via
Activity
mj84 commentedon Nov 12, 2020
I can confirm this with 7.10.0 using a tcp input as well.
4 years ago, someone mentioned that only file and beats inputs were supported with the multiline codec:
https://discuss.elastic.co/t/solved-auto-flush-interval-not-flushing-on-udp-input/60387
The current documentations of the tcp input or multiline codec don't mention such a limitation.
This is rather a limitation of the multiline codec and not the tcp input, and there is also an open issue for that:
logstash-plugins/logstash-codec-multiline#57
There also was a pull request work in progress regarding auto_flush, but it is closed and won't be merged for now:
logstash-plugins/logstash-codec-multiline#63
This issue provides a good roadmap in the last post which explains that an "input_type" option will be introduced for csv codec and later for the multiline codec:
elastic/logstash#11885
Sorry for pinging all the issues and PRs.
I though it was a good idea to summarize the state of auto_flush when using multiline with tcp since I ran into this behaviour today.
Google brought me to this issue, hopefully this explains the situation to others facing the same challenge.