parser_syslog: accept a space after RFC3164 priority - #5449
Conversation
There was a problem hiding this comment.
The following test should pass. It works on the master branch, but on this PR the behavior changes and it fails.
data(
'boundary space' => '<14> Apr 25 16:43:29.5',
)
def test_truncated_subsecond_rfc3164_is_rejected_without_raising(text)
@parser.configure(
'parser_engine' => 'string',
'message_format' => 'rfc3164',
'with_priority' => true,
'time_format' => '%b %d %H:%M:%S.%N',
)
result = :not_yielded
assert_nothing_raised do
@parser.instance.parse(text) do |time, record|
result = [time, record]
end
end
assert_equal([nil, nil], result)
endOn master, the input <14> Apr 25 16:43:29.5 (a truncated message with a boundary space, no host/message) is gracefully rejected as [nil, nil].
On this PR it raises NoMethodError(<undefined method '-' for nil>) instead.
Please add above test case in this PR.
|
Thanks for catching this. Addressed in
Validation on the published SHA:
There is no inline review thread to resolve, so I’m replying in the PR conversation. The branch is ready for re-review. Hosted |
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
The byte-index path added for the space after the RFC3164 priority mixed byte offsets with the character offsets used by String#slice. A UTF-8 header carrying a multibyte character was handed to the time parser instead of being rejected. Drop that path and check the priority digits on the substring itself, guard the space_count branch against a truncated header in both the string and the regexp parser, and search for the subsecond delimiter from the end of the timestamp. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Accepting the boundary space moves the cursor one character forward, so a record whose timestamp is malformed can line up with the fixed-width lookup and reach the time parser. The string parser then raised Fluent::TimeParser::TimeParseError where it used to yield nil, nil. RFC 3164 asks receivers to be liberal in what they accept and not to malfunction on malformed messages, and in_tail cannot apply emit_unmatched_lines once the parser raises. Turn the error back into the documented nil, nil rejection. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
65e84da to
942ef83
Compare
Which issue(s) this PR fixes:
Fixes #4158
What this PR does / why we need it:
Some syslog senders emit RFC3164 records with an ASCII space between the priority and header. Depending on whether
in_syslogor the parser owns priority extraction, the RFC3164 parser sees that space either immediately after>or as the first character of its input, and currently attempts to parse it as the start of the timestamp.This change accepts the RFC3164 boundary space in
rfc3164and the RFC3164 path ofauto, across both parser engines and both priority-ownership models. Canonical records without the space continue to parse unchanged, malformed priorities remain invalid, and RFC5424 parsing is unchanged.For the default RFC3164 time format, regression coverage verifies that two boundary spaces and tabs remain invalid. An explicit RFC3164
time_formatbeginning with a space treats that leading space as timestamp syntax, so this change does not impose or claim a one-space maximum for that configuration.This also corrects the string parser's dynamic timestamp slice cursor and the field-count accounting needed by RFC3164 time formats that begin with a space. Truncated subsecond records with the boundary space are rejected through the normal
[nil, nil]path instead of raising while looking for the missing field delimiter.Docs Changes:
None. This broadens input compatibility without adding or changing configuration.
Release Note:
parser_syslog: accept one optional ASCII space between RFC3164 priority and header.
Testing:
bundle exec ruby -Itest test/plugin/test_parser_syslog.rb --name test_truncated_subsecond_rfc3164_is_rejected_without_raising— 4 tests, 12 assertions, 0 failures, 0 errors acrossrfc3164/autoand parser-owned/input-owned prioritybundle exec ruby -Itest test/plugin/test_parser_syslog.rb— 132 tests, 521 assertions, 0 failures, 0 errorsbundle exec ruby -Itest test/plugin/test_in_syslog.rb— 41 tests, 280 assertions, 0 failures, 0 errorsbundle exec rake test— 4,401 tests, 16,154 assertions, 0 failures, 0 errors, 3 pendings, 36 omissionsgem exec rubocop— 460 files inspected, 0 offensesfluentd --no-supervisorprocesses were started, and packets were sent by external UDP clients. The original 16-scenario matrix covered both parser engines, both priority-ownership models,rfc3164/auto, canonical and spaced packets, whitespace rejection, RFC5424 compatibility, and explicit leading-space time formats. Four additional string-parser scenarios verified that the truncated subsecond packet takes the normal unmatched path without an exception and that a valid.5sibling is parsed by the same live process acrossrfc3164/autoand both priority-ownership modes.