我有以下日志文??件，每個 Logstash 記錄都應該包含以破折號 '----' 結尾的多行行，從示例日志中您可以看到我有四個鍵，最后一個鍵是 'Message' 。我已經配置了以下組態檔來處理除以“訊息：”開頭的行之外的所有行，正如您在某些情況下從日志中看到的那樣，“訊息：”包含不止一行，對于這些情況，我需要“訊息：”之后的所有行都將成為“訊息：”值的一部分，而不是單獨的行。
請幫我解決這個問題。
日志檔案 (input.log)
Timestamp :2022-11-03 09:42:08.095
User :USER1
Type :warning
Message :Return code : EXCI_NO_ERROR 0
------------------------------------------
Timestamp :2022-11-03 09:42:08.095
User :USER1
Type :warning
Message :Abend code : 1564
------------------------------------------
Timestamp :2022-11-03 09:42:08.095
User :USER1
Type :warning
Message :Buffer received from xxx
line1
line2
line3
line4
------------------------------------------
Timestamp :2022-11-03 09:42:08.095
User :USER1
Type :warning
Message :Return code : EXCI_NO_ERROR 0
------------------------------------------
Timestamp :2022-11-03 09:42:08.095
User :USER1
Type :warning
Message :Abend code : 1564
------------------------------------------
組態檔
input {
file {
path => "/etc/logstash/input.log"
start_position => "beginning"
sincedb_path => "/dev/null"
codec => multiline {
# pattern = This says that any line not starting with a '-----' should be merged with the previous line
pattern => "^-----"
negate => true
what => "previous"
}
}
}
filter {
kv {
#source => "message"
field_split => "\n"
value_split => ":"
}
}
output {
file {
path => "/etc/logstash/output.log"
}
#stdout { codec => rubydebug }
}
uj5u.com熱心網友回復：
Tldr；
退后一步，使用其他過濾器型別。
- 解剖
- 變異
您將在下面找到一種可能的解決方案。
解決方案
- 變異，
----\n從訊息中洗掉 - 洗掉空訊息
- 將訊息分解為必要的欄位
input {
file {
path => "/tmp/input.log"
start_position => "beginning"
sincedb_path => "/dev/null"
mode => "read"
codec => multiline {
# pattern = This says that any line not starting with a '-----' should be merged with the previous line
pattern => "^- $"
negate => true
what => "previous"
}
}
}
filter {
mutate { gsub => ["message", "^- (\n)?", ""] }
if [message] == ""{
drop {}
}
dissect {
mapping => {
"message" => "Timestamp :%{Timestamp}
User :%{User}
Type :%{Type}
Message :%{Message}"
}
}
}
output {
stdout { codec => rubydebug }
}
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/533383.html
標籤：弹性搜索日志存储神交