我已經安裝了 elasticsearch-8.2.3 logstash-8.2.3 和 kibana-8.2.3 我已經配置了 logstash conf 檔案以將內容攝取到 elasticsearch 中，logstash 運行沒有任何錯誤，但它沒有攝取內容。
下面是conf檔案：
input {
#stdin {type => "stdin-type" }
file
{
path => "D:/logstash-8.2.3/inspec/*.*"
type => "file"
start_position=>"beginning"
sincedb_path => "NUL"
ignore_older => 0
}
}
filter {
csv
{
columns =>
[
"itemid","itemtitle","rlabel","ayear","rid","rsid","anotatedby","anotatetime","antype","astate","broaderlevel3","broaderlevel2","broaderlevel1","categorylabel","toppreferedlabel"
]
separator => ","
remove_field => ["type","host"]
}
mutate
{
split => { "antype" => ";" }
split => { "broaderlevel3" => ";" }
split => { "broaderlevel2" => ";" }
split => { "broaderlevel1" => ";" }
split => { "categorylabel" => ";" }
split => { "toppreferedlabel" => ";" }
}
}
output {
stdout { }
elasticsearch
{
hosts => ["localhost"]
index => "iet-tv"
}
}
我在運行 logstash 時沒有收到任何錯誤訊息，但內容沒有被攝取到 Elasticsearch 中。
以下是日志：
[2022-06-29T14:03:03,579][INFO ][logstash.runner ] Log4j configuration path used is: D:\logstash-8.2.3\config\log4j2.properties
[2022-06-29T14:03:03,595][WARN ][logstash.runner ] The use of JAVA_HOME has been deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with each specific version of Logstash, and generally provides best performance and reliability. If you have compelling reasons for using your own JDK (organizational-specific compliance requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2022-06-29T14:03:03,598][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.2.3", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.15 10 on 11.0.15 10 indy jit [mswin32-x86_64]"}
[2022-06-29T14:03:03,600][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX: UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX: UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX: HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-06-29T14:03:03,736][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-06-29T14:03:11,340][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-06-29T14:03:12,628][INFO ][org.reflections.Reflections] Reflections took 153 ms to scan 1 urls, producing 120 keys and 395 values
[2022-06-29T14:03:15,580][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-06-29T14:03:15,662][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost"]}
[2022-06-29T14:03:16,210][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2022-06-29T14:03:16,532][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2022-06-29T14:03:16,549][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.2.3) {:es_version=>8}
[2022-06-29T14:03:16,553][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-06-29T14:03:16,627][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-06-29T14:03:16,627][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-06-29T14:03:16,632][WARN ][logstash.outputs.elasticsearch][main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-06-29T14:03:16,652][INFO ][logstash.filters.csv ][main] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2022-06-29T14:03:16,694][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2022-06-29T14:03:16,762][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["D:/logstash-8.2.3/conf/inspec.conf"], :thread=>"#<Thread:0x48e38277 run>"}
[2022-06-29T14:03:18,017][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.25}
[2022-06-29T14:03:18,102][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-06-29T14:03:18,171][INFO ][filewatch.observingtail ][main][2c845ee5978dc5ed1bf8d0f617965d2013df9d31461210f0e7c2b799e02f6bb8] START, creating Discoverer, Watch with file and sincedb collections
[2022-06-29T14:03:18,220][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
任何建議都非常感謝。
謝謝
達門德拉·庫馬爾·辛格
uj5u.com熱心網友回復：
在 filebeat 中，ignore_older => 0關閉基于年齡的過濾。在 logstash 檔案輸入中，它告訴過濾器忽略任何超過零秒的檔案，并且由于檔案輸入在其定期輪詢新檔案之間休眠，這可能意味著它忽略所有檔案，即使它們正在更新。
轉載請註明出處，本文鏈接：https://www.uj5u.com/ruanti/498384.html