I am running docker containers on EC2 backed ECS. On each ec2 machine I have a fluentbit container running to send all logs to elasticsearch. I am using aws-for-fluentbit image.
This all works fine and the indexes in elasticsearch are log-current-date
, with a new index per day.
However, ideally I would like some ecs service's logs to go to a different index, for example logs-ecs-service-name-current-date
My understanding is I would need to add a filter, tag each and then two outputs.
But I'm not sure what I am filtering on?!
[SERVICE]
Flush 5
Daemon Off
Log_Level INFO
Parsers_File /fluent-bit/etc/parsers.conf
[INPUT]
Name Forward
Port 24224
Buffer_Max_Size 256KB
[FILTER]
Name parser
Parser docker
Match *
Key_Name log
Reserve_Data On
Preserve_Key On
[OUTPUT]
Name es
Match *
Buffer_Size 128KB
Host [redacted]
Port [redacted]
Logstash_Format On
Logstash_Prefix logs-
Include_Tag_Key On
Tag_Key image
Type _doc