Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON array serialization format to S3 output plugin #9888

Open
vcschapp opened this issue Jan 29, 2025 · 0 comments
Open

Add JSON array serialization format to S3 output plugin #9888

vcschapp opened this issue Jan 29, 2025 · 0 comments

Comments

@vcschapp
Copy link

vcschapp commented Jan 29, 2025

Is your feature request related to a problem? Please describe.

I need to deliver log events to Amazon S3 as a JSON array rather than newline-delimited JSON.

Describe the solution you'd like

  1. Add a new format configuration parameter to the Amazon S3 output plugin with supported values: jsonl (default) and json_array.
  2. The default /jsonl behavior continues to be writing newline-delimited JSON.
  3. In the json_array variant, the plugin writes a leading [ as the first character of the object, uses , as the delimiter instead of newline, and writes a final ] when the object is finished.

Describe alternatives you've considered

I need my objects to contain JSON arrays for compatibility with an existing web service I do not control. My only options are:

  1. Don't use FB.
  2. Write a new custom plugin specifically for my use case that does what I want.
  3. Make the existing Amazon S3 plugin more versatile.

Additional context

  • I am trying to send usage logs to an existing web service whose "public interface" is an S3 bucket that MUST receive objects containing JSON arrays.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant