Clarification Needed on Log Format Suitability for Data Structures

[agrahari-himanshu]

I would like to raise a discussion point regarding one of the statements in our coding standards documentation related to the choice of NDJSON (Newline Delimited JSON) for logging.

The statement in question is:

"Suitable for a lot of data structures in the log, especially if we are logging stack trace."

While I agree that NDJSON is easy to read, parse, and is suitable for streaming and processing large amounts of data, I believe the assertion about it being particularly suitable for logging stack traces and accommodating many data structures needs further clarification.

My Concerns:

Log Format Flexibility:

• Any log format that supports JSON should be able to accommodate complex data structures, including stack traces. This isn't a unique advantage of NDJSON over other formats like regular JSON files or even other structured logging formats.

Stack Trace Logging:

Logging stack traces typically involves multiline strings, which can be challenging to handle in a strictly line-oriented format like NDJSON. Other formats might offer similar or better support for such use cases without the constraint of newline separation.

Proposed Action:

I suggest we review this statement to ensure it accurately reflects the strengths of NDJSON without overstating its capabilities compared to other formats. A more accurate wording could be:

"NDJSON is easy to read and parse, with each line representing a standalone JSON object. It is suitable for streaming and processing large amounts of data and works well with ELK. While NDJSON handles various data structures well, any JSON-based format can accommodate complex logs, including stack traces."