Structured Data

Processing Structured Data (JSON) in Metatab.

How to transform structured JSON into flat CSV?

The most flexible way would be to have an ingestion schema that has a JSON path for each column. The Row Generator uses JSONPath to generate 'rows' that are JSON dict or other structures, then the table Schema uses JSONPath, or just name matching, to extract fields.

Structured Rows

In some cases, the data stream is already in rows, but the rows have some implicit structure, such as a set of columns that represent a nested dict or lists. For instance, a 'Student' record may have columns for each of the students courses, names 'Course1', 'Course2', etc.

For these cases, the system should be able to output rows that consist of a key and individual subsets from a single row, then moving on to the next row. So, the output might be:

    Student1, Course1
    Student1, Course2
    Student1, Course3
    Student2, Course1
    Student2, Course2
    Student2, Course3

This could be specified in the Table term as:

    Table: tablename
    Table.KeyValues: StudentId
    Table.RowValues: ( Course1, Course2, Course3)

Or, if more than one value should be emitted for each row:

    Table: tablename
    Table.KeyValues: StudentId
    Table.RowValues: ( (Course1, Desc1), (Course2,Desc2), (Course3,Desc3) )