I ingest data from a web service in JSON that I then clean using JQ. With the cleaned data, I want to create Hive tables, for which I use hive-serde-schema-gen, to create JSON tables. However, since JSON tables do not support schema evolution and Avro does, I wanted to convert JSON to Avro. First, I used the Kite CLI to get the Avro schema from the JSON document. Then, when I try to generate an Avro file using this schema, I get the error: "Unable to parse schema". Can you please tell me what's going on? Thanks!
I've had to upload these files as txt as GitHub doesn't allow me to upload JSON/avsc files.
1995q.json.txt
1995.avsc.txt
I ingest data from a web service in JSON that I then clean using JQ. With the cleaned data, I want to create Hive tables, for which I use hive-serde-schema-gen, to create JSON tables. However, since JSON tables do not support schema evolution and Avro does, I wanted to convert JSON to Avro. First, I used the Kite CLI to get the Avro schema from the JSON document. Then, when I try to generate an Avro file using this schema, I get the error: "Unable to parse schema". Can you please tell me what's going on? Thanks!
I've had to upload these files as txt as GitHub doesn't allow me to upload JSON/avsc files.
1995q.json.txt
1995.avsc.txt