BrAPI extraction improvements

[gcornut]

TODO:

• Describe BrAPI object (identifier, list calls, detail call, links to other objects)
• Flexible BrAPI extraction
  • Check /brapi/v1/calls call for implemented calls
  • Error handling by BrAPI source (if required call not implemented)
• Download BrAPI objects list (ex: /brapi/v1/studies-search) and for each:
  • Download object details (if any) (ex: /brapi/v1/studies/{studyDbId})
  • Download object links (if any) (ex: /brapi/v1/studies/{studyDbId}/germplasm)
• Create reversed links (ex: if study -> germplasm then create germplasm -> study)
• [ ] (Optional) Create BrAPI object index (ID => JSON File, File offset to object)
• Report per data source (WARNINGS & ERRORS) on required BrAPI calls and required fields, required entity links, etc.
• [ ] Basic JSON validation (minimum fields required, minimum links between objects) (moved to elasticsearch transform)

Extracting link from BrAPI object to another:

1. GET object (ex: /brapi/v1/studies/1)

{ "studyDbId": "1", "studyName": "Study 1" }

2. GET linked objects (ex: /brapi/v1/studies/1/germplasm)

{ "germplasmDbId": "1", "germplasmName": "Germplasm 1" },
{ "germplasmDbId": "2", "germplasmName": "Germplasm 2" }

3. Add linked object ids to parent object

{ "studyDbId": "1", "studyName": "Study 1", "germplasmDbIds": ["1", "2"]}

4. Generate reversed links

{ "germplasmDbId": "1", "germplasmName": "Germplasm 1", "studyDbIds": ["1"] },
{ "germplasmDbId": "2", "germplasmName": "Germplasm 2", "studyDbIds": ["1"] }