Improve and mention the Metadata API

[sabahfromlondon]

• Can we include all of the metadata in the registry in an API so that users can automate the process to check: last update, update frequency, last retrieved, available formats etc. to see what has changed with each of the OCDS datasets?

[jpmckinney]

Yes, it's possible. It can just be a URL that returns JSON with all this metadata.

[sabahfromlondon]

Dare I ask... any idea on timeframe? :)

[jpmckinney]

• We should also track metrics for the API (at minimum: datetime, requested path & query string, IP).
• We can set CORS headers as well (not sure if API will be used in such context, but can't hurt).

[yolile]

From the PR:

@jpmckinney Also, I'm not adding this endpoint anywhere in the front end yet, so external users won't be aware of it yet. We can decide where to put it (with Sebastian, maybe) later. (That is why I'm not closing the related issue with this PR yet)

[jpmckinney]

We should also track metrics for the API (at minimum: datetime, requested path & query string, IP).

rg /publications.json /var/log/apache2/other_vhosts_access.log seems fine.

We can set CORS headers as well (not sure if API will be used in such context, but can't hurt).

Done in 5e9045c

---

API is https://data.open-contracting.org/publications.json

Download URLs are derivable from this metadata, e.g. https://data.open-contracting.org/en/publication/{id}/download?name={year}.jsonl.gz where year is determined from date_from and date_to.

I suppose we don't always have the Excel for a given year, but the URL will respond with 404, which seems like a reasonable error for clients to handle. The alternative is to list all downloads in this (or a new) endpoint.

Sample:

  {
    "id": 68,
    "title": "OpenTender",
    "country": "Lithuania",
    "last_retrieved": "2024-10-30",
    "retrieval_frequency": "HALF_YEARLY",
    "update_frequency": "HALF_YEARLY",
    "frozen": false,
    "source_id": "lithuania_digiwhist",
    "source_url": "https://opentender.eu/lt",
    "region": "EU",
    "language": "",
    "date_from": "2006-01-01",
    "date_to": "2024-01-26"
  }