We currently give datagov-catalog read-only access to our database by connecting the a read-replica of the Postgres database. We should do a similar thing for our OpenSearch index by giving the OpenSearchInterface class in datagov-catalog just the read-only, search-related functionality. Then, all OpenSearch updating can happen in datagov-harvester with an OpenSearchInterface that is read-write and focused on syncing rather than search.
Expected behavior
The datagov-catalog app can't make any changes in the OpenSearch index
Actual behavior
datagov-catalog has methods like delete_all_datasets and index_datasets.
Sketch
The opensearch.py code in the two apps should be split-up along the lines given above with search-related functionality in datagov-catalog and sync-related functionality in datagov-harvester. Then we will need to move the Github Actions for Opensearch sync from datagov-catalog to datagov-harvester.
We currently give datagov-catalog read-only access to our database by connecting the a read-replica of the Postgres database. We should do a similar thing for our OpenSearch index by giving the
OpenSearchInterfaceclass in datagov-catalog just the read-only, search-related functionality. Then, all OpenSearch updating can happen in datagov-harvester with anOpenSearchInterfacethat is read-write and focused on syncing rather than search.Expected behavior
The datagov-catalog app can't make any changes in the OpenSearch index
Actual behavior
datagov-catalog has methods like
delete_all_datasetsandindex_datasets.Sketch
The
opensearch.pycode in the two apps should be split-up along the lines given above with search-related functionality in datagov-catalog and sync-related functionality in datagov-harvester. Then we will need to move the Github Actions for Opensearch sync from datagov-catalog to datagov-harvester.