ArgoCD deployment of eoAPI in develop cluster failing (Postgres)

[j08lue]

In a recent ArgoCD sync of data-access-core, the pgstac-migrate step failed repeatedly, with the below traceback:

https://argocd.develop.eoepca.org/applications/argocd/data-access-core?view=tree&resource=&node=%2FPod%2Fdata-access%2Fpgstac-migrate-2dx6d%2F0&tab=summary

Waiting for database to be ready...
Running PgSTAC migrations...
Traceback (most recent call last):
  File "/usr/local/bin/pypgstac", line 10, in <module>
    sys.exit(cli())
  File "/opt/src/pypgstac/src/pypgstac/pypgstac.py", line 125, in cli
    fire.Fire(PgstacCLI)
  File "/usr/local/lib/python3.9/dist-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/usr/local/lib/python3.9/dist-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/usr/local/lib/python3.9/dist-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/opt/src/pypgstac/src/pypgstac/pypgstac.py", line 61, in migrate
    return migrator.run_migration(toversion=toversion)
  File "/opt/src/pypgstac/src/pypgstac/migrate.py", line 157, in run_migration
    cur.execute(migration_sql)
  File "/usr/local/lib/python3.9/dist-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
psycopg.errors.InsufficientPrivilege: permission denied to create role
DETAIL:  Only roles with the CREATEROLE attribute may create roles.
CONTEXT:  SQL statement "CREATE ROLE pgstac_admin"
PL/pgSQL function inline_code_block line 5 at SQL statement

The manifests are here: https://github.com/EOEPCA/eoepca-plus/tree/deploy-develop/argocd/eoepca/data-access

What is the root cause and how can we fix this?

[emmanuelmathot]

Is the values used this one? https://github.com/EOEPCA/eoepca-plus/blob/deploy-develop/argocd/eoepca/data-access/parts/values/values-eoapi.yaml

[j08lue]

Here is the manifest for data-access-core: https://github.com/EOEPCA/eoepca-plus/blob/deploy-develop/argocd/eoepca/data-access/parts/app-data-access-core.yaml

[j08lue]

So, afaik, the issue was that eoapi-k8s 0.7.0 requires a dedicated role for pgstac administration (to limit the use of master admin). That role was not present in 0.6.0 and we are missing a migration step that creates it.

We will add this migration support, since there might be more projects that will migrate from 0.6.0 to 0.7.0.

[emmanuelmathot]

issue addressed in this PR: developmentseed/eoapi-k8s#228
released 0.7.1 planned as soon as the PR is approved

[pantierra]

Let's go: EOEPCA/eoepca-plus#63

[pantierra]

We are having another problem, it doesn't seem to pick up the migration logic here. Probably because of ArgoCD not handling upgrades ( with .Release.IsUpgrade) but rather applying the charts idempotently. Some context: argoproj/argo-cd#7496

[pantierra]

initidb seems to be still idempotent anyway. @emmanuelmathot what about we just run it without checking for version or whether it is an upgrade?

[j08lue]

Has this been resolved, @pantierra?

[emmanuelmathot]

This should be solved with version 0.7.3 of eoapi-k8s

[j08lue]

Upgrade happened in EOEPCA/eoepca-plus@79912d5 I see. 👌