Update to Debian Bookworm, drop deprecated apt-key usage

[hashworks]

As documented in https://wiki.postgresql.org/wiki/Apt#Manual_Repository_Configuration .

[df7cb]

RUN pip3 install pyroute2>=0.5.17

This should be installed as a Debian package instead, it's version 0.7.2 in bookworm.

[hashworks]

RUN pip3 install pyroute2>=0.5.17

This should be installed as a Debian package instead, it's version 0.7.2 in bookworm.

Added in 0c61f11.

But I see there is a Pipfile... I had to look up what that is. As far as I see the project is dead, should I:
A) Simply remove Pipfile and Pipfile.lock since we expect people to use docker or
B) Convert it to requirements.txt ?

[dimitri]

A Quick Look shows that the PIP file was used to install python packages pyroute2, nose, and psycopg2 all from pypi. I see that we have apt-get coverage for those 3 now in the Dockerfile (with your addition of pyroute2 there), so please see about removing the pipfiles entirely, and maybe also python3-pip from the package list installed with apt-get.

[hashworks]

Test failures seem to be due to the updated pyroute2 package, I'll take a look tomorrow.

ERROR: test_multi_ifdown.test_012_prepare_replication_quorums
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/src/pg_auto_failover/tests/test_multi_ifdown.py", line 207, in test_012_prepare_replication_quorums
    assert node2.set_replication_quorum("false")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/pg_auto_failover/tests/pgautofailover_utils.py", line 1516, in set_replication_quorum
    raise e
  File "/usr/src/pg_auto_failover/tests/pgautofailover_utils.py", line 1506, in set_replication_quorum
    command.execute(
  File "/usr/src/pg_auto_failover/tests/pgautofailover_utils.py", line 2030, in execute
    out, err = self.pgnode.cluster.communicate(proc, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/pg_auto_failover/tests/pgautofailover_utils.py", line 249, in communicate
    return proc.communicate(timeout=1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/nslink/nspopen.py", line 224, in proxy
    return _handle(self.channel_in.get())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/nslink/nspopen.py", line 25, in _handle
    raise result['data']
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 289: invalid start byte

[dimitri]

Test failures seem to be due to the updated pyroute2 package, I'll take a look tomorrow.

My advice would then be to remove that change from this PR, and make another PR that modernizes the installation of Python dependencies for our unit testing.

[hashworks]

My advice would then be to remove that change from this PR, and make another PR that modernizes the installation of Python dependencies for our unit testing.

Moved to #1070.

[hashworks]

ERROR: test_multi_ifdown.test_012_prepare_replication_quorums
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/src/pg_auto_failover/tests/test_multi_ifdown.py", line 207, in test_012_prepare_replication_quorums
    assert node2.set_replication_quorum("false")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
  File "/usr/lib/python3/dist-packages/pyroute2/nslink/nspopen.py", line 224, in proxy
    return _handle(self.channel_in.get())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/pyroute2/nslink/nspopen.py", line 25, in _handle
    raise result['data']
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 289: invalid start byte

I wasn't able to find a fix for this, if anyone has any ideas I would appreciate it.

[dimitri]

On a first look I am not seeing anything related in the pyroute2 changelog at https://docs.pyroute2.org/changelog.html. From https://packages.debian.org/search?keywords=pyroute2 it seems we are upgrading from version 0.5.14 to version 0.7.2. Or it might be a Python 3.x upgrade thing with unicode support?

Also might be the bytes that are reported back from the pg_autoctl command. Did you try manually running the set replication quorum command on a test cluster?

[dimitri]

Still having the same issues when updating python3/pyroute2 (UnicodeDecodeError). Let's go ahead with the release and see about modernizing our CI later?