issue with newer version of borg/borgmatic than 1.5?

[dldewey]

hello... trying to run borgreport from a single server against all (5, each with redundancy) repos. Everything works perfectly except for two redundant repos (different locations) created by these versions:
RHEL 9:
borgbackup-1.2.7-1.el9.x86_64
borgmatic-1.8.13-1.el9.noarch

The reports work fine in my other environments with these versions:
Centos 7:
borgmatic-1.1.15-3.el7.x86_64
borgbackup-1.1.18-2.el7.x86_64
RHEL 7:
borgbackup-1.1.18-2.el7.x86_64
borgmatic-1.1.15-3.el7.x86_64
Ubuntu 20.04.6 LTS:
borgmatic 1.5.1
borg 1.1.15

The rest of the reports complete successfully, but prior to outputting the report to the console using the command "borgreport --env-dir /mnt/Backups/borg/.env_files," I get the following errors for the repos created with the new borg/borgmatic, one for each repo/.env file:

Platform: Linux luna 3.10.0-1160.119.1.el7.x86_64 #1 SMP Tue Jun 4 14:43:51 UTC 2024 x86_64
Linux: CentOS Linux 7.9.2009 Core
Borg: 1.1.18 Python: CPython 3.6.8 msgpack: 0.5.6
PID: 12027 CWD: /mnt/Backups/borg/.env_files
sys.argv: ['/usr/bin/borg', '--bypass-lock', 'info', '--last', '1', '--json', '::']
SSH_ORIGINAL_COMMAND: None

• HOSTNAME_OBFUSCATED: Local Exception
  Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4886, in main
  exit_code = archiver.run(args)
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4818, in run
  return set_ec(func(args))
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 176, in wrapper
  return method(self, args, repository=repository, cache=cache_, **kwargs)
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 1551, in do_info
  return self._info_archives(args, repository, manifest, key, cache)
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 1599, in _info_archives
  'archives': output_data,
  File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 2400, in json_print
  print(json_dump(obj))
  File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 2396, in json_dump
  return json.dumps(obj, sort_keys=True, indent=4, cls=BorgJsonEncoder)
  File "/usr/lib64/python3.6/json/init.py", line 238, in dumps
  **kw).encode(obj)
  File "/usr/lib64/python3.6/json/encoder.py", line 201, in encode
  chunks = list(chunks)
  File "/usr/lib64/python3.6/json/encoder.py", line 430, in _iterencode
  yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib64/python3.6/json/encoder.py", line 404, in _iterencode_dict
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 325, in _iterencode_list
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 404, in _iterencode_dict
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 325, in _iterencode_list
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 437, in _iterencode
  o = _default(o)
  File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 2374, in default
  return super().default(o)
  File "/usr/lib64/python3.6/json/encoder.py", line 180, in default
  o.class.name)
  TypeError: Object of type 'bytes' is not JSON serializable

  Platform: Linux luna 3.10.0-1160.119.1.el7.x86_64 issue with newer version of borg/borgmatic than 1.5? #1 SMP Tue Jun 4 14:43:51 UTC 2024 x86_64
  Linux: CentOS Linux 7.9.2009 Core
  Borg: 1.1.18 Python: CPython 3.6.8 msgpack: 0.5.6
  PID: 12033 CWD: /mnt/Backups/borg/.env_files
  sys.argv: ['/usr/bin/borg', '--bypass-lock', 'info', '--last', '1', '--json', '::']
  SSH_ORIGINAL_COMMAND: None
  HOSTNAME_OBFUSCATED: Local Exception
  Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4886, in main
  exit_code = archiver.run(args)
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4818, in run
  return set_ec(func(args))
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 176, in wrapper
  return method(self, args, repository=repository, cache=cache_, **kwargs)
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 1551, in do_info
  return self._info_archives(args, repository, manifest, key, cache)
  File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 1599, in _info_archives
  'archives': output_data,
  File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 2400, in json_print
  print(json_dump(obj))
  File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 2396, in json_dump
  return json.dumps(obj, sort_keys=True, indent=4, cls=BorgJsonEncoder)
  File "/usr/lib64/python3.6/json/init.py", line 238, in dumps
  **kw).encode(obj)
  File "/usr/lib64/python3.6/json/encoder.py", line 201, in encode
  chunks = list(chunks)
  File "/usr/lib64/python3.6/json/encoder.py", line 430, in _iterencode
  yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib64/python3.6/json/encoder.py", line 404, in _iterencode_dict
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 325, in _iterencode_list
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 404, in _iterencode_dict
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 325, in _iterencode_list
  yield from chunks
  File "/usr/lib64/python3.6/json/encoder.py", line 437, in _iterencode
  o = _default(o)
  File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 2374, in default
  return super().default(o)
  File "/usr/lib64/python3.6/json/encoder.py", line 180, in default
  o.class.name)
  TypeError: Object of type 'bytes' is not JSON serializable

[dldewey]

Should have mentioned, running borgreport from the RHEL 9 server with the new borg/borgmatic produces successful output without error.

[bbx0]

Thanks for reporting! Your traces indicate some issue with borg version 1.1.18.

Looks like there is a general issue with the JSON interface in the borg 1.1.x series. I can reproduce the error with the official standalone binaries of borg. (Command: borg info --last 1 --json ::) The last release of borg 1.1.x where --json seems working is 1.1.4. borgreport requires the JSON interface of borg to gather the data for the report. I took a brief look at the 1.1-maint branch, but it does not look like there is a fix for borg 1.1.18.

For the borg 1.2.x series I was not able to reproduce the issue so far.

Since borgreport is read-only, you can try the --borg-binary CLI option (or use the BORGREPORT_BORG_BINARY variable in an *.env file) to point to a newer borg version. I presume borg-linux-glibc228 of 1.4.0 may work for you to generate a report.