Skip to content

Conversation

@arnaldo2792
Copy link
Contributor

Description of changes:

The first patch of the splits SendReceiveRecord to their own structures. I did this after discussing with @cbgbt that it will be best to have them separated to have a clear separation of concerns and make it harder for developers to construct the wrong type from a Rule.

The second patch of the series fixes the default value used by SendRecord and ReceiveRecord for the max_fds, to match what the dbus-launcher uses. As part of this change, I also fixed how whippet handles wildcards * while translating Rule to Send and Receive records, and replaces those values with empty strings following what the dbus-launcher does.

Testing done:

Prior to this change, systemd-nspawn in systemd 257 failed to create containers as one of the Dbus messages included max_fds = 1 which failed to match the broker condition MESSAGE_max_fds <= RULE_max_fds, since RULE_max_fds was set to 0 for all rules.

After this change, systemd-nspawn runs successfully:

bash-5.1# systemd-analyze --version
systemd 257 (257.7)
-PAM -AUDIT +SELINUX -APPARMOR -IMA +IPE -SMACK +SECCOMP -GCRYPT -GNUTLS -OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP -LIBCRYPTSETUP_PLUGINS +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 -BZIP2 -LZ4 -XZ -ZLIB -ZSTD -BPF_FRAMEWORK -BTF -XKBCOMMON -UTMP -SYSVINIT -LIBARCHIVE
bash-5.1# systemctl status whippet.service
● whippet.service - D-Bus System Message Bus
     Loaded: loaded (/x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/whippet.service; enabled; preset: enabled)
    Drop-In: /x86_64-bottlerocket-linux-gnu/sys-root/usr/lib/systemd/system/service.d
             └─00-aws-config.conf
     Active: active (running) since Thu 2025-11-06 04:54:25 UTC; 18h ago
 Invocation: 6278eaaeb65f404c9dc940b2a226692e
TriggeredBy: ● dbus.socket
   Main PID: 1272 (whippet)
      Tasks: 6 (limit: 18596)
     Memory: 5.5M (peak: 21.8M)
        CPU: 4.372s
     CGroup: /system.slice/whippet.service
             ├─1272 /usr/bin/whippet
             └─1277 /usr/bin/dbus-broker --log 12 --controller 11 --machine-id ec2aaf8da556039d3f042c6c3ce12549 --max-bytes 536870912 --max-fds 4096 --max-matches 16384

Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: POLICY_NAME_CHECK: RECV name=org.freedesktop.systemd1 subject_id=1 subject_seclabel=system_u:system_r:init_t:s0
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: POLICY_NAME_RESULT: RECV name=org.freedesktop.systemd1 subject_id=1 result=rules_found
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: POLICY_RULE_EVAL: RECV subject_id=1 name=org.freedesktop.systemd1 rule_priority=7905747460161236529 priority_match=YES type_match=YES path_match=YES interface_match=YES member_match=YES broadcast_match=YES f…
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: POLICY_RULE_MATCH: RECV name=org.freedesktop.systemd1 path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitRemoved type=4 broadcast=true n_fds=0 rule_path…1236529 verdict=ALLOW
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: POLICY_RULE_EVAL: RECV subject_id=1 name=org.freedesktop.systemd1 rule_priority=2635249153387079026 priority_match=NO type_match=YES path_match=YES interface_match=YES member_match=YES broadcast_match=YES fd…
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: POLICY_FINAL_VERDICT: RECV subject_id=1 interface=org.freedesktop.systemd1.Manager method=UnitRemoved path=/org/freedesktop/systemd1 type=4 broadcast=true n_fds=0 priority=7905747460161236529 verdict=ALLOW
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: [BROKER] Driver dispatch result - peer :1.1, result=0
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: [BROKER] No message available - peer :1.1
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: [BROKER] No message available - peer :1.1
Nov 06 23:29:05 ip-172-31-46-10.us-west-2.compute.internal whippet[1277]: [BROKER] No message available - peer :1.2
Hint: Some lines were ellipsized, use -l to show in full.
bash-5.1# systemd-nspawn -D /var/lib/machines/test-container/ --register=no bash
░ Spawning container test-container on /var/lib/machines/test-container.
░ Press Ctrl-] three times within 1s to kill container.
bash-5.2#

I tested creating a container with its own ENI, and confirmed that all the plumbing to set that up works:

bash-5.1# systemd-nspawn -D /var/lib/machines/test-container/ --register=no --network-interface=eth1 --resolv-conf=off --boot
░ Spawning container test-container on /var/lib/machines/test-container.
░ Press Ctrl-] three times within 1s to kill container.
systemd 252.23-8.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Detected virtualization systemd-nspawn.
Detected architecture x86-64.

Welcome to Amazon Linux 2023.9.20251105!

bpf-lsm: BPF LSM hook not enabled in the kernel, BPF LSM not supported
Queued start job for default target graphical.target.
[  OK  ] Created slice system-getty.slice - Slice /system/getty.
[  OK  ] Created slice system-modprobe.slice - Slice /system/modprobe.
[  OK  ] Created slice user.slice - User and Session Slice.
[  OK  ] Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[  OK  ] Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target local-fs.target - Local File Systems.
[  OK  ] Reached target paths.target - Path Units.
[  OK  ] Reached target remote-fs.target - Remote File Systems.
[  OK  ] Reached target slices.target - Slice Units.
[  OK  ] Reached target swap.target - Swaps.
[  OK  ] Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[  OK  ] Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[  OK  ] Listening on systemd-journald.socket - Journal Socket.
[  OK  ] Listening on systemd-networkd.socket - Network Service Netlink Socket.
[  OK  ] Listening on systemd-userdbd.socket - User Database Manager Socket.
         Mounting dev-hugepages.mount - Huge Pages File System...
         Starting systemd-journald.service - Journal Service...
         Starting systemd-network-generator.service - Generate network units from Kernel command line...
[  OK  ] Mounted dev-hugepages.mount - Huge Pages File System.
[  OK  ] Finished systemd-network-generator.service - Generate network units from Kernel command line.
[  OK  ] Reached target network-pre.target - Preparation for Network.
         Starting systemd-networkd.service - Network Configuration...
[  OK  ] Started systemd-journald.service - Journal Service.
         Starting systemd-journal-flush.service - Flush Journal to Persistent Storage...
[  OK  ] Finished systemd-journal-flush.service - Flush Journal to Persistent Storage.
         Starting systemd-tmpfiles-setup.service - Create Volatile Files and Directories...
[  OK  ] Started systemd-networkd.service - Network Configuration.
[  OK  ] Finished systemd-tmpfiles-setup.service - Create Volatile Files and Directories.
         Starting systemd-resolved.service - Network Name Resolution...
         Starting systemd-update-utmp.service - Record System Boot/Shutdown in UTMP...
[  OK  ] Finished systemd-update-utmp.service - Record System Boot/Shutdown in UTMP.
[  OK  ] Started systemd-resolved.service - Network Name Resolution.
[  OK  ] Reached target network.target - Network.
[  OK  ] Reached target nss-lookup.target - Host and Network Name Lookups.
[  OK  ] Reached target sysinit.target - System Initialization.
[  OK  ] Started systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories.
[  OK  ] Reached target timers.target - Timer Units.
[  OK  ] Listening on dbus.socket - D-Bus System Message Bus Socket.
[  OK  ] Reached target sockets.target - Socket Units.
[  OK  ] Reached target basic.target - Basic System.
         Starting systemd-logind.service - User Login Management...
         Starting systemd-user-sessions.service - Permit User Sessions...
         Starting dbus-broker.service - D-Bus System Message Bus...
[  OK  ] Finished systemd-user-sessions.service - Permit User Sessions.
[  OK  ] Started console-getty.service - Console Getty.
[  OK  ] Reached target getty.target - Login Prompts.
[  OK  ] Started dbus-broker.service - D-Bus System Message Bus.
[  OK  ] Started systemd-logind.service - User Login Management.
[  OK  ] Reached target multi-user.target - Multi-User System.
[  OK  ] Reached target graphical.target - Graphical Interface.
         Starting systemd-update-utmp-runlevel.service - Record Runlevel Change in UTMP...
[  OK  ] Finished systemd-update-utmp-runlevel.service - Record Runlevel Change in UTMP.

Amazon Linux 2023.9.20251105
Kernel 6.1.156 on an x86_64 (-)

test-container login:

I had internet access from within the container:

[root@test-container /]# cat /etc/os-release
NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023.9.20251105"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/"
DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/"
SUPPORT_URL="https://aws.amazon.com/premiumsupport/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
VENDOR_NAME="AWS"
VENDOR_URL="https://aws.amazon.com/"
SUPPORT_END="2029-06-30"
[root@test-container /]# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=5.89 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=5.88 ms
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 5.875/5.883/5.891/0.008 ms
[root@test-container /]#

I tested networkctl can configure the extra ENI in my host:

bash-5.1# networkctl list
IDX LINK    TYPE     OPERATIONAL SETUP
  1 lo      loopback carrier     unmanaged
  2 eth0    ether    routable    configured
  3 docker0 bridge   no-carrier  unmanaged
  6 eth1    ether    routable    configured

4 links listed.
bash-5.1# ping -I eth1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 172.31.40.46 eth1: 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=5.97 ms
^C
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.974/5.974/5.974/0.000 ms
bash-5.1#  resolvectl status eth1
Link 6 (eth1)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 8.8.8.8
       DNS Servers: 8.8.8.8 172.31.0.2
     Default Route: yes
bash-5.1#

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Split Send and Recieve records to their own structs to establish a
clearer separation of concerns while transforming a Rule to either type
of record.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Use the correct default value for the `max_fds` fields of send and
receive records.

Replace wildcard values for send and receive rules with an empty string
as this is what the broker uses for fallback rules.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
@arnaldo2792
Copy link
Contributor Author

Forced push implements the impl_wildcard_getter macro to avoid repetition of similar code

@arnaldo2792 arnaldo2792 merged commit ee7ec8f into bottlerocket-os:develop Nov 11, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants