CrowdSec Unban all action does not clear nft ruleset

[stephdl]

Steps to reproduce

• Use the 'Unban all' button in CrowdSec web interface.
• Check the nft ruleset with:
  • nft list ruleset | grep xx.xx.xxx.xxx
  • nft list ruleset
• Observe that previously banned IPs are still present in the set, e.g.:

set crowdsec-blacklists-crowdsec {
                type ipv4_addr
                flags timeout
                elements = { 134.199.207.24 timeout 51m56s expires 48m55s381ms, 178.16.52.38 timeout 1h55m54s expires 1h54m3s391ms,
                             193.46.255.159 timeout 3h3m51s expires 3h3m40s382ms }
        }

• Attempt new brute force attacks from different IPs.
• Notice that new attacker IPs are not blocked anymore.

Expected behavior

• All IPs should be removed from the blacklist set after using 'Unban all'.
• New attacker IPs should continue to be blocked by CrowdSec.

Actual behavior

• IPs are not removed from the nft ruleset after performing 'Unban all'.
• New brute force IPs are not blocked until CrowdSec and its firewall are restarted.
• This issue has been reported by multiple users in the Discourse forum.

Components

• NethServer/CrowdSec container bouncer
• nftables ruleset
• CrowdSec web interface

See also

• Discourse forum reports about attackers able to brute force until CrowdSec restart
• The correct command and option to unban (--all) is used, but does not work as expected
• Upstream recommends not running the bouncer in a container, and this setup may be related to the bug
• Decision: Do not report upstream since container usage is not supported/recommended

[stephdl]

QA

target is 1.0.17-dev.4

do an upgrade from testing, and test it also from a direct install, the only mandatory things is to see the ban from CAPI, you can see them by cscli decisions list --all or nft list ruleset

• test you can cancel and reopen the modal of a single ban or unban all (once the modal is visible, click on cancel and try to reopen the modal)
• test you can ban an IP
• try to unban_all feature, the ban from capi must still be here