Skip to content

filter

Jump to bottom
Jesús Daniel Colmenares Oviedo edited this page Nov 13, 2025 · 7 revisions

Filtering network traffic

The principle of least privilege can be defined as “A security principle that a system should restrict the access privileges of users (or processes acting on behalf of users) to the minimum necessary to accomplish assigned tasks.”, and in the context of FreeBSD jails, this is where it really shines. We provide access only to the devices that a jail needs to work properly, isolate processes, isolate the network stack, restrict access to mount points, and much more using FreeBSD jails; however, it's still necessary to isolate the network traffic that a jail can access.

Suppose you use a FreeBSD jail to run a web server with the most restricted user you have ever encountered, and an attacker has gained access to your isolated environment using an exploit against your web server. Even if the attacker cannot escalate privileges to the root account, it discover that, from that jail, access to your router is allowed...

Note: This document is intended for those who use Virtual Networks.

Hooks

A not so well-known feature of AppJail is hooks. Hooks are executed before or after an AppJail subcommand is executed, and can be either a script or a binary program; they just need to be in the pre.d or post.d subdirectory, depending on when they should be executed. pre.d or post.d are subdirectories of the directory specified by HOOKSDIR (see appjail.conf(5)). Hooks must have execute permission. When hooks are executed, the arguments used to run AppJail are shared with them.

There is a directory with examples, but we only need two for this document:

We'll describe each hook in more detail in its own section. For now, let's install the hooks and configure AppJail to use them.

mkdir -p /usr/local/etc/appjail/hooks/pre.d
cp -a /usr/local/share/examples/appjail/hooks/pre.d/security-group.sh \
    /usr/local/etc/appjail/hooks/pre.d
cp -a /usr/local/share/examples/appjail/hooks/pre.d/security-table.sh \
    /usr/local/etc/appjail/hooks/pre.d

/usr/local/etc/appjail/appjail.conf:

HOOKSDIR=/usr/local/etc/appjail/hooks

PF Configuration

The AppJail Handbook only mentions a few anchors, necessaries for appjail-nat(1) and appjail-expose(1), but we need many more components to restrict access and provide a fine-grainer control.

Before proceeding, consider your threat model, or at least how much access should be restricted and where. For example, my "policy" is that I want to restrict communication between jails, but they must at least have Internet access to install applications and should not have access to other hosts on my network.

/etc/pf.conf:

#
# The following macros should correspond to ```appjail network list```:
#
appjail_bridge = "ajnet"
appjail_network = "10.0.0.0/10"

#
# See https://www.rfc-editor.org/rfc/rfc6890.html
#
table <rfc6890> const { 0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16          \
                        172.16.0.0/12 192.0.0.0/24 192.0.0.0/29 192.0.2.0/24 192.88.99.0/24    \
                        192.168.0.0/16 198.18.0.0/15 198.51.100.0/24 203.0.113.0/24            \
                        240.0.0.0/4 255.255.255.255/32 }

#
# Options that may or may not make sense for you.
#
set skip on lo
set block-policy return
set fail-policy return
#
# I use pflog0 when debugging, but as you see below, the ```log``` option is not
# used in any rule.
#
set loginterface pflog0

scrub in

#
# AppJail anchors used by ```appjail-nat(1)``` and ```appjail-expose(1)```.
#
nat-anchor "appjail-nat/jail/*"
nat-anchor "appjail-nat/network/*"
rdr-anchor "appjail-rdr/*"

antispoof for lo0

#
# Strictest environments may block all traffic, but this further complicates
# our ruleset and does not meet my needs or, probably, those of most people.
#
block in
pass out
pass in proto { icmp, icmp6 }

#
# This is not necessary, since in the previous rule we are already blocking all
# incoming traffic on all interfaces, but it is useful to have the following rule,
# since we can copy-paste it into other hosts that do not block all incoming packets,
# but want to block all incoming packets for jails.
#
block in on $appjail_bridge from <rfc6890>
#
# This is crucial because with this rule we allow Internet access, but not to other
# hosts on the same network as the host.
#
pass in on $appjail_bridge from $appjail_network to ! <rfc6890>

#
# The anchor used by ```security-group```.
#
anchor "appjail-filter/jail/*"

The above is the most generic pf.conf(5) file I can recommend, but of course, adapt it to your needs.

/etc/sysctl.conf:

net.link.bridge.pfil_member=1
net.link.bridge.pfil_bridge=1

To filter traffic for if_bridge(4) bridges and its members, we need the above tunables. Of course, configure them at runtime as well:

sysctl net.link.bridge.pfil_member=1
sysctl net.link.bridge.pfil_bridge=1

The above tunables may not be modified unless you have already loaded if_bridge(4) before the sysctl's rc script is executed, so you must include the following in your loader.conf(5):

if_bridge_load="YES"
bridgestp_load="YES"

Security Groups

Security groups are a hook that can dynamically add and remove rules to the appjail-filter/jail/* anchor using labels defined in the jail. Rules are loaded before the jail starts and removed before it stops.

Let's create two jails to test our rule using nc(1):

appjail quick nc-server \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> default" \
    nat \
    label="security-group:1" \
    label="security-group.rules.allow-p8484:pass on appjail_epair proto tcp to %i port 8484"
appjail quick nc-client \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> default" \
    nat

If we inspect the anchor, we can see that our rule is already loaded:

# pfctl -a 'appjail-filter/jail/nc-server' -sr
pass on appjail_epair inet proto tcp from any to 10.0.0.5 port = 8484 flags S/SA keep state

Run netcat as a server in the nc-server jail in another terminal and netcat as a client in the nc-client jail to connect to the nc-server jail.

Server:

# appjail cmd jexec nc-server nc -l 8484
Hello, world!

Client:

# appjail cmd jexec nc-client nc -v 10.0.0.5 8484
Connection to 10.0.0.5 8484 port [tcp/*] succeeded!
Hello, world!

We are allowing all jails to connect to nc-server using port 8484, but we can restrict access much more if we know the jail's IPv4 address in advance, so for this use case it's necessary to use a fixed IPv4 address.

Server:

# appjail quick nc-server \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> address:10.0.0.43 default" \
    nat
...
# appjail cmd jexec nc-server nc -l 8484
Hello, world!

Client:

# appjail quick nc-client \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> default" \
    nat \
    label="security-group:1" \
    label="security-group.rules.connect-to-nc-server:pass on appjail_epair proto tcp from %i to 10.0.0.43/32 port 8484"
...
# appjail cmd jexec nc-client nc -v 10.0.0.43 8484
Connection to 10.0.0.43 8484 port [tcp/*] succeeded!
Hello, world!

Remember that jails have restricted communication to private hosts, but sometimes it may be necessary to create a jail with access to a host on our network. Let's create a jail to access the router's admin panel:

appjail quick w3m \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> default" \
    pkg=w3m \
    copydir=/ \
    file=/usr/local/etc/pkg/repos/Latest.conf \
    nat \
    label="security-group:1" \
    label="security-group.rules.connect-to-router:pass on ajnet proto tcp from %i to 192.168.0.1 port 80"

Security Tables

Security groups provide a simple way to add rules dynamically, but sometimes it is desirable to keep our ruleset as small as possible by using common patterns. In this way, we can combine a rule that matches one or more jails using pf(4) tables, which is the motivation for the Security tables hook, to dynamically add and remove jails to specific tables.

For example, following the example in the previous section, we can further restrict our jails to be private jails, meaning that they must not communicate with the Internet.

Server:

# appjail quick nc-server \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> address:10.0.0.43 default" \
    nat \
    label="security-group:1" \
    label="security-group.tables.t0:private-jails"
...
# appjail cmd jexec nc-server nc -l 8484
Hello!

Client:

# appjail quick nc-client \
    start \
    overwrite=force \
    ephemeral \
    virtualnet=":<random> default" \
    nat \
    label="security-group:1" \
    label="security-group.rules.connect-to-nc-server:pass on appjail_epair proto tcp from %i to 10.0.0.43/32 port 8484" \
    label="security-group.tables.t0:private-jails"
# appjail cmd jexec nc-client nc -v 10.0.0.43 8484
Connection to 10.0.0.43 8484 port [tcp/*] succeeded!
Hello!

Table:

# pfctl -T show -t private-jails
   10.0.0.5
   10.0.0.43

Of course, we need to modify our pf.conf(5) a bit for the above example to work properly:

table <private-jails> persist
...
block in on $appjail_bridge from <private-jails> to ! <rfc6890>

Notes

  1. For each Virtual Network in which the jail is located, %i is replaced with the jail's IPv4 address. However, if the jail is not located in any Virtual Network, the security-group hook will add the rule to the anchor without replacing %i, and in the case of security-table hook, no additional processing will be performed.
  2. security-group.include-only, whose default value is .+, is a regular expression to include only IPv4 addresses that match this regular expression.
  3. You can specify more than one rule or table; just prefix each label with security-group.{rules|tables}.<string>, where <string> is an arbitrary string that must be a valid label for AppJail.

Clone this wiki locally