tests/gnrc_sock_dns: port to scapy

[miri64]

Contribution description

This

• ports the existing tests to a shell based approach,
• that then also is used by a scapy version of the test, and
• adds failure tests to prevent a regression to before sock_dns: Security issues (including remote code execution) #10739.

Testing procedure

Build and flash the application to a board of your choice (must support ethos or netdev_tap). Create a TAP interface and run

make test

The tests should succeed.

Issues/PRs references

Adds tests for #10739
Requires #10896 (merged) for tests to finish successfully.

[miri64]

I mentioned this in the commit that does this change: this bind causes (at least my version of dnsmasq) to send RAs from that address. However, sending RAs from a non-link-local address is invalid.

[miri64]

Rebased to current master to include fixes from #10896.

[aabadie]

sudo make test

I have to say that I don't like this. The files generated by the build (if the application was not built already) will be owned by root and it will create a mess with file permissions in the riot directory. For me this is also a big problem with automated tests but also for developers without root access on their machine. The same issue is already in master with 2 other gnrc test applications.
The fact that it passes on Murdock is probably because the tests are run within a container environment, using a root user. This doesn't reflect the way a developer is supposed to use the build system to test something.
Running tests as root could cause many troubles, just imagine a line that just do os.system('rm -rf /').

If there is a way to use scapy without the need to be root, that would be more useful and it should also be documented.

The purpose of tests is to be runnable by any developer, without strong knowledge of all the corner cases and setup details. It should also be easy to understand and help understand the code being tested.

[miri64]

The files generated by the build

make test isn't supposed to build just execute a test script.

For the most part I agree with you (the tests are currently not run on Murdock btw for exactly that reason). Sadly, there is no way to run without root, since the scapy tests

• needs to be able to send Ethernet or at least raw IPv6 packets
• need to be able to sniff the tap interface (even wireshark also effectively needs root permissions for that).

To execute them without sudo, scapy (and thus python) would require certain permissions, like e. g. wireshark gets when the user is added to the wireshark group. Since we are talking raw networking capabilities for the python executionable here I rather like to not have that on a system like Murdock (unless we can ensure better isolation there).

[edit: darn mobile interface]

[miri64]

If and only if I find a way to send scapy-cooked packets over UDP sockets, this test at least could be ported to UDP sockets, since it is testing DNS, an application layer protocol. I will try that. However, this is not possible with the other two tests (gnrc_ipv6_ext and gnrc_rpl_srh), since they test parts of the IPv6 implementation and are thus required to at least be able to send hand-cooked IPv6 datagrams.

[miri64]

The purpose of tests is to be runnable by any developer, without strong knowledge of all the corner cases and setup details. It should also be easy to understand and help understand the code being tested.

Apart for developers without administrative access to the machine scapy allows for exactly that. As a bonus, the script even provides you a bold red warning if you try to run it without root permissions ;)

[miri64]

scapy allows for exactly that.

To elaborate on that: the dependency-less alternative to scapy would be either the struct module or raw byte strings + sockets but I think those solutions are neither readable nor maintainable for the uninformed or informed developer ;)

[miri64]

If and only if I find a way to send scapy-cooked packets over UDP sockets, this test at least could be ported to UDP sockets, since it is testing DNS, an application layer protocol. I will try that. However, this is not possible with the other two tests (gnrc_ipv6_ext and gnrc_rpl_srh), since they test parts of the IPv6 implementation and are thus required to at least be able to send hand-cooked IPv6 datagrams.

Worked. The tests are now even simpler than with pure scapy :-).

[miri64]

They still require a TAP interface though, so no way to run them with Murdock at the moment :(

[MrKevinWeiss]

I looked at the changes. I really like the moving to shell tests and implementing logic in python. The fact that @aabadie comments regarding sudo have been addressed means it doesn't cost anything.

Though I am not a networking guy I was able to run the tests and they seem much more in depth than the test that is replaced.

I would be ok to move this forward if nobody else... @kaspar030... has an opinion as it appears to be an improvement to me.

[MrKevinWeiss]

Also rebase seems needed.

[miri64]

Rebased and adapted for master

[MrKevinWeiss]

Well since no response, I trust @miri64 that is testing the correct thing.

Please squash!

[miri64]

Squashed

[MrKevinWeiss]

ACK. We can wait for the CI.