gcoap: Add file server

[benpicco]

Contribution description

This is a continuation of @chrysn's GCoAP fileserver implementation with only minimal changes.
Files are served block-wise, directory listings are now too.

My use case is to provide a CoAP fileserver to serve SUIT update files from SD card.

Testing procedure

Instead of extending the already cluttered examples/filesystem, I added a new examples/gcoap_fileserver that leverages vfs_default and is thus much cleaner.

To get some data on the (virtual) file system you can use #17937 to download some files from aiocoap-fileserver, then if you are on native copy MEMORY.bin to examples/gcoap_fileserver.

You should now be able to request the files via aiocoap-client.

main(): This is RIOT! (Version: 2022.07-devel-107-ge39b5-coapfileserver)
> ls /nvm0
ls /nvm0
./
../
LICENSE	24310 B
hello.txt	15 B
total 2 files

benpicco@T430s ~/dev/RIOT (git)-[coapfileserver] % aiocoap-client coap://[fe80::748f:e6ff:fed7:426d%tapbr0]/vfs/
</vfs/LICENSE>,</vfs/hello.txt>
benpicco@T430s ~/dev/RIOT (git)-[coapfileserver] % aiocoap-client coap://[fe80::748f:e6ff:fed7:426d%tapbr0]/vfs/hello.txt
Hello from RIOT
benpicco@T430s ~/dev/RIOT (git)-[coapfileserver] % aiocoap-client coap://[fe80::748f:e6ff:fed7:426d%tapbr0]/vfs/LICENSE > /tmp/LICENSE
benpicco@T430s ~/dev/RIOT (git)-[coapfileserver] % md5sum LICENSE /tmp/LICENSE 
a4ce26a8dcc018faa99a6cd08cbcd1c3  LICENSE
a4ce26a8dcc018faa99a6cd08cbcd1c3  /tmp/LICENSE

Issues/PRs references

taken over from #14397

[fabian18]

some feedback

[fabian18]

some more feedback

[fjmolinas]

My use case is to provide a CoAP fileserver to serve SUIT update files from SD card.

Nice! I had the same usecase!

[chrysn]

Tbh I didn't fully wrap my head around the whole etag thing and I don't quite see how useful it is, so it's high on my culling list.

The ETag is what stands between files being modified and a client silently taking a frankenfile for the response.

[fabian18]

Is there already a WIP branch which implements file/directory creation?
The usecase for #17962 is to make an already existing file available as a CoAP resource right?

[benpicco]

Is there already a WIP branch which implements file/directory creation?

No, my use case was just serving firmware updates, so I don't have any plans to implement this right now.

The usecase for #17962 is to make an already existing file available as a CoAP resource right?

That's to upload a file to a CoAP server via PUT, the CoAP server does not have to run RIOT (e.g. could be aiocoap-fileserver).

[fabian18]

Ok, I want to transfer a file to the VFS using coap. So I guess my use case is not covered so far. But no problem, I think I am going to try to add this.

[benpicco]

I want to transfer a file to the VFS using coap.

You can use #17937 for that

[fabian18]

Thanks for the hint. This also kind of solves the problem but I would prefer to push the files to the nodes.

[fabian18]

Just another round of testing.

• blockwise transfer works
• file retrieval works
• directory retrieval works
• Etag validation works

> vfs ls /nvm0
2022-05-21 13:00:29,497 # vfs ls /nvm0
2022-05-21 13:00:29,504 # ./
2022-05-21 13:00:29,504 # ../
2022-05-21 13:00:29,526 # anotherdir/
2022-05-21 13:00:29,553 # cacert.der    608 B
2022-05-21 13:00:29,555 # total 1 files

$ coap-client -m get -a fe80::3478:9157:9fd5:81c3%eth0 coap://[fe80::4c49:6fff:fe54:0]/vfs/cacert.der
$ coap-client -m get -O 4,0x60829555  -a fe80::3478:9157:9fd5:81c3%eth0 coap://[fe80::4c49:6fff:fe54:0]/vfs/cacert.der
$ coap-client -m get -O 23,0x00 -a fe80::3478:9157:9fd5:81c3%eth0 coap://[fe80::4c49:6fff:fe54:0]/vfs/

Note: tested with libcoap example client and additional module CFLAGS += -DCONFIG_GNRC_IPV6_NIB_ARSM=1 because I simply tested with link local addresses over ethernet.

I think it can go in if murdock becomes green.

[fabian18]

cpu/avr8_common/avr_libc_extra/include/vendor/sys/stat.h does not #define st_atime st_atim.tv_sec and we should better memset() the whole struct timespec

Suggested change

	memset(&stat.st_atime, 0, sizeof(stat.st_atime));
	memset(&stat.st_atim, 0, sizeof(stat.st_atim));

[benpicco]

uh now esp* and pic32 don't know .st_atim but only .st_atime 😕

[maribu]

Why not strncpy, since path_len already contains the current length of path?

Beware: I think that strncpy and strncat have a stupid API, as they may leave the destination path without terminating zero byte if the destination buffer is not large enough.

[benpicco]

Do you mean

    strncpy(path + path_len, name, COAPFILESERVER_PATH_MAX - path_len - 1);

That looks a bit more fragile to me, but might be more efficient.

[fabian18]

strncpy(path + path_len + 1, name, COAPFILESERVER_PATH_MAX - path_len - 2);

+1 because of '/' and -2 also for the '/' and a '\0'.

[benpicco]

I think you are proving my point 😅

[fabian18]

I am fine with both. @maribu, choose readability over efficiency, or not ?

[maribu]

I just checked: strncat indeed does always zero-terminate the string, even if it has to truncate. That makes it safer to use than strncpy, so I guess strncat has won ;)

[fabian18]

ACK

[benpicco]

Thank you for the review!