fs: Filesystem module (WIP) by x3ro · Pull Request #2474 · RIOT-OS/RIOT

x3ro · 2015-02-19T19:35:40Z

Filesystem module

This is a continuation of @authmillenon's PR #1265, which she agreed to delegate to me since it seems like I'll be working on the topic of FS for RIOT in my masters thesis :)

Summary of the discussion so far

@authmillenon's implementation proposal inspired by FUSE
errno should not be used since its no thread-safe/reentrant. The functions should just return negated error codes instead.
@Kijewski said that "we should not target to implement all these functions. We don't want to become a less well-tested Linux."
Filesystems should be added as packages
- CFS (used by Contiki, BSD license)
- Coffee, also by Contiki, which is meant to be used with Flash memory
- FatFS (custom license compatible to BSD)
  - @gebart: "FAT would be useful for SD card storage, which provide internal wear levelling, but it would probably wear out simpler flash chips too quickly to be useful on those."
CFS is closely integrated into the Contiki repo. How do we add it as a package?
- @gebart: "Can we add it as a pkg with some script to fetch a git revision and strip everything except the CFS pieces?"
- @OlegHahm: "Maybe we should reach out to the Contiki people and see if we can maintain CFS as an external library together on the long run."
File descriptors
- stdin, stdout and stderr are specified as 0, 1, 2 respectively

@authmillenon: Some thought on file descriptors I had I'd like to share: Maybe any value of kernel_pid_t with > MAXTHREADS is the best solution, since in POSIX even pthreads have fds and this way we can just have them be the actual PID, while anything else is uniquely identified. This way we have a unique numbering scheme and have the POSIX wrapper for pthreads and file operations as easy as possible. Sockets would also grab from the same pool then (and we have the option to run every socket in it's own light-weight thread (if we'll have something like that) in the future, maybe).

ToDo

Determine which FS operations RIOT needs and which one FS implementations require
Figure out what state the SDHC support on the MSB-A2 is in. Hasn't been tested in a while according to @OlegHahm
Verify that the code to be used has is license-compatible (possible with the help of http://wiki.osdev.org/Licensing). We should be allowed to pull BSD sources into LPGL.
Figure out how to integrate CFS/Coffee
Fully understand @authmillenon's thoughts on file descriptors
Should setting errno be completely removed from the code?

This seems to be stemming from miri64/RIOT@6f9b03e569ba where an initial SCFS module is/was being developed.

Otherwise it will be undefined for the rest of the function

Perhaps @authmillenon can help? :)

This will be a FS that accesses the filesystem of the underlying host in RIOTs native mode

x3ro · 2015-02-19T19:42:48Z

sys/include/fs.h

Does fs_unmount actually need both mount_point and device? Wouldn't just the mount_point suffice, as the device is contained in fs_table_t?

If you mount a device on multiple locations you'll need to know which one to unmount. I do not know if the implementation is supposed to support that use case, however.

I guess my original idea was, that you can choose if you want to select the device or the mount point as your identifier for unmounting (similar to the umount command). But yeah, I guess you can implement this in other ways and don't have to hardcode it into the API.

benpicco · 2015-03-05T16:54:25Z

spiffs is another embedded flash file system with wear levelling you might consider.

OlegHahm · 2016-02-26T16:20:58Z

@x3ro, what's the state?

x3ro · 2016-02-26T16:26:26Z

@OlegHahm This can be closed, as the FS module envisioned here won't be implemented (at least by me :D), as discussed on the mailing list. I'm currently finishing up my thesis (due end of march) and will work on a mergeable pull request afterwards.

OlegHahm · 2016-02-26T16:40:11Z

Close as memo then.

OlegHahm · 2016-04-13T10:01:00Z

Maybe we can use https://github.com/yarrick/pingfs instead? Thanks, @mfrey for the pointer. ;-)

x3ro · 2016-04-13T10:17:51Z

👍

miri64 · 2016-04-13T14:31:44Z

a specific file system implementation instead of a general API to a file system... sounds legit :P

kYc0o · 2016-04-13T14:52:55Z

Do you think that for large file is it useful? IMO more data -> more transmissions (ping) -> more power...

miri64 · 2016-04-13T14:57:03Z

@kYc0o I think that is more of a fun project, than something that is to be considered for production context ;-)

kYc0o · 2016-04-13T15:03:23Z

Yeah... I didn't feel the sarcasm on the comments :-P

jnohlgard · 2016-07-08T18:09:41Z

Just leaving a message here to notify that I am currently working on a simple VFS layer with POSIX semantics. I'll open a WIP PR when there is anything to show.

jnohlgard · 2016-07-08T20:37:07Z

See #5616 for a new attempt to take on this old topic

Martin Lenders and others added 16 commits January 29, 2015 12:16

fs: Start filesystem module

acd5a49

[SQUASH ME] Use strncpy instead of memcpy to copy strings

e2d114f

[SQUASH ME] Store strlen of fs_table_t::mount_point in fs_table_t

9fa346d

[SQUASH ME] Search for longest path in fs_search_entry

ec5daae

Not needed anymore

0f20eab

Remove missing header file

7e730c4

This seems to be stemming from miri64/RIOT@6f9b03e569ba where an initial SCFS module is/was being developed.

Update headers

fdf19e5

Fix compiletime errors

6911334

Pull the definition of i out of the for-loop

83195cf

Otherwise it will be undefined for the rest of the function

size_t is unsigned, thus no check for < 0

da358a5

No idea what "int 1;" does there

ef5a906

Perhaps @authmillenon can help? :)

Document fs_mount and fs_umount

e88825c

Add "native" filesystem type

3a9a2fd

This will be a FS that accesses the filesystem of the underlying host in RIOTs native mode

Play around with unit tests for FS module

bdd8d70

Update license headers

d8becf3

Fix doxygen warnings

8bb57f4

OlegHahm added State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet Type: new feature The issue requests / The PR implemements a new feature for RIOT labels Feb 19, 2015

OlegHahm assigned miri64 Feb 19, 2015

x3ro reviewed Feb 19, 2015
View reviewed changes

miri64 assigned jnohlgard and unassigned miri64 Mar 24, 2015

OlegHahm force-pushed the master branch 3 times, most recently from 9f184dd to 45554bf Compare March 31, 2015 13:01

miri64 mentioned this pull request Mar 31, 2015

sys: scfs: Add filesystem that allows to mount shell_command array as directory of executable files #1418

Closed

OlegHahm added this to the Release 2015.12 milestone Dec 8, 2015

OlegHahm modified the milestones: Release 2015.12, Release 2016.03 Dec 8, 2015

OlegHahm added the State: archived State: The PR has been archived for possible future re-adaptation label Feb 26, 2016

OlegHahm closed this Feb 26, 2016

kYc0o mentioned this pull request Mar 23, 2016

Flash interface #2239

Closed

jnohlgard mentioned this pull request Jul 8, 2016

vfs: file system abstraction #5616

Merged

55 tasks

MichelRottleuthner mentioned this pull request Nov 8, 2016

package: add generic FatFs #6072

Merged

jnohlgard added the Area: fs Area: File systems label Feb 19, 2018

Conversation

x3ro commented Feb 19, 2015

Filesystem module

Summary of the discussion so far

ToDo

Uh oh!

x3ro Feb 19, 2015

Choose a reason for hiding this comment

Uh oh!

jnohlgard Feb 19, 2015

Choose a reason for hiding this comment

Uh oh!

miri64 Feb 20, 2015

Choose a reason for hiding this comment

Uh oh!

benpicco commented Mar 5, 2015

Uh oh!

OlegHahm commented Feb 26, 2016

Uh oh!

x3ro commented Feb 26, 2016

Uh oh!

OlegHahm commented Feb 26, 2016

Uh oh!

OlegHahm commented Apr 13, 2016

Uh oh!

x3ro commented Apr 13, 2016

Uh oh!

miri64 commented Apr 13, 2016

Uh oh!

kYc0o commented Apr 13, 2016

Uh oh!

miri64 commented Apr 13, 2016

Uh oh!

kYc0o commented Apr 13, 2016

Uh oh!

jnohlgard commented Jul 8, 2016

Uh oh!

jnohlgard commented Jul 8, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants