git-lfs: add support for board documentation files

[jia200x]

Contribution description

This PR adds support for using git-lfs for board documentation images (and related files). It's IMO a good trade-off between having files closer to the code and not bloating the repository.

This PR tracks all files undes board/<board_name>/doc folder as git-lfs files.

git-lfs in a nutshell

Assuming the git-lfs extension is installed (apt-get install git-lfs), simply install git-lfs in the RIOT repo:

git lfs install --local

Then, work with the github repo as you normally do. When pushing to origin or another remote, git-lfs will push a symlink-like file in the repo and upload the real file to another repo. This way the original repo doesn't bloat and only contains symlinks pointing to the actual file.

Github handles LFS files natively, as seen here (there is a Stored with Git LFS message).

So if git-lfs is installed, is exactly the same as using standard git. Both from the local computer and from the github.com file viewer.

FAQ

• Where are LFS files stored?: Github provides a LFS server in the same repo hidden under RIOT.git/info/lfs
• If I push files to my origin, will the symlink file point to my repo?: This seems to be handled by Github, the remote location is not included in the symlink file (only a hash and some references to git-lfs.github.com)
• If someone doesn't have git-lfs, can still use the RIOT repo? It's possible to use the repo without Git LFS. However, make doc will fail wih a Please install git-lfs message. The new make doc target not only checks if git-lfs is installed but also executes git lfs install --local as a dependency.
• How does it work with Pull Requests?: If someone uploads a picture without using git-lfs, Github will treat it as Git LFS file instead of Binary (see different files here). It's up to maintainers to make sure only Git LFS file get to boards/<board_name>/doc (see Possible improvements)

Possible improvements

• Git hook for detecting if LFS is installed when changing <board_name>/doc files.

Testing procedure

Try running make doc with and without git-lfs.

Note this doesn't add <board_name>/doc to the RIOT doxygen file, it will be done in a further PR.

Issues/PRs references

None so far.

[jia200x]

Murdock should fail if it tries to run make doc. It's intended, and should install the git-lfs dependency before merging this PR (if accepted by the community)

[jcarrano]

should install the git-lfs dependency before merging this PR

I set the 'Waiting' label. I don't know how applicable it is, but at least it lets people know it depends on something else.

[jia200x]

I set the 'Waiting' label. I don't know how applicable it is, but at least it lets people know it depends on something else.

That's right, thanks!

[jcarrano]

This PR tracks all files undes board/<board_name>/doc folder as git-lfs files.

Non-binary files as well? Can we filter on extension?

[jia200x]

Murdock should fail if it tries to run make doc. It's intended, and should install the git-lfs dependency before merging this PR (if accepted by the community)

Oh, it passed... does Murdock run make doc?

[jia200x]

Non-binary files as well? Can we filter on extension?

Yes, wildcards and basic regex patterns are accepted

[jcarrano]

In my machine:

$ make doc
"make" -BC doc/doxygen
make[1]: Entering directory '/home/jcarrano/source/vanillaRIOT/doc/doxygen'
Please install git lfs.
make[1]: *** [Makefile:52: git-lfs-check] Error 1
make[1]: Leaving directory '/home/jcarrano/source/vanillaRIOT/doc/doxygen'
make: *** [Makefile:10: doc] Error 2

But the check script runs "fine":

Running './dist/tools/doccheck/check.sh' ✓

The reason is that doccheck does not test the exit code of make or any error other than the filters it has configured.

[jia200x]

The reason is that doccheck does not test the exit code of make or any error other than the filters it has configured.

I see. I will patch that

[jcarrano]

@jia200x I already patched it!

[PeterKietzmann]

This PR has the "waiting for other PR" label set. What is it waiting for?

[jcarrano]

@PeterKietzmann it was waiting for #9819. If LFS was not available, the build would fail silently.

[jcarrano]

It needs a couple of small makefile fixes, and then it's in.

[jcarrano]

Make these things phony.

[jcarrano]

Also, install should depend on check. With the current definitions there is no guarantee that install won't try to run before check.

[danpetry]

ping @jia200x

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[chrysn]

I still like this approach and would love to see it resumed.

[chrysn]

Maybe that should say *.jpg *.png immaterial of the precise location?