Add support for setup of rclone#3085
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
bbockelm
added
enhancement
|
@bbockelm What is accepted for the Regex filter is ideal, just need something so the web ui can provide some guidance to validity. |
|
It's perhaps overly-permissive (examples are in the docs and unit test) so a regex may be surprisingly hard. Accepts a number (float or integer) followed by a unit. The unit can be any common SI prefix followed by '/s' or 'ps' or other time specification. |
|
@bbockelm Sounds good, I will tack on a PR tomorrow to sort the config piece. |
|
One thing to note - this only affects a "hidden" knob currently. I don't have a feel for whether that impacts the web UI code. |
This commit adds new sub-commands for installing and configuring rclone to work with a Pelican data federation. Particularly, rclone has the ability to invoke a command to renew the token *if* a 401 is sent; this is done and the director now returns a 401 on expiration.
Previously, the code iterated through all the existing scopes and checked to see if they were 'acceptable'. It's unclear the goal of this: what we really wanted was to specify all the needed functionality (read, write, delete) and ask if this functionality would be satisfied by the existing scopes. Noticed when rclone kept trying to reuse a token with only storage.create.
- Shared permission implies the token should be generated for exactly the path -- it's shared like a "pre-signed URL", as specific as possible. - Regular permission may ask for more authorization to avoid getting tokens repeatedly.
bbockelm
force-pushed
the
rclone-support
branch
from
62998fe to
c2ef9e0
Compare
|
Note: for a reasonable user experience, we'll need to wait until after rclone/rclone#9152 (or equivalent) is part of an |
3 participants
This commit adds new sub-commands for installing and configuring rclone to work with a Pelican data federation.
Particularly, rclone has the ability to invoke a command to renew the token if a 401 is sent; this is done and the director now returns a 401 on expiration.