Specify your own filename for a screenshot

[rufuspollock]

Allow users to specify a short name to save their screenshot at (a bit like bit.ly but for screenshots). E.g. you could do ?filename=... and then you could get that screenshot forever at:

webshot-service/f/{filename}.png

[rossjones]

Almost certainly going to clash, might be easier to somehow specify you want it cached and then get given a URL that is a base 62 encoded int of some description, or a UUID or something?

[rufuspollock]

Hmm, but isn't this just like bit.ly - sure there can be clashes but that's life ...

[rossjones]

I think I'd rather have a URL that had a UUID in it than use it on a site and have someone 'accidentally' overwrite it with a picture of kittens.

[rufuspollock]

@rossjones you wouldn't be able to overwrite - like bit.ly this would be write once ...

@simong what do you reckon? how easy is this to implement do you think?

[simong]

It could work on a first-come first-served basis, similar to dns names?

We'd need to work out how to store the actual files and stick a small database behind the app. I don't think it'll be terribly hard. We just need to figure out a way to easily do this on both okfn as dev. I'll try to come up with a prototype in the next week(s).

[rufuspollock]

@simong agree re first-come, first-served (let's do the simplest thing possible). I suggest we use s3 as the backend for the present (no need for a proper database) plus we need to store images ...

[simong]

Example:
url: https://openspending.org
name: openspending.jpg

The file would be accessible at
https://webshot.okfnlabs.org/f/openspending.jpg

which would actually be stored at:

    var hash = md5('openspending.org'); // ex: f1ab4edaec312de12c...
    S3 Url: https://s3.amazon.com/okfn-webshot/f1/ab/4e/da/openspending.org

(The hashing is in place so we don't end up with thousands of files in a single folder)
If the file doesn't exist S3 will return a 404, for which we can setup a custom 404 page.

If we would ever be worried about taking up too much space we could make use of S3 their object expiration [1].

WDYT?

[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectExpiration.html

[rufuspollock]

@simong sorry to miss this at the time. Proposal looks really good. If you need s3 space we can get s3 bucket setup for you.

Small suggestions:

• we do need to check for conflicts (i.e. we should not be allowed to overwrite an existing filename).

• suggest we have an app config option to set the bucket and config option for any root path (so we could locate data in s3 bucket at

  s3.amazon.com/okfn-webshot/my/base/path/...

Also implementation detail: suggest

GET webshot.okfnlabs.org/f/openspending.jpg

redirect to s3 bucket rather than proxying to save on b/w and hassle

Would be awesome to get this implemented!

[rufuspollock]

@simong I was just using webshot again (I use it a lot!) and really missed this feature. I think this would be super useful to have.

Following my last comment my only thoughts were:

• Is it worth doing the hash - s3 does not have a problem with files in a "directory" (since it does not have directories)
• Should we consider putting in support for users (even if we don't have login yet) e.g. if username were set files would be stored at `/{username}/{filename}
• Last option would be support for allowing '/' in specified file name (so you could fake subdirectories that way)

[rufuspollock]

@simong any time to work further on this - I use this service all the time and I would love to have this ability to save feature.

I'm also wondering if we really care about namespace issues about allowing users to login with e.g. github - it is so easy to implement (but that may be overkill!)

[simong]

Hi all, I finally had some time available to move this ahead. Currently my s3-store branch has the following:

• There's an /api/store endpoint that allows you to make a POST request with all the same parameters as the /api/generate endpoint with the addition of a name parameter. That parameter allows you to retrieve the image later at /f/<name>.
• When calling /api/store we check S3 for conflicts. If the name is already used, we respond with a 400.
• Images are stored in S3, credentials (aws access key, secret key, bucket, region and directory prefix) can be defined through process environment variables. This should work nicely with Heroku
• /f/<name> redirect to a signed S3 URL, so the app isn't serving anything

Some questions:

• Should we keep this at a separate endpoint or just integrate it in the existing one?
• Depending on the above, any suggestions for the UI
• Who do I talk to to get some S3 credentials so I can deploy it? :)

[pietercolpaert]

@simong some HTTP nitpicking: if you let your client choose the name, you should use PUT. If your server can choose the name, you should use POST.

[rufuspollock]

@simong first this is fantastic - great work :-) Thoughts

• Naming: 2 options. We may want to do one, both or neither of these. However, it would massively reduce conflicts which is nice. I sort of like the date idea because you'd automatically get a sense of age (though you might also get that s3 headers)
  • What about prefixing with the date yyyymmdd format - this would much reduce conflicts on naming.
  • We could also prefix with a random 3-4 digits from [azA-Z-_0-9] - that would also massively reduce collisions.
• Really nice on the design
  • I suggest we just integrate with existing one and just add a "name" parameter
  • Re UI i think we just want to add an "Save this image" checkbox which then shows a name field
  • S3 credentials - i will get you those with assistance of @danfowler

---

@pietercolpaert personally I'm not too zealous on PUT vs POST - i know that if we are being nicely RESTful we would do it but happy not to be perfect ;-)

[simong]

It feels like you'd lose some of the value of providing a custom name if we're prefixing timestamps/random strings? Each url would look like /f/mything-Xfe or /f/2015/09/03/mything. It isn't something you'd be able to easily remember so you'd have to copy/paste it everywhere. Maybe that's OK though?