Config.DialContext hook and http.RoundTripper support#31
Config.DialContext hook and http.RoundTripper support#31ydnar wants to merge 37 commits intosecsy:masterfrom
Conversation
muirmanders
left a comment
muirmanders
left a comment
There was a problem hiding this comment.
Thank you for your contribution. Can you help me understand why Config.DialContext hook is needed?
client.go
Outdated
| var conn net.Conn | ||
|
|
||
| ctx := context.Background() | ||
| if c.config.Timeout != 0 { |
There was a problem hiding this comment.
config.Timeout has a default so it will never be 0.
There was a problem hiding this comment.
Saw that. I’m not sure I agree with the design decision (versus defaulting timeout if zero at runtime), but it’s your library. Happy to change.
There was a problem hiding this comment.
Done (719b580). I also noted where the library should wire up a parent Context in the future to allow overall FTP operations to be governed by a caller’s Context.
transport.go
Outdated
| // Transport implements the http.RoundTripper interface. | ||
| // Typical usage would be to register a Transport to handle | ||
| // ftp:// and/or ftps:// URLs with http.Transport.RegisterProtocol. | ||
| type Transport struct { |
There was a problem hiding this comment.
Can we name this HTTPTransport so it isn't confused for an ftp transport?
There was a problem hiding this comment.
I think bare Transport works better because it is an FTP transport that happens to implement the http.RoundTripper interface.
There was a problem hiding this comment.
Eliminated the separate Transport struct in favor of a method on Config.
|
To answer your question (“why do you need a Our systems connect to many (i.e. hundreds) of systems that allow-list our IP addresses. We maintain proxy hosts with static IPs at various cloud providers and our production hosts connect to the remote hosts via an SSH+SOCKS5 tunnel. We need to be able to connect to remote FTP servers and control how the network connection is established. Our backend has dialers (with a Your FTP library was the most mature/robust, so we forked it to add the |
|
While we’re on the subject of naming things, it feels itchy to me that the Would you be opposed to adding a top-level function |
Agreed.
Bad name is bad, but having a bad name and another name doesn't help much at this point. Once modules are the thing we can do a v2 and go crazy renaming. |
|
Re: |
This enables clients to specify their own connection logic, to permit creation of connections through a proxy (for IP whitelisting).
Not all net.Conn implementations provide a valid RemoteAddr(), in particular crypto/ssh.Conn: https://godoc.org/golang.org/x/crypto/ssh#Client.Dial
This enables clients to handle GET requests of ftp:// or ftps:// URLs with a normal http.Client using http.Transport.RegisterProtocol().
Also note where parent Context should be wired up in future.
…RL.User is non-nil
- Update proftpd to 1.3.7a and remove monkey patches - Update pure-ftpd to 1.0.49 - Parameterized version numbers - Check-before-download to speed up repeated builds
This is disabled in .travis.yml
ydnar
changed the title
…ously get error and set res.Status
2 participants
This PR implements two related features that we needed in production to enable downloads of files from FTP(S) servers that filter connections to IP allow-lists.
Config.DialContexthook, mirrored from the implementation innet/http.Transport. If set, this is used instead ofnet.Dialto establish TCP connections.http.RoundTripperinterface onConfigto allow anhttp.Clientdownloadftp://URLs.A few additional things for 2021:
go.modfile (required for Go 1.16+ support).