Explore the possibility of a new, async network stack

[jpd236]

While Volley's client interface is asynchronous, the internals it uses to perform requests are not - there are fixed-size thread pools for cache and network interactions which get blocked for the duration of any requests. This is inherent to the public APIs that Volley currently exposes.

If we wanted to resolve this and support asynchronous HTTP client stacks, we would either need to accept a breaking API change to Volley, or else explore building an alternative network stack API structure that would coexist with the existing synchronous stack. We'd need AsyncRequestQueue/AsyncNetwork/AsyncCache at a minimum.

Notably, there is no method for sending asynchronous HTTP requests built into the Android SDK. (Disk caching might be doable with the nio package). So we wouldn't actually be able to provide an out-of-the-box implementation in Volley's core (at least without taking on a new dependency). We could consider starting separate projects e.g. volley-okhttp or volley-cronet which provide implementations of async queues.

Filing this for tracking ideas, but at the moment this is not on the roadmap.

[jjoslin]

I'd be interested in helping out here.

[jpd236]

One random piece: per #80, the way our RetryPolicy interface is intended to work is by increasing request timeouts on each subsequent attempt; however, it doesn't introduce any time delay between request attempts. Currently, it's "safe" to throw a sleep() in the retry method to effectively introduce this gap, since it's the NetworkDispatcher thread which is calling it, and that thread is already blocking on network traffic. But in a world with Async dispatchers, we'd want another way to trigger a retry after a delay.

This may involve a similar change as we did for HttpStack -> BaseHttpStack, since RetryPolicy is an interface - deprecate RetryPolicy, introduce a new class which implements it and provide a new default method to return the delay in between requests which defaults to 0.

[jpd236]

Very, very rough proof-of-concept here: jpd236@18acc6f

There are certainly some details still to be worked out, but I'm reasonably confident that there is a path forward here which would avoid breaking existing clients while still allowing development of a new async layer. The next step here will be to write a design doc.

[sphill99]

Things that aren't completed yet for this:

• Thread safety in the cache
• Support for traffic stats tags
• Provide a Google Cronet HTTP Stack
• Optimization in Cache to only put metadata when the response is not modified
• New retry API that supports waiting before retrying the request (WIP: New retry api for supporting timeouts before retrying again #363)

[jpd236]

I'm currently leaning towards reverting the DiskBasedCache changes and punting the asynchronous cache implementation to a later date. Trying to achieve thread safety while allowing parallel reads/writes seems very challenging to do correctly, even with something like ReentrantReadWriteLock, given the combination of global in-memory state with per-entry on-disk state and the need to support an asynchronous API surface. The value is also somewhat diminished given that these NIO APIs require API 26+, which currently excludes about 40% of devices, and maintaining two code paths here is a maintenance burden that will be hard to test well. And if you're using Cronet, there's already an HTTP cache which can optionally be configured/used there, so Volley's cache layer is probably not adding much value at all.

We can complete the rest of the stack and possibly retain support for the AsyncCache interface without too much trouble; we just won't provide an implementation out of the box from day 1, and will use the fallback path instead. Longer term, we can see if we can improve the parallelization of DiskBasedCache by using ReentrantReadWriteLock, and/or provide an implementation of AsyncCache on top of another I/O platform (perhaps LevelDB, which has at least some degree of asynchronicity).