getter: get: raise exception for http errors in file_cache

[BibianaC]

related to #37

[BibianaC]

@R2ZER0 before going ahead with the task, I wanted to double check that this is the right aproach.

[michaelwood]

Issue #37 was more about reducing the noise in the datastore log (and triggering the monitoring) from http exceptions caused by unsuccessful fetches by ensuring that they are handled.

The full backtrace of an exception from requests can be quite long. e.g. fetching an invalid url: https://wefwefwef.example.com/test can fill the log with [1] , whereas with the datagetter we know this is a case/scenario we normally handle and therefore isn't an application error.

[1]

----> 1 requests.get("https://wefwefwef.example.com/test")

File /usr/lib/python3/dist-packages/requests/api.py:73, in get(url, params, **kwargs)
     62 def get(url, params=None, **kwargs):
     63     r"""Sends a GET request.
     64 
     65     :param url: URL for the new :class:`Request` object.
   (...)
     70     :rtype: requests.Response
     71     """
---> 73     return request("get", url, params=params, **kwargs)

File /usr/lib/python3/dist-packages/requests/api.py:59, in request(method, url, **kwargs)
     55 # By using the 'with' statement we are sure the session is closed, thus we
     56 # avoid leaving sockets open which can trigger a ResourceWarning in some
     57 # cases, and look like a memory leak in others.
     58 with sessions.Session() as session:
---> 59     return session.request(method=method, url=url, **kwargs)

File /usr/lib/python3/dist-packages/requests/sessions.py:589, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    584 send_kwargs = {
    585     "timeout": timeout,
    586     "allow_redirects": allow_redirects,
    587 }
    588 send_kwargs.update(settings)
--> 589 resp = self.send(prep, **send_kwargs)
    591 return resp

File /usr/lib/python3/dist-packages/requests/sessions.py:703, in Session.send(self, request, **kwargs)
    700 start = preferred_clock()
    702 # Send the request
--> 703 r = adapter.send(request, **kwargs)
    705 # Total elapsed time of the request (approximately)
    706 elapsed = preferred_clock() - start

File /usr/lib/python3/dist-packages/requests/adapters.py:519, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
    515     if isinstance(e.reason, _SSLError):
    516         # This branch is for urllib3 v1.22 and later.
    517         raise SSLError(e, request=request)
--> 519     raise ConnectionError(e, request=request)
    521 except ClosedPoolError as e:
    522     raise ConnectionError(e, request=request)

ConnectionError: HTTPSConnectionPool(host='wefwefwef.example.com', port=443): Max retries exceeded with url: /test (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7acc92300470>: Failed to resolve 'wefwefwef.example.com' ([Errno -5] No address associated with hostname)"))

[BibianaC]

@michaelwood thanks for your comment. Let's see if what I have done now is better.

[R2ZER0]

Thanks, looks good 👍