Hello, I have a question.

[jeonghs9]

hello. I downloaded 200,000 images through the Google Drive you uploaded. However, the remaining 500,000 pieces of data were needed, so they were being downloaded using an additional data code. However, an hour after the "Start downloading files for test data..." message appears, the error appears. Is there any other way to download the remaining 500,000 additional data? I need a total of 760,000 image data. Help me.

[hahashu]

Hi, I'll have a look during the weekend

[jeonghs9]

Thanks! The error is:

Start downloading files for test data...
Traceback (most recent call last):
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in validate_conn
conn.connect()
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/connection.py", line 414, in connect
self.sock = ssl_wrap_socket(
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/home/kmu/micromamba/lib/python3.9/ssl.py", line 501, in wrap_socket
return self.sslsocket_class._create(
File "/home/kmu/micromamba/lib/python3.9/ssl.py", line 1041, in _create
self.do_handshake()
File "/home/kmu/micromamba/lib/python3.9/ssl.py", line 1310, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/kmu/micromamba/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/home/kmu/micromamba/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.sh-sunwardpack.com', port=443): Max retries exceeded with url: /functional-polyolefin-shrink-film/waterproof-polyolefin-shrink-film.html (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1129)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/kmu/TextileNet/prepare_data.py", line 58, in
download_files(f'{args.parent_dir}/test', test_data)
File "/home/kmu/TextileNet/prepare_data.py", line 29, in download_files
img_data = requests.get(url).content
File "/home/kmu/micromamba/lib/python3.9/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/home/kmu/micromamba/lib/python3.9/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/home/kmu/micromamba/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/home/kmu/micromamba/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/home/kmu/micromamba/lib/python3.9/site-packages/requests/adapters.py", line 563, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.sh-sunwardpack.com', port=443): Max retries exceeded with url: /functional-polyolefin-shrink-film/waterproof-polyolefin-shrink-film.html (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1129)')))

[hahashu]

Hi, thanks for the question.
I guess some of the URLs are broken since last time I have downloaded them. I've added a try clause to handle the exceptions in the download function, think it should work now. Thanks.

[jeonghs9]

Thank you so much for your help! Let's try!