Issue When Trying to Import AttributeQuery and TextQuery

[yenon118]

Hi RCSB PDB group,

I was trying to use the rcsb-api (https://github.com/rcsb/py-rcsb-api) to get some data today. However, I was experiencing connection refuse error when I was trying to import AttributeQuery and TextQuery. Could you please help me with this issue? If that is an issue with your server, please let me know as well. The error messages are attached below:

>>> from rcsbapi.search import AttributeQuery, TextQuery
Traceback (most recent call last):
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
    yield
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_transports/default.py", line 250, in handle_request
    resp = self._pool.handle_request(req)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request
    raise exc from None
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request
    response = connection.handle_request(
        pool_request.request
    )
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
    raise exc
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_sync/connection.py", line 78, in handle_request
    stream = self._connect(request)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_sync/connection.py", line 124, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_backends/sync.py", line 207, in connect_tcp
    with map_exceptions(exc_map):
         ~~~~~~~~~~~~~~^^^^^^^^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    from rcsbapi.search import AttributeQuery, TextQuery
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/rcsbapi/search/__init__.py", line 4, in <module>
    from rcsbapi.search.search_query import SEARCH_SCHEMA  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/rcsbapi/search/search_query.py", line 662, in <module>
    SEARCH_SCHEMA = SearchSchema(Attr)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/rcsbapi/search/search_schema.py", line 195, in __init__
    self.struct_schema = self._reload_schema(struct_attr_schema_url, struct_attr_schema_file, refetch, use_fallback)
                         ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/rcsbapi/search/search_schema.py", line 209, in _reload_schema
    sD = self._fetch_schema(schema_url)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/rcsbapi/search/search_schema.py", line 223, in _fetch_schema
    response = httpx.get(url, timeout=None, headers={"User-Agent": const.USER_AGENT}, follow_redirects=True)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_api.py", line 195, in get
    return request(
        "GET",
    ...<9 lines>...
        trust_env=trust_env,
    )
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_api.py", line 109, in request
    return client.request(
           ~~~~~~~~~~~~~~^
        method=method,
        ^^^^^^^^^^^^^^
    ...<8 lines>...
        follow_redirects=follow_redirects,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
        request,
    ...<2 lines>...
        history=[],
    )
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
        request,
        follow_redirects=follow_redirects,
        history=history,
    )
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_client.py", line 1014, in _send_single_request
    response = transport.handle_request(request)
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_transports/default.py", line 249, in handle_request
    with map_httpcore_exceptions():
         ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/mnt/pixstor/joshitr-lab/chanye/tools/miniconda/envs/rcsb-api/lib/python3.13/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 111] Connection refused

Thank you.

Best,
Yen

[piehld]

Hi @yenon118, thank you for reporting this. This appears to have been a temporary issue following some website maintenance (in combination with a minor oversight in the rcsb-api package—which will be addressed soon). However, this should be working now.

Can you please try again now?

[qpxu007]

I also had this issue for a few weeks now, was thinking probably due to a network glitch, but the problem has been persisted. Upgraded to latest version (from 1.4.2), still the same issue.

e.g.

from rcsbapi.search import AttributeQuery, TextQuery

would hang for a long time.

[piehld]

Hi @qpxu007, sorry to hear you're still facing this. Can you please try upgrading to the latest version, 1.5.0 and trying again? (https://pypi.org/project/rcsb-api/)

pip install "rcsb-api>=1.5.0"

[qpxu007]

I am already on 1.5.0.

I dug a little deeper, the issue appears to be ssl related

response = httpx.get("https://search.rcsb.org/rcsbsearch/v2/metadata/schema", timeout=10, headers={"User-Agent": f"py-rcsb-api/{version} (+https://github.com/rcsb/py-rcsb-api)"}, follow_redirects=True)

Traceback (most recent call last):
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_transports/default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request
raise exc from None
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request
response = connection.handle_request(
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
raise exc
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 78, in handle_request
stream = self._connect(request)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 156, in _connect
stream = stream.start_tls(**kwargs)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_backends/sync.py", line 170, in start_tls
raise exc
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/contextlib.py", line 137, in exit
self.gen.throw(typ, value, traceback)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectTimeout: _ssl.c:1112: The handshake operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_api.py", line 195, in get
return request(
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_api.py", line 109, in request
return client.request(
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_client.py", line 825, in request
return self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_client.py", line 914, in send
response = self._send_handling_auth(
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_client.py", line 1014, in _send_single_request
response = transport.handle_request(request)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_transports/default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/contextlib.py", line 137, in exit
self.gen.throw(typ, value, traceback)
File "/mnt/beegfs/.software_bl2/px/miniconda3/envs/opencv/lib/python3.9/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectTimeout: _ssl.c:1112: The handshake operation timed out

qxu@bl2ws5:~/data-analysis$ curl -v --tls-max 1.2 https://search.rcsb.org/rcsbsearch/v2/metadata/schema

• Trying 128.6.158.50:443...
• Connected to search.rcsb.org (128.6.158.50) port 443 (#0)
• ALPN, offering h2
• ALPN, offering http/1.1
• CAfile: /mnt/software/px/CCP4/ccp4-9/etc/ssl/cacert.pem
• CApath: /etc/ssl/certs
• TLSv1.0 (OUT), TLS header, Certificate Status (22):
• TLSv1.2 (OUT), TLS handshake, Client hello (1):
• SSL connection timeout
• Closing connection 0
  curl: (28) SSL connection timeout

but the same command works fine on ubuntu 24.04 variants, just not on ubuntu 22 OS variants

Thanks.

[qpxu007]

PS adding a timeout option to this call would help jump out of the stalled import

[piehld]

Thanks for all the digging and details @qpxu007! Curiously, it was working for me earlier, but then later on I encountered the long hang time, so it seems to be a bit intermittent.

This definitely seems to be related to a server issue rather than an issue with the package, so I've reached out to some other members of our team to help take a look. We did undergo quite a significant change to our infrastructure recently, so this seems to directly coincides with that. Apologies for the inconveniences.

As I'm typing this, I've just heard that they did make one change. Can you please try again (both via Python and curl) and report back? Also, we are curious if you see this happening intermittently, or consistently?

[qpxu007]

still the same issue on Mint 21.3 (ubuntu 22) but fine with Mint 22.2 (ubuntu 24).

openssl s_client -connect search.rcsb.org:443 -servername search.rcsb.org -showcerts

CONNECTED(00000003)

hangs. Firefox on mint 21.3 (has its own ssl stack) also had no issue.

the ssl handshake never gets a response back when hang.

[qpxu007]

the problem might be due to mtu size, we use mtu 9000 on our servers

[qpxu007]

seems to work today, not sure whether it is rcsb has changed something or packet routing has changed (which would explain why it fails intermittently). thanks

[piehld]

Thank you for the further testing and reports, @qpxu007. The potential MTU issue was definitely a relevant one, and one that we've certainly encountered many times before. We did make another adjustment to that earlier today, so potentially that is why it began working for you recently.

I will plan to add a timeout and more explicit warning to users about this issue. In the meantime though, if you (or anyone else reading this) encounter(s) this again, please ping us again. I will leave this ticket open for that purpose. Following our recent infrastructure transition, we obviously want to make sure access isn't interrupted for anyone.