Threading Race Condition in boto3 Waiters

[Vladyyy]

Description

When running 100+ parallel boto3 waiter operations after the jmespath cache exceeds 128 entries, the cache eviction mechanism triggers a race condition.

StackTrace

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/lib/python3.9/site-packages/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/lib/python3.9/site-packages/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "<application_code>", line 89, in attach_volume
    volume._attach(instance_id, device_name)
  File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "<application_code>", line 341, in _attach
    waiter.wait(VolumeIds=[self.id], Filters=filters)
  File "/usr/lib/python3.9/site-packages/botocore/waiter.py", line 58, in wait
    Waiter.wait(self, **kwargs)
  File "/usr/lib/python3.9/site-packages/botocore/waiter.py", line 339, in wait
    acceptors = list(self.config.acceptors)
  File "/usr/lib/python3.9/site-packages/botocore/waiter.py", line 167, in acceptors
    acceptor = AcceptorConfig(acceptor_config)
  File "/usr/lib/python3.9/site-packages/botocore/waiter.py", line 178, in __init__
    self.matcher_func = self._create_matcher_func()
  File "/usr/lib/python3.9/site-packages/botocore/waiter.py", line 211, in _create_matcher_func
    return self._create_path_matcher()
  File "/usr/lib/python3.9/site-packages/botocore/waiter.py", line 226, in _create_path_matcher
    expression = jmespath.compile(self.argument)
  File "/usr/lib/python3.9/site-packages/jmespath/__init__.py", line 8, in compile
    return parser.Parser().parse(expression)
  File "/usr/lib/python3.9/site-packages/jmespath/parser.py", line 91, in parse
    self._free_cache_entries()
  File "/usr/lib/python3.9/site-packages/jmespath/parser.py", line 492, in _free_cache_entries
    for key in random.sample(self._CACHE.keys(), int(self._MAX_SIZE / 2)):
  File "/usr/lib/python3.9/random.py", line 431, in sample
    population = tuple(population)
RuntimeError: dictionary changed size during iteration

Environment

Python: 3.9
jmespath: 0.9.x (pre-1.0.0)

[nateprewitt]

Thanks for the issue @Vladyyy! This should be addressed in #331 and available in the next release. I'll close this as resolved but you can reopen if you encounter anything. Feel free to test against develop in the meantime and let us know if you're still seeing any further issues.