OutOfRangeError: tf.train.shuffle_batch of load_data() method in data_loader.py

[gardenlee21]

I faced the OutOfRangeError at the start point of train, in epoch 0.
Specifically, the error is occurred at the line below (in data_loader.py):
data_vol_batch, brainmask_batch, label_batch = tf.train.shuffle_batch([data_vol, brainmask, label], batch_size, 50, 10, num_threads=1)

I tried to fix it by changing the value 50 and 10, these values represent capacity and min_after_dequeue, respectively.
Also, I tried to solve it by changing the value in tf.train.string_input_producer, in _decode_samples() method. (It is called in load_data() method)

Below is my whole error logs. I cannot figure out how to fix it.

2023-02-14 17:57:25.297258: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2023-02-14 17:57:25.700173: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties:
name: NVIDIA GeForce RTX 3090 major: 8 minor: 6 memoryClockRate(GHz): 1.695
pciBusID: 0000:e1:00.0
totalMemory: 23.70GiB freeMemory: 23.45GiB
2023-02-14 17:57:25.700223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
2023-02-14 17:57:26.545429: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2023-02-14 17:57:26.545481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0
2023-02-14 17:57:26.545490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N
2023-02-14 17:57:26.545638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 22733 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:e1:00.0, compute capability: 8.6)
In the epoch  0
2023-02-14 17:57:35.257638: W tensorflow/core/kernels/queue_base.cc:285] _22_input_producer_10: Skipping cancelled dequeue attempt with queue not closed
2023-02-14 17:57:40.246947: W tensorflow/core/kernels/queue_base.cc:285] _22_input_producer_10: Skipping cancelled dequeue attempt with queue not closed
2023-02-14 17:57:40.565829: W tensorflow/core/kernels/queue_base.cc:285] _22_input_producer_10: Skipping cancelled dequeue attempt with queue not closed
2023-02-14 17:57:40.898393: W tensorflow/core/kernels/queue_base.cc:285] _22_input_producer_10: Skipping cancelled dequeue attempt with queue not closed
Traceback (most recent call last):
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
    return fn(*args)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_15_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: shuffle_batch_1 = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch_1/random_shuffle_queue, shuffle_batch_1/n)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 428, in <module>
    main()
  File "main.py", line 424, in main
    multimodal_model.train()
  File "main.py", line 301, in train
    data_feed_valid, label_feed_valid = sess.run([data_valid, label_valid])
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run
    run_metadata_ptr)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1100, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
    run_metadata)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_15_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: shuffle_batch_1 = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch_1/random_shuffle_queue, shuffle_batch_1/n)]]

Caused by op 'shuffle_batch_1', defined at:
  File "main.py", line 428, in <module>
    main()
  File "main.py", line 424, in main
    multimodal_model.train()
  File "main.py", line 258, in train
    gt_flag=True, crop_size=CROP_SIZE, num_cls=NUM_CLS)
  File "/home/gardenlee21/RobustMseg-main/data_loader.py", line 64, in load_data
    data_vol_batch, brainmask_batch, label_batch = tf.train.shuffle_batch([data_vol, brainmask, label], batch_size, 50, 10, num_threads=5, allow_smaller_final_batch=True)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 1300, in shuffle_batch
    name=name)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/training/input.py", line 844, in _shuffle_batch
    dequeued = queue.dequeue_up_to(batch_size, name=name)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/ops/data_flow_ops.py", line 520, in dequeue_up_to
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3644, in queue_dequeue_up_to_v2
    component_types=component_types, timeout_ms=timeout_ms, name=name)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    return func(*args, **kwargs)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    op_def=op_def)
  File "/home/gardenlee21/anaconda3/envs/rms_0213/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    self._traceback = tf_stack.extract_stack()

OutOfRangeError (see above for traceback): RandomShuffleQueue '_15_shuffle_batch_1/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: shuffle_batch_1 = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch_1/random_shuffle_queue, shuffle_batch_1/n)]]