Skip to content

Race Condition: Error is thrown if multiple processes try to roll over a log file #712

@christophertubbs

Description

@christophertubbs

Encountered while testing the evaluation service:

Traceback (most recent call last):
  File "/Frameworks/Python.framework/Versions/3.8/lib/python3.8/logging/handlers.py", line 70, in emit
    self.doRollover()
  File "/Frameworks/Python.framework/Versions/3.8/lib/python3.8/logging/handlers.py", line 394, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "/Frameworks/Python.framework/Versions/3.8/lib/python3.8/logging/handlers.py", line 111, in rotate
    os.rename(source, dest)
FileNotFoundError: [Errno 2] No such file or directory: 'DMOD/python/services/evaluationservice/dmod/evaluationservice/evaluations.log' -> 'DMOD/python/services/evaluationservice/dmod/evaluationservice/evaluations.log.2024-08-13'

Two processes (runners for the evaluation service) tried to roll over the rotating handler at the same time. Python doesn't perform interprocessing communication when logging, so it does not consider this a race condition or something python itself should fix.

Something needs to be put into place to prevent this in the future.

This is default handling. The environment variable LOGGING_CONFIGURATION may be set to change how logging operates in the case that a remote handler may be needed or file writing needs to be removed in a particular environment.

This is low priority since it may be hard to hit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Low Hanging FruitThis should not take a lot of timeLow PriorityThis should not take the place of more important workenhancementNew feature or requestgood first issueGood for newcomers

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions