Skip to content

Allow TasksMax=infinity to avoid "Pthread_create failed" error #576

@cbartz

Description

@cbartz

Steps to reproduce

Deploy mongodb and relate with 26 units.

Expected behavior

System is operational. The charm either sets

[Service]
TasksMax=infinity

in snap.charmed-mongodb.mongod.service

or allows to set it via charm configuration.

Actual behavior

We observed connection drops from clients and observed log lines like

{"t":{"$date":"2025-09-04T07:34:26.741+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.141.131.64:52308","uuid":"428dcb73-907e-4568-82e0-0e6244b12375","connectionId":20688,"connectionCount":4585}}
{"t":{"$date":"2025-09-04T07:34:26.758+00:00"},"s":"E",  "c":"-",        "id":4850900, "ctx":"listener","msg":"pthread_create failed","attr":{"error":"Resource temporarily unavailable"}}
{"t":{"$date":"2025-09-04T07:34:26.758+00:00"},"s":"E",  "c":"-",        "id":22948,   "ctx":"listener","msg":"Thread creation failed","attr":{"error":"pthread_create failed"}}
{"t":{"$date":"2025-09-04T07:34:26.758+00:00"},"s":"W",  "c":"EXECUTOR", "id":22993,   "ctx":"listener","msg":"Unable to schedule a new loop for the service state machine","attr":{"error":{"code":1,"codeName":"InternalError","errmsg":"Failed to create service entry worker thread: pthread_create failed"}}}

indicating that pthread_create failed. After finding https://www.mongodb.com/community/forums/t/pthread-create-failed-resource-temporarily-unavailable-in-mongo/100392 we saw that the Task Limit was reached

sudo systemctl status snap.charmed-mongodb.mongod.service 
● snap.charmed-mongodb.mongod.service - Service for snap application charmed-mongodb.mongod
     Loaded: loaded (/etc/systemd/system/snap.charmed-mongodb.mongod.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2025-09-04 06:33:41 UTC; 2h 41min ago
   Main PID: 632 (mongod)
      Tasks: 4647 (limit: 4647)
     Memory: 1.5G
        CPU: 1h 27min 31.481s
     CGroup: /system.slice/snap.charmed-mongodb.mongod.service
             └─632 /snap/charmed-mongodb/131/usr/bin/mongod --config /var/snap/charmed-mongodb/131/etc/mongod/mongod.conf --bind_ip_all --replSet=mongodb --dbpath=/var/snap/charmed-mo>

We could fix it by setting the TaskLimit to infinity, as outlined in the article above

ubuntu@juju-3918b7-prod-ps6-github-runner-33:~$ cat /etc/systemd/system/snap.charmed-mongodb.mongod.service.d/override.conf 
[Service]
TasksMax=infinity

Versions

Operating system: Ubuntu 22.04.5 LTS

Juju CLI: 3.6.8-ubuntu-amd64

Juju agent: 3.6.9

Charm revision: 224

LXD:

Log output

Juju debug log:

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working as expected

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions