-
Notifications
You must be signed in to change notification settings - Fork 4.1k
server: gosigar ProcFDUsage can be expensive #147829
Copy link
Copy link
Closed
Labels
C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.C-performancePerf of queries or internals. Solution not expected to change functional behavior.Perf of queries or internals. Solution not expected to change functional behavior.O-25.2.1-scale-testingO-testclusterIssues found or occurred on a test cluster, i.e. a long-running internal clusterIssues found or occurred on a test cluster, i.e. a long-running internal clusterP-2Issues/test failures with a fix SLA of 3 monthsIssues/test failures with a fix SLA of 3 monthsT-db-serverbranch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.o-perf-efficiencyRelated to performance efficiencyRelated to performance efficiencyv25.3.0-prerelease
Metadata
Metadata
Assignees
Labels
C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.C-performancePerf of queries or internals. Solution not expected to change functional behavior.Perf of queries or internals. Solution not expected to change functional behavior.O-25.2.1-scale-testingO-testclusterIssues found or occurred on a test cluster, i.e. a long-running internal clusterIssues found or occurred on a test cluster, i.e. a long-running internal clusterP-2Issues/test failures with a fix SLA of 3 monthsIssues/test failures with a fix SLA of 3 monthsT-db-serverbranch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.o-perf-efficiencyRelated to performance efficiencyRelated to performance efficiencyv25.3.0-prerelease
Describe the problem
In a recent large scale test, I was surprised to see a non-trivial amount of CPU time being spent getting our FD usage:
One potential improvement is that the linux implementation in the underlying library is using ioutil.ReadDir which is going to stat every file:
https://github.com/elastic/gosigar/blob/d69e91c3bbdd073073eb0b8e3ba2dde0d969b892/sigar_linux.go#L93C14-L98
It appears that removing these calls would probably be rather helpful.
Using
os.ReadDirfor this would probably help.Jira issue: CRDB-51271