Remove -m flag from pbzip2 decompression (fixes #2030) #2032
Conversation
pbzip2 on some systems (e.g. macOS/Homebrew) has a hard-coded memory limit of 2000MB. Passing -m with a larger value caused decompression to fail and Rally to loop instead of falling back to the standard library. Dropping the -m flag lets pbzip2 use its default memory limit.
gareth-ellis
left a comment
gareth-ellis
left a comment
There was a problem hiding this comment.
I dont quite understand how this has become an issue right now - since the most recent pbzip release was in 2015 - however reading the source code, if we dont set the memory limit, it will default to 100MB, but then ramp up if we provide -p# for the number of cores - why don't we keep the max at 2000?
Which are possible issues not setting it? Some out of memory could happen? Is there any track that will create the issue? May be we are just overthinking and this is not required. How we can know why it has been introduced? If it is a corner case, should we make it configurable? |
|
I didn't realize there where already a fix for this from Jason: #2028 |
|
The issue has been closed after #2028, so I close this. The solution merged to master branch looks good to me. |
Pull request was closed
2 participants
pbzip2on some systems (e.g. macOS/Homebrew) has a hard-coded memory limit of 2000MB. Passing -m with a larger value caused decompression to fail and Rally to loop instead of falling back to the standard library. Dropping the-mflag letspbzip2use its default memory limit.Fixes #2030