Skip to content

Increase reserve on ScaleIO disk formatting for fragmentation#7955

Merged
yadvr merged 1 commit intoapache:4.18from
mlsorensen:4.18-scaleio-fragmentation
Sep 14, 2023
Merged

Increase reserve on ScaleIO disk formatting for fragmentation#7955
yadvr merged 1 commit intoapache:4.18from
mlsorensen:4.18-scaleio-fragmentation

Conversation

@mlsorensen
Copy link
Copy Markdown
Contributor

Description

This PR increases the overhead calculated for ScaleIO encrypted volumes slightly from 32MB to 128MB, due to edge cases found with lots of small writes over many weeks causing excessive fragmentation. Ideally discard/fstrim is being issued in the guest to free some meaningful amount of space occasionally, but this should help for situations where that isn't guaranteed.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Tested locally, and updated the unit tests to verify the return value is as expected.

Signed-off-by: Marcus Sorensen <mls@apple.com>
@mlsorensen mlsorensen changed the base branch from main to 4.18 September 8, 2023 23:07
@codecov
Copy link
Copy Markdown

codecov bot commented Sep 8, 2023

Codecov Report

Merging #7955 (acf194b) into 4.18 (57c61fb) will increase coverage by 1.33%.
Report is 3 commits behind head on 4.18.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               4.18    #7955      +/-   ##
============================================
+ Coverage     13.06%   14.40%   +1.33%     
- Complexity     9097    10114    +1017     
============================================
  Files          2720     2748      +28     
  Lines        257465   259424    +1959     
  Branches      40146    40385     +239     
============================================
+ Hits          33643    37368    +3725     
+ Misses       219594   217222    -2372     
- Partials       4228     4834     +606     
Files Changed Coverage Δ
.../hypervisor/kvm/storage/ScaleIOStorageAdaptor.java 12.31% <100.00%> (+1.86%) ⬆️

... and 175 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Copy Markdown
Member

@yadvr yadvr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - didn't test it though. I'm wondering if this could be an env-specific attribute (in which case this may be tuneable as a global settings or property in agent.properties)

@mlsorensen
Copy link
Copy Markdown
Contributor Author

LGTM - didn't test it though. I'm wondering if this could be an env-specific attribute (in which case this may be tuneable as a global settings or property in agent.properties)

It shouldn't change based on environment, it should just be accounted for as part of the storage implementation.

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Sep 11, 2023

Makes sense, thanks @mlsorensen

@yadvr yadvr added this to the 4.18.2.0 milestone Sep 11, 2023
@yadvr
Copy link
Copy Markdown
Member

yadvr commented Sep 11, 2023

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7014

Copy link
Copy Markdown
Member

@harikrishna-patnala harikrishna-patnala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Sep 14, 2023

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@rohityadavcloud a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Sep 14, 2023

Merging based on simulator as code path won't be tested by smoketests/kvm that uses nfs/local storage and not scaleio.

@yadvr yadvr merged commit f049d4d into apache:4.18 Sep 14, 2023
@blueorangutan
Copy link
Copy Markdown

[SF] Trillian test result (tid-7667)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39825 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7955-t7667-kvm-centos7.zip
Smoke tests completed. 108 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants