Populate instance config and data node config during server startup #3188
Conversation
| @Config("clustermap.nimbus.service.metadata.file.path") | ||
| @Default("./etc/metadata/nimbus-service.json") | ||
| public final String nimbusServiceMetadataFilePath; | ||
|
|
||
| /** | ||
| * Path to the LiStatefulSet metadata file containing Kubernetes StatefulSet information | ||
| */ | ||
| @Config("clustermap.listatefulset.metadata.file.path") | ||
| @Default("./etc/metadata/liStatefulSet.json") | ||
| public final String liStatefulSetMetadataFilePath; |
There was a problem hiding this comment.
These values are internal linkedin concepts, please keep them out of the open source repo. We can define them in the AmbryLI configs instead.
| // Pattern to match df -h output lines for Ambry mount points | ||
| // Example: /dev/sdh1 21T 14T 6.5T 68% /mnt/u001/ambrydata | ||
| private static final Pattern DF_PATTERN = Pattern.compile( | ||
| "^(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\d+)%\\s+(/mnt/u\\d+/ambrydata)$"); |
There was a problem hiding this comment.
Can we make this pattern configurable too?
| /** | ||
| * Collect disk information by running 'df -h' command. | ||
| * @return map of mount point to DiskInfo | ||
| */ | ||
| public static Map<String, DiskInfo> collectDiskInfo() { |
There was a problem hiding this comment.
Wondering if we should use some library like OSHI instead of running df -h
| NimbusServiceMetadata nimbusMetadata = NimbusServiceMetadata.readFromFile(clusterMapConfig.nimbusServiceMetadataFilePath); | ||
| if (nimbusMetadata != null) { | ||
| LOGGER.info("Loaded nimbus service metadata - AppInstanceID: {}, NodeName: {}, MaintenanceZone: {}", | ||
| nimbusMetadata.getAppInstanceID(), nimbusMetadata.getNodeName(), nimbusMetadata.getMaintenanceZone()); | ||
| instanceConfigBuilder.setDomain(String.format(DOMAIN_TEMPLATE, nimbusMetadata.getMaintenanceZone(), nimbusMetadata.getNodeName(), nimbusMetadata.getAppInstanceID())); | ||
| } | ||
|
|
||
| LiStatefulSetMetadata liStatefulSetMetadata = LiStatefulSetMetadata.readFromFile(clusterMapConfig.liStatefulSetMetadataFilePath); |
There was a problem hiding this comment.
Similar to earlier we will need to keep internal LI concepts out since this is a generic open-source repo. We will either need to extend the HelixFactory in AmbryLI or convert this class to some interface that we can implement instead.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3188 +/- ##
=============================================
- Coverage 64.24% 51.29% -12.95%
+ Complexity 10398 8650 -1748
=============================================
Files 840 933 +93
Lines 71755 79218 +7463
Branches 8611 9457 +846
=============================================
- Hits 46099 40635 -5464
- Misses 23004 35241 +12237
- Partials 2652 3342 +690 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
3 participants
Summary
Testing Done