From 8edd496f004bf9ad1ec514cfdcfa69f9b20c43ec Mon Sep 17 00:00:00 2001 From: "G.Reijn" Date: Wed, 19 Nov 2025 02:37:35 +0100 Subject: [PATCH 1/2] Add `shallowMerge` function --- .../schemas/config/functions/shallowMerge.md | 426 ++++++++++++++++++ .../schemas/config/functions/tryWhich.md | 211 +++++++++ dsc/tests/dsc_functions.tests.ps1 | 97 ++++ lib/dsc-lib/locales/en-us.toml | 3 + lib/dsc-lib/src/functions/mod.rs | 2 + lib/dsc-lib/src/functions/shallow_merge.rs | 162 +++++++ 6 files changed, 901 insertions(+) create mode 100644 docs/reference/schemas/config/functions/shallowMerge.md create mode 100644 docs/reference/schemas/config/functions/tryWhich.md create mode 100644 lib/dsc-lib/src/functions/shallow_merge.rs diff --git a/docs/reference/schemas/config/functions/shallowMerge.md b/docs/reference/schemas/config/functions/shallowMerge.md new file mode 100644 index 000000000..7f6bc035e --- /dev/null +++ b/docs/reference/schemas/config/functions/shallowMerge.md @@ -0,0 +1,426 @@ +--- +description: Reference for the 'shallowMerge' DSC configuration document function +ms.date: 11/19/2025 +ms.topic: reference +title: shallowMerge +--- + +## Synopsis + +Combines an array of objects into a single object where only the top-level properties are merged. + +## Syntax + +```Syntax +shallowMerge() +``` + +## Description + +The `shallowMerge()` function takes an array of objects and combines them into a single +object by merging their properties. When the same property name appears in multiple objects, +the value from the last object in the array takes precedence. + +This is a **shallow merge**, meaning: + +- Top-level properties are merged from all objects +- If a property value is an object, it replaces the entire object from previous objects + rather than merging the nested properties +- Arrays and other complex types are also replaced entirely, not combined + +This function is useful for: + +- Building composite configuration objects from multiple sources +- Applying configuration overrides where later values take precedence +- Combining default settings with user-specified customizations +- Merging environment-specific configurations + +The shallow merge behavior differs from a deep merge (like [`union()`][00]) where nested +objects would be recursively merged. With `shallowMerge()`, nested structures are replaced +entirely by the last object's value. + +## Examples + +### Example 1 - Merge configuration objects + +The following example demonstrates merging two configuration objects using [`createArray()`][02] +and [`createObject()`][03], where the second object overrides properties from the first. + +```yaml +# shallowMerge.example.1.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[shallowMerge(createArray(createObject('host', 'localhost', 'port', 8080), createObject('port', 9000, 'ssl', true())))]" +``` + +```bash +dsc config get --file shallowMerge.example.1.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + host: localhost + port: 9000 + ssl: true +messages: [] +hadErrors: false +``` + +Notice how the `port` value from the second object (9000) replaces the value from the first +object (8080), while properties that only exist in one object (`host` and `ssl`) are +preserved. + +### Example 2 - Apply multiple configuration layers + +The following example shows combining multiple configuration layers using parameters, where +later objects in the array override properties from earlier objects. + +```yaml +# shallowMerge.example.2.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + defaults: + type: object + defaultValue: + timeout: 30 + retries: 3 + debug: false + environment: + type: object + defaultValue: + timeout: 60 + userPrefs: + type: object + defaultValue: + debug: true +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[shallowMerge(createArray(parameters('defaults'), parameters('environment'), parameters('userPrefs')))]" +``` + +```bash +dsc config get --file shallowMerge.example.2.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + timeout: 60 + retries: 3 + debug: true +messages: [] +hadErrors: false +``` + +The final configuration shows `timeout` overridden by environment settings, `debug` +overridden by user preferences, and `retries` preserved from defaults. + +### Example 3 - Shallow merge replaces nested objects + +The following example demonstrates the key difference between shallow and deep merge. When a +property contains a nested object, the entire nested object is replaced rather than merged. + +```yaml +# shallowMerge.example.3.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[shallowMerge(createArray(createObject('database', createObject('host', 'localhost', 'port', 5432, 'ssl', true())), createObject('database', createObject('host', 'prod.db.local'))))]" +``` + +```bash +dsc config get --file shallowMerge.example.3.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + database: + host: prod.db.local +messages: [] +hadErrors: false +``` + +The second object's `database` property completely replaces the first object's `database`, +losing the `port` and `ssl` properties. This is the shallow merge behavior. + +### Example 4 - Merge with empty objects + +The following example shows that empty objects in the array don't affect the merge result. + +```yaml +# shallowMerge.example.4.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[shallowMerge(createArray(createObject('name', 'Service1', 'enabled', true()), createObject(), createObject('version', '2.0')))]" +``` + +```bash +dsc config get --file shallowMerge.example.4.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + name: Service1 + enabled: true + version: '2.0' +messages: [] +hadErrors: false +``` + +The empty object in the middle doesn't remove or affect any properties. + +### Example 5 - Build feature flags from multiple sources + +The following example merges base flags with team-specific and environment-specific overrides, +where each subsequent object overrides specific flags while preserving others. + +```yaml +# shallowMerge.example.5.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[shallowMerge(createArray(createObject('newUI', false(), 'darkMode', true(), 'beta', false()), createObject('newUI', true()), createObject('beta', true())))]" +``` + +```bash +dsc config get --file shallowMerge.example.5.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + newUI: true + darkMode: true + beta: true +messages: [] +hadErrors: false +``` + +Each subsequent object overrides specific flags while preserving others. + +### Example 6 - Merge array results from parameters + +The following example demonstrates merging parameter-based objects where the last occurrence +of each property wins, resulting in a single merged object. + +```yaml +# shallowMerge.example.6.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + objects: + type: array + defaultValue: + - name: alpha + priority: 1 + - name: beta + priority: 2 + - name: beta + priority: 10 + critical: true +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: "[shallowMerge(parameters('objects'))]" +``` + +```bash +dsc config get --file shallowMerge.example.6.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + name: beta + priority: 10 + critical: true +messages: [] +hadErrors: false +``` + +The last occurrence of each property wins, resulting in a single merged object. + +### Example 7 - Combine with objectKeys for validation + +The following example uses `shallowMerge()` with [`objectKeys()`][01] and [`contains()`][05] +to validate that all expected configuration keys are present after merging. + +```yaml +# shallowMerge.example.7.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +parameters: + baseConfig: + type: object + defaultValue: + timeout: 30 + retries: 3 + overrides: + type: object + defaultValue: + timeout: 60 +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: + merged: "[shallowMerge(createArray(parameters('baseConfig'), parameters('overrides')))]" + keys: "[objectKeys(shallowMerge(createArray(parameters('baseConfig'), parameters('overrides'))))]" + hasRetries: "[contains(objectKeys(shallowMerge(createArray(parameters('baseConfig'), parameters('overrides')))), 'retries')]" +``` + +```bash +dsc config get --file shallowMerge.example.7.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + merged: + timeout: 60 + retries: 3 + keys: + - timeout + - retries + hasRetries: true +messages: [] +hadErrors: false +``` + +This pattern ensures the merged configuration includes required properties. + +### Example 8 - Empty array returns empty object + +The following example shows that `shallowMerge()` returns an empty object when given an +empty array. It uses [`objectKeys()`][01], [`length()`][06], and [`equals()`][07] to verify +the result is empty. + +```yaml +# shallowMerge.example.8.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: + result: "[shallowMerge(createArray())]" + isEmpty: "[equals(length(objectKeys(shallowMerge(createArray()))), 0)]" +``` + +```bash +dsc config get --file shallowMerge.example.8.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + result: {} + isEmpty: true +messages: [] +hadErrors: false +``` + +## Parameters + +### inputArray + +An array of objects to merge. Each element in the array should be an object. Non-object +elements in the array are silently ignored during the merge process. + +```yaml +Type: array +Required: true +Position: 1 +``` + +## Output + +Returns a single object containing all properties from the input objects. When the same +property appears in multiple objects, the value from the last object in the array is used. + +```yaml +Type: object +``` + +## Error conditions + +The function will return an error in the following cases: + +- **Not an array**: The input is not an array (e.g., object, string, number, null) + +## Notes + +- This is a **shallow merge** - nested objects are replaced, not merged recursively +- Properties from objects later in the array override properties from earlier objects +- Empty objects in the array don't affect the merge +- Non-object elements in the array are ignored +- An empty array returns an empty object +- The function processes objects in array order, so the last object has highest precedence +- For recursive/deep merging of nested objects, consider using [`union()`][00] instead + +## Related functions + +- [`union()`][00] - Combines arrays or performs deep merge of objects +- [`createArray()`][02] - Creates an array from values +- [`createObject()`][03] - Creates an object from key-value pairs +- [`objectKeys()`][01] - Returns an array of keys from an object +- [`items()`][04] - Converts an object to an array of key-value pairs +- [`contains()`][05] - Checks if an array contains a specific value +- [`length()`][06] - Returns the number of elements in an array +- [`equals()`][07] - Compares two values for equality + + +[00]: ./union.md +[01]: ./objectKeys.md +[02]: ./createArray.md +[03]: ./createObject.md +[04]: ./items.md +[05]: ./contains.md +[06]: ./length.md +[07]: ./equals.md diff --git a/docs/reference/schemas/config/functions/tryWhich.md b/docs/reference/schemas/config/functions/tryWhich.md new file mode 100644 index 000000000..2da29d5d8 --- /dev/null +++ b/docs/reference/schemas/config/functions/tryWhich.md @@ -0,0 +1,211 @@ +--- +description: Reference for the 'tryWhich' DSC configuration document function +ms.date: 11/19/2025 +ms.topic: reference +title: tryWhich +--- + +## Synopsis + +Attempts to locate an executable in the system PATH and returns its full path, or null if not found. + +## Syntax + +```Syntax +tryWhich() +``` + +## Description + +The `tryWhich()` function searches for an executable in the system's PATH environment variable +and returns the full path to the executable if found. If the executable is not found, the +function returns `null` instead of generating an error. + +This function is useful for: + +- Checking if a required command-line tool is available before using it +- Conditionally configuring resources based on available system tools +- Validating prerequisites in configurations +- Finding the exact path to executables for use in scripts or commands + +The function searches the PATH in the same way the operating system would when executing a +command. On Windows, it automatically checks for common executable extensions (.exe, .cmd, +.bat, etc.). + +Unlike a strict path lookup that would fail if the executable is missing, `tryWhich()` +gracefully returns `null`, making it ideal for conditional logic with [`if()`][00] or +[`coalesce()`][01]. + +## Examples + +### Example 1 - Check if tool exists before using it + +The following example uses `tryWhich()` with [`if()`][00] to conditionally set a property +based on whether the `git` command is available. + +```yaml +# tryWhich.example.1.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: + gitPath: "[tryWhich('git')]" + hasGit: "[if(equals(tryWhich('git'), null()), false(), true())]" +``` + +```bash +dsc config get --file tryWhich.example.1.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + gitPath: /usr/bin/git + hasGit: true +messages: [] +hadErrors: false +``` + +If `git` is not installed, `gitPath` would be `null` and `hasGit` would be `false`. + +### Example 2 - Provide fallback paths with coalesce + +The following example uses `tryWhich()` with [`coalesce()`][01] to provide fallback options +when searching for an executable, returning the first non-null path found. + +```yaml +# tryWhich.example.2.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: + pythonPath: "[coalesce(tryWhich('python3'), tryWhich('python'), '/usr/bin/python')]" +``` + +```bash +dsc config get --file tryWhich.example.2.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + pythonPath: /usr/bin/python3 +messages: [] +hadErrors: false +``` + +This tries `python3` first, falls back to `python`, and finally uses a default path if +neither is found in PATH. + +### Example 3 - Validate multiple prerequisites + +The following example demonstrates checking for multiple required tools and building a status +report using [`createObject()`][02]. + +```yaml +# tryWhich.example.3.dsc.config.yaml +$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json +resources: +- name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: + prerequisites: + docker: "[tryWhich('docker')]" + kubectl: "[tryWhich('kubectl')]" + helm: "[tryWhich('helm')]" + allFound: "[and(not(equals(tryWhich('docker'), null())), not(equals(tryWhich('kubectl'), null())), not(equals(tryWhich('helm'), null())))]" +``` + +```bash +dsc config get --file tryWhich.example.3.dsc.config.yaml +``` + +```yaml +results: +- name: Echo + type: Microsoft.DSC.Debug/Echo + result: + actualState: + output: + prerequisites: + docker: /usr/bin/docker + kubectl: /usr/local/bin/kubectl + helm: null + allFound: false +messages: [] +hadErrors: false +``` + +This checks for three tools and determines if all are available. In this example, `helm` is +not found, so `allFound` is `false`. + +## Parameters + +### commandName + +The name of the executable to locate. On Windows, the function automatically checks for +common executable extensions (.exe, .cmd, .bat, .ps1, etc.) if no extension is provided. + +```yaml +Type: string +Required: true +Position: 1 +``` + +## Output + +Returns the full path to the executable as a string if found in the system PATH. Returns +`null` if the executable is not found. + +```yaml +Type: string or null +``` + +## Error conditions + +The function returns `null` instead of generating errors when the executable is not found. +It will return an error only if: + +- **Not a string**: The input is not a string (e.g., number, array, object, null) + +## Notes + +- The function searches the PATH environment variable in the same order as the operating system +- On Windows, common executable extensions are automatically checked (.exe, .cmd, .bat, .ps1, etc.) +- Returns `null` (not an error) when the executable is not found +- The returned path is always absolute +- Use with [`if()`][00] or [`coalesce()`][01] for conditional logic based on tool availability +- The search is case-insensitive on Windows and case-sensitive on Unix-like systems +- Symbolic links are resolved to their target paths + +## Related functions + +- [`if()`][00] - Conditional expression for checking if a tool exists +- [`coalesce()`][01] - Returns the first non-null value from a list +- [`equals()`][03] - Compares values for equality +- [`null()`][04] - Returns a null value +- [`and()`][05] - Logical AND for checking multiple conditions +- [`not()`][06] - Logical NOT for negating conditions +- [`createObject()`][02] - Creates an object from key-value pairs + + +[00]: ./if.md +[01]: ./coalesce.md +[02]: ./createObject.md +[03]: ./equals.md +[04]: ./null.md +[05]: ./and.md +[06]: ./not.md diff --git a/dsc/tests/dsc_functions.tests.ps1 b/dsc/tests/dsc_functions.tests.ps1 index 3da03441e..453bc5767 100644 --- a/dsc/tests/dsc_functions.tests.ps1 +++ b/dsc/tests/dsc_functions.tests.ps1 @@ -998,6 +998,103 @@ Describe 'tests for function expressions' { $out.results[0].result.actualState.output | Should -Be $expected } + It 'shallowMerge function basic usage: ' -TestCases @( + @{ expression = "[shallowMerge(createArray(createObject('one', 'a'), createObject('two', 'b'), createObject('two', 'c')))]"; expected = [pscustomobject]@{ one = 'a'; two = 'c' } } + @{ expression = "[shallowMerge(createArray(createObject('a', 1, 'b', 2), createObject('b', 3, 'c', 4)))]"; expected = [pscustomobject]@{ a = 1; b = 3; c = 4 } } + @{ expression = "[shallowMerge(createArray(createObject('name', 'John', 'age', 30)))]"; expected = [pscustomobject]@{ name = 'John'; age = 30 } } + @{ expression = "[shallowMerge(createArray())]"; expected = [pscustomobject]@{} } + ) { + param($expression, $expected) + + $escapedExpression = $expression -replace "'", "''" + $config_yaml = @" + `$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json + resources: + - name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: '$escapedExpression' +"@ + $out = $config_yaml | dsc config get -f - | ConvertFrom-Json + $result = $out.results[0].result.actualState.output + + if ($expected -is [PSCustomObject]) { + $expectedHash = @{} + $expected.PSObject.Properties | ForEach-Object { $expectedHash[$_.Name] = $_.Value } + + $resultHash = @{} + $result.PSObject.Properties | ForEach-Object { $resultHash[$_.Name] = $_.Value } + + $resultHash.Count | Should -Be $expectedHash.Count + foreach ($key in $expectedHash.Keys) { + $resultHash[$key] | Should -Be $expectedHash[$key] + } + } + } + + It 'shallowMerge function with nested objects: ' -TestCases @( + @{ expression = "[shallowMerge(createArray(createObject('one', 'a', 'nested', createObject('a', 1, 'nested', createObject('c', 3))), createObject('two', 'b', 'nested', createObject('b', 2))))]"; expectedKeys = @('one', 'two', 'nested'); nestedKeys = @('b') } + @{ expression = "[shallowMerge(createArray(createObject('nested', createObject('x', 1, 'y', 2)), createObject('nested', createObject('z', 3))))]"; expectedKeys = @('nested'); nestedKeys = @('z') } + ) { + param($expression, $expectedKeys, $nestedKeys) + + $escapedExpression = $expression -replace "'", "''" + $config_yaml = @" + `$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json + resources: + - name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: '$escapedExpression' +"@ + $out = $config_yaml | dsc config get -f - | ConvertFrom-Json + $result = $out.results[0].result.actualState.output + + # Verify top-level keys + $result.PSObject.Properties.Name | Should -HaveCount $expectedKeys.Count + foreach ($key in $expectedKeys) { + $result.PSObject.Properties.Name | Should -Contain $key + } + + # Verify nested object was completely replaced (shallow merge) + if ($nestedKeys.Count -gt 0) { + $result.nested.PSObject.Properties.Name | Should -HaveCount $nestedKeys.Count + foreach ($key in $nestedKeys) { + $result.nested.PSObject.Properties.Name | Should -Contain $key + } + } + } + + It 'shallowMerge function with multiple objects: ' -TestCases @( + @{ expression = "[shallowMerge(createArray(createObject('a', 1), createObject('b', 2), createObject('c', 3), createObject('d', 4)))]"; expectedKeys = @('a', 'b', 'c', 'd') } + @{ expression = "[length(objectKeys(shallowMerge(createArray(createObject('x', 10), createObject('y', 20)))))]"; expected = 2 } + ) { + param($expression, $expectedKeys, $expected) + + $escapedExpression = $expression -replace "'", "''" + $config_yaml = @" + `$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json + resources: + - name: Echo + type: Microsoft.DSC.Debug/Echo + properties: + output: '$escapedExpression' +"@ + $out = $config_yaml | dsc config get -f - | ConvertFrom-Json + $result = $out.results[0].result.actualState.output + + if ($expectedKeys) { + $result.PSObject.Properties.Name | Should -HaveCount $expectedKeys.Count + foreach ($key in $expectedKeys) { + $result.PSObject.Properties.Name | Should -Contain $key + } + } + + if ($expected) { + $result | Should -Be $expected + } + } + It 'tryGet() function works for: ' -TestCases @( @{ expression = "[tryGet(createObject('a', 1, 'b', 2), 'a')]"; expected = 1 } @{ expression = "[tryGet(createObject('a', 1, 'b', 2), 'c')]"; expected = $null } diff --git a/lib/dsc-lib/locales/en-us.toml b/lib/dsc-lib/locales/en-us.toml index 48e1450f3..94c3d0a6c 100644 --- a/lib/dsc-lib/locales/en-us.toml +++ b/lib/dsc-lib/locales/en-us.toml @@ -528,6 +528,9 @@ extensionReturnedError = "Extension '%{extension}': %{error}" noExtensions = "No extensions supporting secrets was found" secretNotFound = "Secret '%{name}' not found" +[functions.shallowMerge] +description = "Combines an array of objects where only the top-level objects are merged" + [functions.startsWith] description = "Checks if a string starts with a specific prefix" invoked = "startsWith function" diff --git a/lib/dsc-lib/src/functions/mod.rs b/lib/dsc-lib/src/functions/mod.rs index bd4f3eaf3..a695438a5 100644 --- a/lib/dsc-lib/src/functions/mod.rs +++ b/lib/dsc-lib/src/functions/mod.rs @@ -64,6 +64,7 @@ pub mod range; pub mod reference; pub mod resource_id; pub mod secret; +pub mod shallow_merge; pub mod skip; pub mod starts_with; pub mod string; @@ -200,6 +201,7 @@ impl FunctionDispatcher { Box::new(reference::Reference{}), Box::new(resource_id::ResourceId{}), Box::new(secret::Secret{}), + Box::new(shallow_merge::ShallowMerge{}), Box::new(skip::Skip{}), Box::new(starts_with::StartsWith{}), Box::new(string::StringFn{}), diff --git a/lib/dsc-lib/src/functions/shallow_merge.rs b/lib/dsc-lib/src/functions/shallow_merge.rs new file mode 100644 index 000000000..bc81d5e92 --- /dev/null +++ b/lib/dsc-lib/src/functions/shallow_merge.rs @@ -0,0 +1,162 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use crate::configure::context::Context; +use crate::functions::{Function, FunctionArgKind, FunctionCategory, FunctionMetadata}; +use crate::DscError; +use rust_i18n::t; +use serde_json::{Map, Value}; + +#[derive(Debug, Default)] +pub struct ShallowMerge {} + +impl Function for ShallowMerge { + fn get_metadata(&self) -> FunctionMetadata { + FunctionMetadata { + name: "shallowMerge".to_string(), + description: t!("functions.shallowMerge.description").to_string(), + category: vec![FunctionCategory::Object], + min_args: 1, + max_args: 1, + accepted_arg_ordered_types: vec![vec![FunctionArgKind::Array]], + remaining_arg_accepted_types: None, + return_types: vec![FunctionArgKind::Object], + } + } + + fn invoke(&self, args: &[Value], _context: &Context) -> Result { + let array = args[0].as_array().unwrap(); + + let mut result = Map::new(); + + for item in array { + if let Some(obj) = item.as_object() { + for (key, value) in obj { + // Shallow merge: replace the entire value, even if it's a nested object + result.insert(key.clone(), value.clone()); + } + } + } + + Ok(Value::Object(result)) + } +} + +#[cfg(test)] +mod tests { + use crate::configure::context::Context; + use crate::parser::Statement; + use serde_json::json; + + #[test] + fn shallow_merge_basic() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse_and_execute( + "[shallowMerge(createArray(createObject('one', 'a'), createObject('two', 'b'), createObject('two', 'c')))]", + &Context::new() + ).unwrap(); + + assert_eq!(result, json!({"one": "a", "two": "c"})); + } + + #[test] + fn shallow_merge_with_nested_objects() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse_and_execute( + "[shallowMerge(createArray(createObject('one', 'a', 'nested', createObject('a', 1, 'nested', createObject('c', 3))), createObject('two', 'b', 'nested', createObject('b', 2))))]", + &Context::new() + ).unwrap(); + + // The nested object should be completely replaced, not merged + assert_eq!(result, json!({"one": "a", "nested": {"b": 2}, "two": "b"})); + } + + #[test] + fn shallow_merge_empty_array() { + let mut parser = Statement::new().unwrap(); + let result = parser + .parse_and_execute("[shallowMerge(createArray())]", &Context::new()) + .unwrap(); + + assert_eq!(result, json!({})); + } + + #[test] + fn shallow_merge_single_object() { + let mut parser = Statement::new().unwrap(); + let result = parser + .parse_and_execute( + "[shallowMerge(createArray(createObject('name', 'John', 'age', 30)))]", + &Context::new(), + ) + .unwrap(); + + assert_eq!(result, json!({"name": "John", "age": 30})); + } + + #[test] + fn shallow_merge_overwrite_primitives() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse_and_execute( + "[shallowMerge(createArray(createObject('a', 1, 'b', 2), createObject('b', 3, 'c', 4)))]", + &Context::new() + ).unwrap(); + + assert_eq!(result, json!({"a": 1, "b": 3, "c": 4})); + } + + #[test] + fn shallow_merge_multiple_objects() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse_and_execute( + "[shallowMerge(createArray(createObject('a', 1), createObject('b', 2), createObject('c', 3), createObject('d', 4)))]", + &Context::new() + ).unwrap(); + + assert_eq!(result, json!({"a": 1, "b": 2, "c": 3, "d": 4})); + } + + #[test] + fn shallow_merge_replaces_nested_completely() { + let mut parser = Statement::new().unwrap(); + // First object has nested.x and nested.y, second has nested.z + // Result should only have nested.z (complete replacement) + let result = parser.parse_and_execute( + "[shallowMerge(createArray(createObject('nested', createObject('x', 1, 'y', 2)), createObject('nested', createObject('z', 3))))]", + &Context::new() + ).unwrap(); + + assert_eq!(result, json!({"nested": {"z": 3}})); + } + + #[test] + fn shallow_merge_mixed_types() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse_and_execute( + "[shallowMerge(createArray(createObject('str', 'text', 'num', 42), createObject('bool', true(), 'arr', createArray(1, 2, 3))))]", + &Context::new() + ).unwrap(); + + assert_eq!( + result, + json!({"str": "text", "num": 42, "bool": true, "arr": [1, 2, 3]}) + ); + } + + #[test] + fn shallow_merge_not_array_error() { + let mut parser = Statement::new().unwrap(); + let result = parser.parse_and_execute("[shallowMerge('not an array')]", &Context::new()); + + assert!(result.is_err()); + } + + #[test] + fn shallow_merge_object_error() { + let mut parser = Statement::new().unwrap(); + let result = + parser.parse_and_execute("[shallowMerge(createObject('a', 1))]", &Context::new()); + + assert!(result.is_err()); + } +} From 132908a47ac82a4fab0cf3a60e61c56b4a754d83 Mon Sep 17 00:00:00 2001 From: "G.Reijn" Date: Thu, 20 Nov 2025 02:35:48 +0100 Subject: [PATCH 2/2] Apply suggestions Mikey --- .../schemas/config/functions/shallowMerge.md | 136 +++++++++++++----- .../schemas/config/functions/tryWhich.md | 78 ++++++---- 2 files changed, 152 insertions(+), 62 deletions(-) diff --git a/docs/reference/schemas/config/functions/shallowMerge.md b/docs/reference/schemas/config/functions/shallowMerge.md index 7f6bc035e..272be1673 100644 --- a/docs/reference/schemas/config/functions/shallowMerge.md +++ b/docs/reference/schemas/config/functions/shallowMerge.md @@ -19,25 +19,30 @@ shallowMerge() The `shallowMerge()` function takes an array of objects and combines them into a single object by merging their properties. When the same property name appears in multiple objects, -the value from the last object in the array takes precedence. +the value from the last object in the array with that property takes precedence. -This is a **shallow merge**, meaning: +This is a _shallow merge_, which applies the following rules: -- Top-level properties are merged from all objects -- If a property value is an object, it replaces the entire object from previous objects - rather than merging the nested properties -- Arrays and other complex types are also replaced entirely, not combined +- The first object in the array defines the base value for the merged object. +- The function processes each object in the array in the order they're defined. +- When processing each object, the function iterates over every top-level property defined for that + object and: + + - If the merged object doesn't already have the property, the function adds that property to the + merged object with the value from the current object. + - If the merged object does have the property, the function _replaces_ the existing value with + the value from the current object, even when the value is an object or array. This function is useful for: -- Building composite configuration objects from multiple sources -- Applying configuration overrides where later values take precedence -- Combining default settings with user-specified customizations -- Merging environment-specific configurations +- Building composite configuration objects from multiple sources. +- Applying configuration overrides where later values take precedence. +- Combining default settings with user-specified customizations. +- Merging environment-specific configurations. -The shallow merge behavior differs from a deep merge (like [`union()`][00]) where nested -objects would be recursively merged. With `shallowMerge()`, nested structures are replaced -entirely by the last object's value. +The shallow merge behavior differs from a deep merge (like [`union()`][00]) where nested +objects are recursively merged. The `shallowMerge()` function replaces nested structures +entirely with the value defined by the last object with that property in the input array. ## Examples @@ -53,7 +58,13 @@ resources: - name: Echo type: Microsoft.DSC.Debug/Echo properties: - output: "[shallowMerge(createArray(createObject('host', 'localhost', 'port', 8080), createObject('port', 9000, 'ssl', true())))]" + output: >- + [shallowMerge( + createArray( + createObject('host', 'localhost', 'port', 8080), + createObject('port', 9000, 'ssl', true()) + ) + )] ``` ```bash @@ -74,9 +85,8 @@ messages: [] hadErrors: false ``` -Notice how the `port` value from the second object (9000) replaces the value from the first -object (8080), while properties that only exist in one object (`host` and `ssl`) are -preserved. +In this example, the `port` value from the second object (`9000`) replaces the value from the first +object (`8080`), while properties that only exist in one object (`host` and `ssl`) are preserved. ### Example 2 - Apply multiple configuration layers @@ -105,7 +115,14 @@ resources: - name: Echo type: Microsoft.DSC.Debug/Echo properties: - output: "[shallowMerge(createArray(parameters('defaults'), parameters('environment'), parameters('userPrefs')))]" + output: >- + [shallowMerge( + createArray( + parameters('defaults'), + parameters('environment'), + parameters('userPrefs') + ) + )] ``` ```bash @@ -141,7 +158,19 @@ resources: - name: Echo type: Microsoft.DSC.Debug/Echo properties: - output: "[shallowMerge(createArray(createObject('database', createObject('host', 'localhost', 'port', 5432, 'ssl', true())), createObject('database', createObject('host', 'prod.db.local'))))]" + output: >- + [shallowMerge( + createArray( + createObject( + 'database', + createObject('host', 'localhost', 'port', 5432, 'ssl', true()) + ), + createObject( + 'database', + createObject('host', 'prod.db.local') + ) + ) + )] ``` ```bash @@ -175,7 +204,14 @@ resources: - name: Echo type: Microsoft.DSC.Debug/Echo properties: - output: "[shallowMerge(createArray(createObject('name', 'Service1', 'enabled', true()), createObject(), createObject('version', '2.0')))]" + output: >- + [shallowMerge( + createArray( + createObject('name', 'Service1', 'enabled', true()), + createObject(), + createObject('version', '2.0') + ) + )] ``` ```bash @@ -210,7 +246,13 @@ resources: - name: Echo type: Microsoft.DSC.Debug/Echo properties: - output: "[shallowMerge(createArray(createObject('newUI', false(), 'darkMode', true(), 'beta', false()), createObject('newUI', true()), createObject('beta', true())))]" + output: >- + [shallowMerge( + createArray( + createObject('newUI', false(), 'darkMode', true(), 'beta', false()), + createObject('newUI', true()), createObject('beta', true()) + ) + )] ``` ```bash @@ -302,9 +344,34 @@ resources: type: Microsoft.DSC.Debug/Echo properties: output: - merged: "[shallowMerge(createArray(parameters('baseConfig'), parameters('overrides')))]" - keys: "[objectKeys(shallowMerge(createArray(parameters('baseConfig'), parameters('overrides'))))]" - hasRetries: "[contains(objectKeys(shallowMerge(createArray(parameters('baseConfig'), parameters('overrides')))), 'retries')]" + merged: >- + [shallowMerge( + createArray( + parameters('baseConfig'), + parameters('overrides') + ) + )] + keys: >- + [objectKeys( + shallowMerge( + createArray( + parameters('baseConfig'), + parameters('overrides') + ) + ) + )] + hasRetries: >- + [contains( + objectKeys( + shallowMerge( + createArray( + parameters('baseConfig'), + parameters('overrides') + ) + ) + ), + 'retries' + )] ``` ```bash @@ -381,8 +448,9 @@ Position: 1 ## Output -Returns a single object containing all properties from the input objects. When the same -property appears in multiple objects, the value from the last object in the array is used. +Returns a single object containing all properties from the input objects. When the same property +appears in multiple objects, the value from the last object in the array with that property is +retained, replacing all prior values for the property. ```yaml Type: object @@ -396,13 +464,15 @@ The function will return an error in the following cases: ## Notes -- This is a **shallow merge** - nested objects are replaced, not merged recursively -- Properties from objects later in the array override properties from earlier objects -- Empty objects in the array don't affect the merge -- Non-object elements in the array are ignored -- An empty array returns an empty object -- The function processes objects in array order, so the last object has highest precedence -- For recursive/deep merging of nested objects, consider using [`union()`][00] instead +- This function performs a _shallow merge_ - the function replaces nested objects, it doesn't merge + them recursively. +- The function replaces the value for properties defined by earlier objects in the input array with + the value from objects later in the array. +- The function ignores empty objects in the input array. +- The function ignores non-object elements in the input array. +- The function returns an empty object when the input is an empty array. +- The function processes objects in array order, so the last object has highest precedence +- For recursive/deep merging of nested objects, consider using [`union()`][00] instead. ## Related functions diff --git a/docs/reference/schemas/config/functions/tryWhich.md b/docs/reference/schemas/config/functions/tryWhich.md index 2da29d5d8..340491adc 100644 --- a/docs/reference/schemas/config/functions/tryWhich.md +++ b/docs/reference/schemas/config/functions/tryWhich.md @@ -7,7 +7,8 @@ title: tryWhich ## Synopsis -Attempts to locate an executable in the system PATH and returns its full path, or null if not found. +Looks for an executable in the `PATH` environment variable and returns the full path to that +executable or null if not found. ## Syntax @@ -17,20 +18,20 @@ tryWhich() ## Description -The `tryWhich()` function searches for an executable in the system's PATH environment variable -and returns the full path to the executable if found. If the executable is not found, the -function returns `null` instead of generating an error. +The `tryWhich()` function searches for an executable in the `PATH` environment variable and returns +the full path to the executable if found. If the executable isn't discoverable, the function +returns `null` instead of generating an error. This function is useful for: -- Checking if a required command-line tool is available before using it -- Conditionally configuring resources based on available system tools -- Validating prerequisites in configurations -- Finding the exact path to executables for use in scripts or commands +- Checking whether a required command-line tool is available before invoking it. +- Conditionally configuring resources based on available system tools. +- Validating prerequisites in configurations. +- Finding the exact path to executables for use in scripts or commands. -The function searches the PATH in the same way the operating system would when executing a -command. On Windows, it automatically checks for common executable extensions (.exe, .cmd, -.bat, etc.). +The function searches the `PATH` in the same way the operating system would when executing a +command. On Windows, it automatically checks for common executable extensions, like `.exe`, `.cmd`, +and `.bat`, if no extension is provided. Unlike a strict path lookup that would fail if the executable is missing, `tryWhich()` gracefully returns `null`, making it ideal for conditional logic with [`if()`][00] or @@ -52,7 +53,12 @@ resources: properties: output: gitPath: "[tryWhich('git')]" - hasGit: "[if(equals(tryWhich('git'), null()), false(), true())]" + hasGit: >- + [if( + equals(tryWhich('git'), null()), + false(), + true() + )] ``` ```bash @@ -72,7 +78,8 @@ messages: [] hadErrors: false ``` -If `git` is not installed, `gitPath` would be `null` and `hasGit` would be `false`. +If `git` wasn't discoverable in the `PATH` environmental variable, `gitPath` would be `null` and `hasGit` +would be `false`. ### Example 2 - Provide fallback paths with coalesce @@ -87,7 +94,12 @@ resources: type: Microsoft.DSC.Debug/Echo properties: output: - pythonPath: "[coalesce(tryWhich('python3'), tryWhich('python'), '/usr/bin/python')]" + pythonPath: >- + [coalesce( + tryWhich('python3'), + tryWhich('python'), + '/usr/bin/python' + )] ``` ```bash @@ -106,8 +118,9 @@ messages: [] hadErrors: false ``` -This tries `python3` first, falls back to `python`, and finally uses a default path if -neither is found in PATH. +In this example, the function first looks for `python3` in the `PATH` environmental variable. If +that executable isn't discovered, it then looks for `python`. If neither executable is discovered, +it falls back to the specified default value, `/usr/bin/python3`. ### Example 3 - Validate multiple prerequisites @@ -126,7 +139,12 @@ resources: docker: "[tryWhich('docker')]" kubectl: "[tryWhich('kubectl')]" helm: "[tryWhich('helm')]" - allFound: "[and(not(equals(tryWhich('docker'), null())), not(equals(tryWhich('kubectl'), null())), not(equals(tryWhich('helm'), null())))]" + allFound: >- + [and( + not(equals(tryWhich('docker'), null())), + not(equals(tryWhich('kubectl'), null())), + not(equals(tryWhich('helm'), null())) + )] ``` ```bash @@ -156,8 +174,8 @@ not found, so `allFound` is `false`. ### commandName -The name of the executable to locate. On Windows, the function automatically checks for -common executable extensions (.exe, .cmd, .bat, .ps1, etc.) if no extension is provided. +The name of the executable to locate. On Windows, it automatically checks for common executable +extensions, like `.exe`, `.cmd`, and `.bat`, if no extension is provided. ```yaml Type: string @@ -176,20 +194,22 @@ Type: string or null ## Error conditions -The function returns `null` instead of generating errors when the executable is not found. -It will return an error only if: +The function returns `null` instead of generating errors when the executable isn't found. -- **Not a string**: The input is not a string (e.g., number, array, object, null) +The function only returns an error when the input isn't a string. ## Notes -- The function searches the PATH environment variable in the same order as the operating system -- On Windows, common executable extensions are automatically checked (.exe, .cmd, .bat, .ps1, etc.) -- Returns `null` (not an error) when the executable is not found -- The returned path is always absolute -- Use with [`if()`][00] or [`coalesce()`][01] for conditional logic based on tool availability -- The search is case-insensitive on Windows and case-sensitive on Unix-like systems -- Symbolic links are resolved to their target paths +- The function searches the `PATH` environment variable in the same order as the operating system. +- On Windows, the function automatically checks for the executable with common extensions, like + `.exe`, `.cmd`, and `.bat`, when the input string doesn't define an extension. For example, if + the input is `dsc`, the function would return `dsc.exe` if available in `PATH`. +- The function returns `null` when the executable isn't found instead of raising an error. +- The function always returns the absolute path to a discovered executable. +- Use with [`if()`][00] or [`coalesce()`][01] for conditional logic based on tool availability. +- The function searches for the executable case-insensitively on Windows and case-sensitively on + other platforms. +- The function resolves symbolic links to their target paths. ## Related functions