[AQUA] Improve Handling of Default SMM Parameters for Model Deployment

[Aryanag2]

Improve Handling of Default SMM Parameters for Model Deployment (UI & Backend)

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[mrDzurb]

Will this cover the case where a user removes all default params in the UI? Which params will be applied then? The expectation is that SMM default params should not be used in that scenario. Basically only service params coming from the container level should be used in this case.

[Aryanag2]

I added a flag user_explicitly_cleared for this scenario
user_explicitly_cleared = model.params is not None and not model.params

and default params are only applied if
if not user_params and not user_explicitly_cleared:

Scenario: User clears all params in UI → UI sends "params": {}

model.params = {} (empty dict from UI)
user_params = build_params_string({}) → "" (empty string)
user_explicitly_cleared = model.params is not None and not model.params → True and True → True
if not user_params and not user_explicitly_cleared: → if True and False: → False → Skip loading SMM defaults
final_model_params = "" (stays empty, no SMM defaults loaded)
params = f"{params} {final_model_params}".strip() → just the container params

[Aryanag2]

Note: --served-model-name odsc-llm --disable-custom-all-reduce --seed 42 --trust-request-chat-template will be present in all scenarios since they are container level params

All test cases are for single model deployment of Llama-3.2-3B on A10.1 shape
Default SMM config for this model is "--max-model-len 65536"

Test Case 1: Default Behavior
Scenario: The user provides no parameters. Expected Result: The backend loads the system default (--max-model-len 65536).

ads aqua deployment create \
  --container_image_uri "dsmc://odsc-vllm-serving:0.10.1.21" \
  --model_id "..." \
  --instance_shape "VM.GPU.A10.1" \
  --display_name "Aqua_Test_Case1_Defaults" \
  --log_group_id "..." \
  --access_log_id "..." \
  --predict_log_id "..."
  

Test Case 2: Explicit Clear
Scenario: The user sends an empty string "" (simulating the "Clear" button). Expected Result: The backend clears optional params. The default 65536 length should disappear.

ads aqua deployment create \
  --container_image_uri "dsmc://odsc-vllm-serving:0.10.1.21" \
  --model_id "..." \
  --instance_shape "VM.GPU.A10.1" \
  --display_name "Aqua_Test_Case2_Clear" \
  --log_group_id "..." \
  --access_log_id "..." \
  --predict_log_id "..." \
  --env_var '{"PARAMS":""}'

Test Case 3: User Override
Scenario: The user provides a custom value (e.g., 4096). Expected Result: The backend uses 4096 exactly and does not merge in 65536.

ads aqua deployment create \
  --container_image_uri "dsmc://odsc-vllm-serving:0.10.1.21" \
  --model_id "..." \
  --instance_shape "VM.GPU.A10.1" \
  --display_name "Aqua_Test_Case3_Override" \
  --log_group_id "..." \
  --access_log_id "..." \
  --predict_log_id "..." \
  --env_var '{"PARAMS":"--max-model-len 4096"}'

[Aryanag2]

Note: --served-model-name odsc-llm --disable-custom-all-reduce --seed 42 --trust-request-chat-template will be present in all scenarios since they are container level params

All test cases are for multi model deployment of Llama-3.2-3B on A10.2 shape
Default SMM config for this model is "--max-model-len 65536"

Scenarios 1 & 2: Default (Case 1) vs. Explicit Clear (Case 2)
Deployed two models in a single request to test contrasting logic:

Llama_Default2: No params provided (None). Expected to load SMM defaults.

Llama_Clear2: Empty params provided ({}). Expected to not load SMM defaults.

ads aqua deployment create \                                           
  --container_image_uri "dsmc://odsc-vllm-serving:0.10.1.21" \
  --instance_shape "VM.GPU.A10.2" \
  --display_name "Aqua_MMD_Test_Default_vs_Clear" \
  --log_group_id "..." \
  --access_log_id "..." \
  --predict_log_id "..." \
  --models '[
    {
      "model_id": "...",
      "model_name": "Llama_Default2",
      "gpu_count": 1
    },
    {
      "model_id": "...",
      "model_name": "Llama_Clear2",
      "gpu_count": 1,
      "params": {}
    }
  ]'

Verification Results:

Llama_Default2: Success. Contains --max-model-len 65536.

Llama_Clear2: Success. Does not contain --max-model-len.

Scenario 3: User Override (Case 3)
Deployed a model with a specific parameter value (1024) to ensure it overrides the default (65536) without merging

ads aqua deployment create \                       
  --container_image_uri "dsmc://odsc-vllm-serving:0.10.1.21" \
  --instance_shape "VM.GPU.A10.2" \
  --display_name "Aqua_MMD_Test_Override_Final" \
  --log_group_id "..." \
  --access_log_id "..." \
  --predict_log_id "..." \
  --models '[
    {
      "model_id": "...",
      "model_name": "Llama_Override",
      "gpu_count": 1,
      "params": {"--max-model-len": "1024"}
    }
  ]'

Llama_Override: Success. Used 1024. Default 65536 was NOT merged in.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[lu-ohai]

We need to validate find_restricted_params only if deployment_params = user_input_params. Other case may cause error like when deployment_params = config_params and its not needed as this params is not provided by user.

We should put this if deployment_params: statement under line 941.

cc: @mrDzurb

[mrDzurb]

We shouldn’t allow this situation. We must validate both user-provided and default params, since some system parameters are immutable. If the defaults include restricted parameters, we should throw an error and fix the default config on our side. Ideally, we’d offer a way to bypass validation, but that option doesn’t exist today.

[lu-ohai]

I see, so in the case of defaut config, this should raise service error with clear wording instead of aqua value error?

[mrDzurb]

We shouldn’t allow this situation. We must validate both user-provided and default params, since some system parameters are immutable. If the defaults include restricted parameters, we should throw an error and fix the default config on our side. Ideally, we’d offer a way to bypass validation, but that option doesn’t exist today.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

📌 Cov diff with main:

📌 Overall coverage: