Build a Simple ETL Pipeline (MLOps) #550
Conversation
moe18
left a comment
moe18
left a comment
There was a problem hiding this comment.
here are some small comments on the ETL question
| @@ -0,0 +1,12 @@ | |||
| { | |||
| "id": "184", | |||
| @@ -0,0 +1,14 @@ | |||
| [ | |||
| { | |||
| "test": "from solution import run_etl; print(run_etl('user_id,event_type,value\n u1, purchase, 10.0\n u2, view, 1.0\n u1, purchase, 5\n u3, purchase, not_a_number\n u2, purchase, 3.5 \n'))", | |||
There was a problem hiding this comment.
for processing our test cases we need the \n to be \n
There was a problem hiding this comment.
can you elaborate on this?
There was a problem hiding this comment.
I am not exactly sure what do you mean by "need the \n to be \n", did you mean something else?
There was a problem hiding this comment.
sorry for the confusion it should look like this it needs two '' 's for example print(run_etl('user_id,event_type,value\\n u1, purchase, 10.0\\n u2, view, 1.0\\n u1, purchase, 5\\n u3, purchase, not_a_number\\n u2, purchase, 3.5 \\n'))
| @@ -0,0 +1,14 @@ | |||
| [ | |||
| { | |||
| "test": "from solution import run_etl; print(run_etl('user_id,event_type,value\n u1, purchase, 10.0\n u2, view, 1.0\n u1, purchase, 5\n u3, purchase, not_a_number\n u2, purchase, 3.5 \n'))", | |||
There was a problem hiding this comment.
you also do not need to import run_etl the test could look like
print(run_etl('user_id,event_type,value\n u1, purchase, 10.0\n u2, view, 1.0\n u1, purchase, 5\n u3, purchase, not_a_number\n u2, purchase, 3.5 \n'))
| } | ||
| ], | ||
| "description": "## Problem\n\nImplement a basic data drift check comparing two numeric datasets (reference vs. current).\n\nWrite a function `check_drift(ref, cur, mean_threshold, var_threshold)` that:\n\n- Accepts two lists of numbers `ref` and `cur`.\n- Computes the absolute difference in means and variances.\n- Returns a tuple `(mean_drift, var_drift)` where each element is a boolean indicating whether drift exceeds the corresponding threshold:\n\t- `mean_drift = abs(mean(ref) - mean(cur)) > mean_threshold`\n\t- `var_drift = abs(var(ref) - var(cur)) > var_threshold`\n\nAssume population variance (divide by N). Handle empty inputs by returning `(False, False)`.", | ||
| "learn_section": "## Solution Explanation\n\nWe compare two numeric samples (reference vs. current) using mean and variance with user-defined thresholds.\n\n### Definitions\n- Mean: \\( \\mu = \\frac{1}{N}\\sum_i x_i \\)\n- Population variance: \\( \\sigma^2 = \\frac{1}{N}\\sum_i (x_i - \\mu)^2 \\)\n\n### Drift rules\n- Mean drift if \\(|\\mu_{ref} - \\mu_{cur}| > \\tau_{mean}\\)\n- Variance drift if \\(|\\sigma^2_{ref} - \\sigma^2_{cur}| > \\tau_{var}\\)\n\n### Edge cases\n- If either sample is empty, return `(False, False)` to avoid false alarms.\n- Population vs. sample variance: we use population here to match many monitoring setups. Either is fine if used consistently.\n\n### Complexity\n- O(N + M) to compute stats; O(1) extra space.", |
There was a problem hiding this comment.
use $ and $$ for equations in the learn section
| }, | ||
| "test_cases": [ | ||
| { | ||
| "test": "from solution import check_drift; print(check_drift([1,2,3], [1.1,2.2,3.3], 0.05, 0.1))", |
There was a problem hiding this comment.
no need to import anything from solution, the test case could just be
print(check_drift([1,2,3], [1.1,2.2,3.3], 0.05, 0.1))
2 participants
No description provided.