chore(shredder): propagate subplat deletes to cirrus#8401
chore(shredder): propagate subplat deletes to cirrus#8401mikewilli merged 2 commits intomozilla:mainfrom
Conversation
|
this is blocked on mozilla/probe-scraper#963 creating the targeted tables |
|
as I understand it, this is not blocked on https://github.com/mozilla-services/cloudops-infra/pull/6696, because the table has already been created |
mikewilli
left a comment
There was a problem hiding this comment.
Typo fix but otherwise LGTM
...data-shared-prod/subscription_platform_backend_cirrus_derived/delete_events_v1/metadata.yaml
Outdated
Show resolved
Hide resolved
...-fx-data-shared-prod/subscription_platform_backend_cirrus_derived/delete_events_v1/query.sql
Show resolved
Hide resolved
sean-rose
left a comment
There was a problem hiding this comment.
I belatedly thought of a couple more things.
Also, I'm curious whether you've been able to test this ETL code yourself to verify it works end to end. If not, we'll probably want to be careful to deploy this at a time when you're available to quick-fix any issues that may arise.
...-fx-data-shared-prod/subscription_platform_backend_cirrus_derived/delete_events_v1/query.sql
Show resolved
Hide resolved
| CROSS JOIN | ||
| subplat_namespaces | ||
| WHERE | ||
| DATE(delete_events.submission_timestamp) = @submission_date |
There was a problem hiding this comment.
question (blocking): Is it OK for this table to only have records from whenever it's deployed going forward?
If not, you'll probably want to output a different submission_timestamp condition for the table initialization query using Jinja and the is_init() function, including the appropriate historical range of dates.
There was a problem hiding this comment.
going forward is fine, we can truncate the impacted tables to avoid complicating this, esp knowing this is supposed to be "temporary" until some dev time is available after the EoY code freeze
I lack permissions to verify that getting secret values works as intended, I have manually confirmed the uid translation works as intended, as long as it's after 9am PST i'm available to fix |
|
The However, one easy way to unblock deployment of |
|
split out #8548 |
Description
This PR adds shredder config for using fxa delete events to delete subplat cirrus pings,
and adds experiment monitoring config for experimenter_cirrus and subplat_cirrus
Related Tickets & Documents
Reviewer, please follow this checklist