Skip to content

Revert "chore: backfill email-domain member organization dates (CM-1107)" #4063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
skwowet merged 1 commit into from
Apr 28, 2026
Merged

Revert "chore: backfill email-domain member organization dates (CM-1107)" #4063

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions backend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@
"script:refreshGithubRepoSettings": "SERVICE=script TS_NODE_TRANSPILE_ONLY=true tsx src/bin/scripts/refresh-github-repo-settings.ts",
"script:fix-duplicate-members": "SERVICE=script TS_NODE_TRANSPILE_ONLY=true tsx src/bin/scripts/fix-duplicate-members.ts",
"script:fix-members-activities-after-unaffilation": "SERVICE=script TS_NODE_TRANSPILE_ONLY=true tsx src/bin/scripts/fix-members-activities-after-unaffilation.ts",
"script:process-bot-members": "SERVICE=script TS_NODE_TRANSPILE_ONLY=true tsx src/bin/scripts/process-bot-members.ts",
"script:backfill-email-domain-member-organization-dates": "SERVICE=script TS_NODE_TRANSPILE_ONLY=true tsx src/bin/scripts/backfill-email-domain-member-organization-dates.ts"
"script:process-bot-members": "SERVICE=script TS_NODE_TRANSPILE_ONLY=true tsx src/bin/scripts/process-bot-members.ts"
},
"lint-staged": {
"**/*.ts": [
Expand Down
156 changes: 0 additions & 156 deletions backend/src/bin/scripts/backfill-email-domain-member-organization-dates.ts
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions ...ces/apps/script_executor_worker/src/activities/block-project-organization-affiliations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ export async function blockMemberOrganizationAffiliation(

export async function markMemberForAffiliationRecalc(memberIds: string[]): Promise<void> {
try {
await svc.redis.sAdd('recalculate-member-affiliations', memberIds)
await svc.redis.sAdd('queue:recalculate:members:affiliation', memberIds)
} catch (error) {
svc.log.error(error, 'Error marking member for affiliation recalc!')
throw error
Expand All @@ -67,7 +67,7 @@ export async function markMemberForAffiliationRecalc(memberIds: string[]): Promi

export async function getMembersForAffiliationRecalc(batchSize: number): Promise<string[]> {
try {
return svc.redis.sPop('recalculate-member-affiliations', batchSize)
return svc.redis.sPop('queue:recalculate:members:affiliation', batchSize)
Comment on lines 59 to +70
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change renames the Redis set key used to queue member IDs for affiliation recalculation. Any members already queued under the old key will no longer be processed (they’ll remain stranded in the old set). Consider reading/draining from both keys temporarily or adding a one-time migration/drain step before switching the producer/consumer to the new key.

Copilot uses AI. Check for mistakes.
Comment on lines 59 to +70
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR title/description indicate this is a revert of the email-domain backfill script, but this hunk introduces a Redis key rename for the affiliation recalculation queue. If this rename is intended, the PR description should mention it; otherwise it should likely be reverted as part of this revert PR.

Copilot uses AI. Check for mistakes.
} catch (error) {
svc.log.error(error, 'Error getting members for affiliation recalc!')
throw error
Expand Down
67 changes: 0 additions & 67 deletions services/libs/data-access-layer/src/members/organizations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ import {
import { EntityType } from '../old/apps/script_executor_worker/types'
import { QueryExecutor } from '../queryExecutor'

import { EmailDomainMemberOrganizationActivityDate } from './types'

/* eslint-disable @typescript-eslint/no-explicit-any */

export async function fetchMemberOrganizations(
Expand Down Expand Up @@ -61,71 +59,6 @@ export async function fetchMemberOrganizationsBySource(
)
}

export async function fetchEmailDomainMemberOrganizationsWithoutDates(
qx: QueryExecutor,
limit: number,
afterMemberId?: string,
): Promise<string[]> {
const rows = await qx.select(
`
SELECT DISTINCT "memberId"
FROM "memberOrganizations"
WHERE "source" = 'email-domain'
AND "dateStart" IS NULL
AND "dateEnd" IS NULL
AND "deletedAt" IS NULL
${afterMemberId ? `AND "memberId" > $(afterMemberId)` : ''}
ORDER BY "memberId"
LIMIT $(limit)
`,
{ limit, afterMemberId },
)

return rows.map((r) => r.memberId)
}

export async function fetchEmailDomainMemberOrganizationActivityDates(
qx: QueryExecutor,
memberId: string,
): Promise<EmailDomainMemberOrganizationActivityDate[]> {
return qx.select(
`
WITH email_domain_member_orgs AS (
SELECT DISTINCT
mo."memberId",
mo."organizationId",
lower(oi.value) AS domain
FROM "memberOrganizations" mo
INNER JOIN "organizationIdentities" oi
ON oi."organizationId" = mo."organizationId"
AND oi.type = 'primary-domain'
AND oi.verified = true
WHERE mo."memberId" = $(memberId)
AND mo."source" = 'email-domain'
AND mo."deletedAt" IS NULL
)
SELECT DISTINCT
edmo."memberId",
edmo."organizationId",
ar."timestamp"::date::text AS date
FROM email_domain_member_orgs edmo
INNER JOIN "memberIdentities" mi
ON mi."memberId" = edmo."memberId"
AND mi.verified = true
AND mi.type = 'email'
AND mi."deletedAt" IS NULL
AND lower(split_part(mi.value, '@', 2)) = edmo.domain
INNER JOIN "activityRelations" ar
ON ar."memberId" = mi."memberId"
AND ar.platform = mi.platform
AND lower(ar.username) = lower(mi.value)
AND ar."timestamp" IS NOT NULL
ORDER BY edmo."memberId", edmo."organizationId", date
`,
{ memberId },
)
}

export async function fetchOrganizationMemberIds(
qx: QueryExecutor,
organizationId: string,
Expand Down
6 changes: 1 addition & 5 deletions services/libs/data-access-layer/src/members/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { IAttributes, IMemberAttribute, MemberAttributeType, MemberOrgDate } from '@crowd/types'
import { IAttributes, IMemberAttribute, MemberAttributeType } from '@crowd/types'

export interface IQueryNumberOfNewMembers {
segmentIds?: string[]
Expand Down Expand Up @@ -93,7 +93,3 @@ export interface IDbMemberBotSuggestionBySegment {
avatarUrl: string
attributes: IAttributes
}

export interface EmailDomainMemberOrganizationActivityDate extends MemberOrgDate {
memberId: string
}
Loading