Skip to content

refactor: use crustdata enrichment source only for new member profiles (CM-924)#3830

Merged
skwowet merged 7 commits intomainfrom
improve/CM-924
Feb 9, 2026
Merged

refactor: use crustdata enrichment source only for new member profiles (CM-924)#3830
skwowet merged 7 commits intomainfrom
improve/CM-924

Conversation

@skwowet
Copy link
Member

@skwowet skwowet commented Feb 9, 2026

What changed

  • Crustdata now runs only once per member. In isEnrichableBySource, we check if a Crustdata cache row already exists for the member (regardless of whether data is null or not). If it does, the member has already been through Crustdata and we skip it.
  • Preserved existing cache data on re-enrichment. In enrichMember, when a source returns null but the cache already holds valid data, we now refresh the cache timestamp instead of overwriting with null. This prevents Crustdata data from being wiped when its cache becomes obsolete after 90 days and the source intentionally skips the member.

Note

Medium Risk
Changes enrichment eligibility and cache update behavior, which can affect which sources run and whether cached profiles are retained; risk is moderated by being localized to the enrichment worker and primarily gatekeeping/DB-write logic.

Overview
Crustdata enrichment is now one-shot per member: isEnrichableBySource loads cache rows even when data is null and skips Crustdata entirely if any Crustdata cache row already exists for that member.

The enrichment workflow now avoids wiping valid cached data: when a source returns null but the cache has non-null data, it only updates cache.updatedAt instead of overwriting the cache with null (supporting sources that intentionally skip members).

Written by Cursor Bugbot for commit 5c17650. This will update automatically on new commits. Configure here.

@skwowet skwowet self-assigned this Feb 9, 2026
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

@skwowet skwowet merged commit 299d0f3 into main Feb 9, 2026
15 checks passed
@skwowet skwowet deleted the improve/CM-924 branch February 9, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant