Fix 9 production error patterns with tests (~100K+ events)#3225
Draft
cursor[bot] wants to merge 2 commits intomainfrom
Draft
Fix 9 production error patterns with tests (~100K+ events)#3225cursor[bot] wants to merge 2 commits intomainfrom
cursor[bot] wants to merge 2 commits intomainfrom
Conversation
- Fix _extract_license_info None source in OpenAlex mapper (~67K events)
Source field can be explicitly null; use isinstance check
- Fix Author update_or_create MultipleObjectsReturned (~35K events)
Replace with filter().first() pattern to handle duplicates
- Fix openalex_util.py None-in-dict chains
Use (x or {}) pattern instead of .get('key', {}) for null values
Guard against empty display_name causing IndexError
- Fix Persona webhook .lower() on None
Add null safety for status, first_name, last_name, inquiry_id
- Downgrade 'Some authors not found' from Sentry error to logger.warning
- Add explicit properties to UserDocument author_profile ObjectField
Prevents BulkIndexError from dynamic mapping conflicts
- Add prepare_headline normalizer to PersonDocument
Prevents mapper_parsing_exception for non-string headlines
- Remove 'hypothesis' from contribution allowed models
Model no longer exists, causes NoneType errors
- Add BulkIndexError handling in search celery tasks
Ignore benign delete not_found errors, handle NotFoundError
- 6 tests for _extract_license_info null safety (None source, missing, non-dict, etc.) - 7 tests for openalex_util null safety (None author in authorship, empty display_name, null authorships) - 4 tests for Persona webhook null safety (_get_nested_attr with None paths, missing status field) - 4 tests for PersonDocument prepare_headline normalizer (valid string, None, non-string, empty string) - 6 tests for search celery BulkIndexError handling (NotFoundError, benign bulk error, real error propagation, _is_benign_bulk_error)
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3225 +/- ##
==========================================
+ Coverage 78.75% 78.79% +0.03%
==========================================
Files 610 610
Lines 34334 34379 +45
==========================================
+ Hits 27039 27088 +49
+ Misses 7295 7291 -4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Addresses 9 high-impact production error patterns identified through code audit. None of these fixes from previous closed PRs (#3196, #3199, #3209) were merged.
Fixes
1. OpenAlex license extraction
AttributeError(~67K events)primary_location.get("source", {}).get("display_name")fails whensourceis explicitlynullisinstance(source, dict)guard before accessing nested fields2. Author
MultipleObjectsReturned(~35K events)Author.objects.update_or_create(openalex_ids=...)raises when duplicate authors existfilter().first()+ manual update/create pattern3. OpenAlex util
AttributeErroron null dict chains.get("key", {}).get(...)patterns fail when values are explicitlynull(x or {})pattern; guard against emptydisplay_namecausingIndexError4. Persona webhook
AttributeErroron.lower()_get_nested_attr(...).lower()crashes when status path returnsNone(raw_status or "").lower()and defaults for all fields5. "Some authors not found" Sentry noise
log_error(ValueError(...))sends to Sentry for a non-actionable conditionlogger.warning6. UserDocument
BulkIndexErrorfrom dynamic mappingObjectField()causes OpenSearch mapping conflicts forheadlineproperties(id, headline, profile_image)7. PersonDocument
mapper_parsing_exceptionheadlinevalues cause OpenSearch indexing failuresprepare_headlinenormalizer that returns empty string for non-strings8. Contribution views
NoneTypeerrorhypothesismodel no longer exists but is still in allowed models list_get_allowed_models()9. Search indexing
BulkIndexError/NotFoundErrornot_founderrors in bulk operations crash celery tasksNotFoundErrorand benignBulkIndexErrorpatternsTests added
_is_benign_bulk_error)