update ingestion-helper status and version endpoints to support lists of multiple imports#563
Conversation
… of multiple imports
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| CodeStyle | 1 minor |
🟢 Metrics 5 complexity
Metric Results Complexity 5
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Code Review
This pull request updates the import automation workflow and ingestion helper to support batch processing of multiple imports. The API endpoints and schemas for updating import status and version have been modified to accept lists of imports, and corresponding unit tests and documentation have been updated. The review feedback suggests two key improvements: replacing the 'Z' suffix in next_refresh with '+00:00' to maintain compatibility with Python versions prior to 3.11, and extracting the caller identity retrieval outside of the loop in update_import_version to avoid redundant JWT decoding and verification.
vish-cs
left a comment
There was a problem hiding this comment.
Currently, it has a single status for all the imports. Would it make sense to make status as part of the import item so each import can have its own status?
|
We also need to update https://github.com/datacommonsorg/data/blob/master/import-automation/executor/scripts/update_import_version.sh as per this change |
I am approving the PR to unblock DCP work but it might be helpful to update this. |
Updated! |
This enables the DCP ingestion workflow to update multiple imports at once
Submit with: datacommonsorg/datacommons#134