GitHub integration

[sgfost]

adds a feature to the CML which allows for:

• connecting a public GitHub repository with a codebase in the CML and;
• pushing pushing a git repository constructed from public CML releases to the connected GitHub remote
• importing individual releases made on a GitHub repository into the CML as new releases (of a special "imported" type) which can be published/peer reviewed

Setup/management is done by the submitter on a configuration page for each codebase, accessible from the sidebar of a codebase page or the feature overview page (/github/). From there, one is walked through the process including connecting a github account with the existing oauth flow and installing our github app on the same account, then given the option of:

• connecting with a new repository: allows connecting to a blank github repository, then generates a git repo for a codebase which can be pushed (codebase updates are reflected in the git repo and the push can be repeated). Importing GitHub releases back is also allowed
• connecting with an existing repository: allows connecting with an existing GitHub repo and importing GitHub releases as codebase releases

configuration

-> https://github.com/comses/infra/wiki/GitHub-Sync

details

new models

• GithubIntegrationAppInstallation: tracks the github app installation status for a user
• CodebaseGitRemote: represents a remote repository that should be pushed to and/or checked for new releases
• GitRefSyncState: represents the state of a git ref (commit/tag/branch) that was generated for a codebase and the state of the push to GitHub
• ImportedReleaseSyncState: represents the state of a release imported from GitHub, the existence of this means a release was imported from github, and should be treated differently in some cases
• GitHubIntegrationConfiguration: (admin dashboard-editable) feature settings including overview page content

github APIs

• GitHubApi: main wrapper for github operations (auth, pushing, creating releases, etc.)
• GitHubReleaseImporter: handles importing releases from github by downloading the archive, extracting metadata, and creating a new CodebaseRelease

filesystem

• CodebaseGitRepositoryApi: manages local git repository operations like building and updating a git repo from a codebase's (public) releases
• FileCategories/CategoryManifestManager: file categorization changed in order to handle imported releases which are not organized into category directories, information is instead kept in a manifest in the db and updated by the submitter
• ImportedCodebaseReleaseFsApi: sibling filesystem API of the regular CodebaseReleaseFsApi with some special handling and additional functionality
• adds generated LICENSE and CITATION.cff files to new release packages

webhooks

library.views.github_sync_webhook: the github app sends webhook events to the server and we respond to installation events to track user installations

[alee]

@sgfost can you rebase this on main whenever you get the chance?

[Copilot]

Pull request overview

Adds end-to-end GitHub integration to the CoMSES Model Library (CML), including UI workflows and backend APIs for connecting repositories, pushing generated git history, and importing GitHub releases into CML as “imported” releases.

Changes:

• Introduces new Django models/serializers/viewsets/tasks to manage GitHub remotes, sync state, webhook installation tracking, and async push/import jobs.
• Adds new Vue pages/components and API composables for GitHub setup wizard, overview page, and release alignment/push/import UI.
• Updates release-editor UX to support imported releases (locked file editing + file categorization via manifest).

Reviewed changes

Copilot reviewed 82 out of 88 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
frontend/src/types.ts	Adds GitHub integration and imported-release types; extends file/release typing.
frontend/src/stores/releaseEditor.ts	Initializes new file category + release status in editor store state.
frontend/src/scss/_global.scss	Adds shared UI utility classes/badges/layout helpers for new screens.
frontend/src/composables/api/releaseEditor.ts	Adds API call for updating file category in SIP preview.
frontend/src/composables/api/git.ts	New API composable for git remotes, push, import, and listings.
frontend/src/composables/api/codebase.ts	Adds endpoints for GitHub installation status + submitted codebases list.
frontend/src/components/releaseEditor/UploadFormPage.vue	Hides upload UI when originals aren’t editable; shows “preview” message.
frontend/src/components/releaseEditor/ReviewModal.vue	Expands modal size for review request UI.
frontend/src/components/releaseEditor/PublishModal.vue	Disables version edit for imported releases; adds review constraint warning.
frontend/src/components/releaseEditor/ProgressSidebar.vue	Generalizes sidebar labels/routes for imported vs original releases.
frontend/src/components/releaseEditor/ImportedArchivePage.vue	New page for imported release package preview + categorization context.
frontend/src/components/releaseEditor/FileTree.vue	Adds file icons and per-file category reassignment UI for imported releases.
frontend/src/components/releaseEditor/App.vue	Updates release editor header/buttons/sidebar wiring for imported releases.
frontend/src/components/githubIntegration/SelectSyncTypeStep.vue	New wizard step for choosing existing vs new repo sync mode.
frontend/src/components/githubIntegration/ReleaseSyncStatusCell.vue	New status cell for push/import status and import actions/version prompt.
frontend/src/components/githubIntegration/ReleaseManagementSection.vue	New release alignment table + push-all/import orchestration and polling.
frontend/src/components/githubIntegration/LocalReleaseItem.vue	New local release list item display for git mirror/push state.
frontend/src/components/githubIntegration/InstallGitHubAppStep.vue	New wizard step guiding app installation.
frontend/src/components/githubIntegration/GitHubReleaseItem.vue	New GitHub release list item display.
frontend/src/components/githubIntegration/ConnectRepositoryStep.vue	New wizard step to connect a repository by name and disconnect.
frontend/src/components/githubIntegration/ConnectGitHubStep.vue	New wizard step to connect GitHub social account.
frontend/src/components/githubIntegration/AccordionStep.vue	New reusable accordion wizard step container.
frontend/src/components/form/HoneypotField.vue	Makes show optional for honeypot field props.
frontend/src/components/ReviewReminders.vue	Updates peer review reminder messaging for imported+published behavior.
frontend/src/components/ListSidebar.vue	Changes “clear filters” button styling.
frontend/src/components/GitHubIntegrationOverview.vue	New overview UI showing submitted models and GitHub connection status.
frontend/src/components/GitHubIntegrationConfiguration.vue	New per-codebase GitHub sync configuration wizard + management section.
frontend/src/components/CodebaseEditForm.vue	Adds “Import model from GitHub” path for new submissions and redirect logic.
frontend/src/apps/release_editor.ts	Routes imported releases to “package” view vs upload view.
frontend/src/apps/github_overview.ts	New app entry for GitHub overview page mount.
frontend/src/apps/github_config.ts	New app entry for per-codebase GitHub config mount.
frontend/src/apps/codebase_edit.ts	Passes canUseGithubIntegration into codebase edit form app.
django/requirements.txt	Adds GitPython + PyGithub; pins codemeticulous fork+commit.
django/library/views.py	Adds codebase/git remote APIs, webhook handler, overview view, and import/category actions.
django/library/urls.py	Registers git remote viewset; adds webhook + overview page routes.
django/library/tests/test_views.py	Updates category enum usage; adds webhook tests.
django/library/tests/test_metadata.py	Adds tests for ReleaseMetadataConverter behavior and priority.
django/library/tests/test_github_integration.py	New tests for repo validation and release import/reimport flows.
django/library/tests/test_fs.py	Adds git repo API tests; updates sample path usage and category enum.
django/library/tests/samples/releases/animals-model/2.0.0/docs/README.md	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/2.0.0/data/input.csv	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/2.0.0/code/model.py	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/2.0.0/code/animals/sheep.py	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/2.0.0/code/animals/horse.py	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/2.0.0/code/animals/cow.py	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/1.0.0/results/analysis.txt	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/1.0.0/docs/README.txt	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/1.0.0/data/input.csv	Test fixture for git repo build content.
django/library/tests/samples/releases/animals-model/1.0.0/code/model.py	Test fixture for git repo build content.
django/library/tests/samples/archives/nestedcode/src/ex.py	Archive fixture used by FS import tests.
django/library/tests/samples/archives/nestedcode/README.md	Archive fixture used by FS import tests.
django/library/tests/samples/archives/nestedcode/.svn/svn_files_here	Archive fixture used by FS import tests.
django/library/tests/samples/archives/nestedcode/.DS_store	Archive fixture used by FS import tests.
django/library/tests/samples/archives/invalid.zip	Invalid archive fixture for error-path tests.
django/library/tests/samples/archives/.gitignore	Keeps generated nestedcode.zip out of git.
django/library/tests/base.py	Adds TEST_SAMPLES_DIR and optional file creation in publishable release setup.
django/library/tasks.py	Adds async tasks for local repo build, pushing, importing, and metadata sync.
django/library/serializers.py	Adds serializers for git remotes and sync states; extends related/release serializers.
django/library/permissions.py	Adds GitHubIntegrationPermission gate using site configuration.
django/library/migrations/0037_githubintegrationconfiguration_approved_users_and_more.py	Adds feature gating fields (whitelist + global enable).
django/library/migrations/0036_initial_github_integration_overview_content.py	Populates initial overview/FAQ/tutorial content in Wagtail config.
django/library/migrations/0035_alter_githubintegrationfaqentry_options_and_more.py	Refactors GitHub config content models and ordering fields.
django/library/migrations/0034_codebasegitremote_githubintegrationconfiguration_and_more.py	Creates core GitHub integration models and constraints.
django/library/metadata.py	Adds ReleaseMetadataConverter; prefers connected git remote URL in codemeta export.
django/library/management/commands/update_codebase_metadata.py	Uses new metadata rebuild method for releases.
django/library/jinja2/library/review/invitations/update.jinja	Adds reviewer note for imported releases.
django/library/jinja2/library/review/feedback/update.jinja	Adds reviewer note for imported releases.
django/library/jinja2/library/review/email/model_revisions_requested_imported.jinja	New email template for imported-release revision workflow.
django/library/jinja2/library/review/dashboard.jinja	Displays imported-release indicator in editor dashboard list.
django/library/jinja2/library/includes/publish_model_help.jinja	Reworks publish help UI; adds GitHub integration entry point.
django/library/jinja2/library/github_overview.jinja	New GitHub integration overview page template (mounts Vue app).
django/library/jinja2/library/codebases/releases/retrieve.jinja	Adds GitHub panel, imported-from section, and license null-handling.
django/library/jinja2/library/codebases/releases/edit.jinja	Passes isImported prop into release editor app mount.
django/library/jinja2/library/codebases/macros.jinja	Adds imported-release indicator macro.
django/library/jinja2/library/codebases/git.jinja	New per-codebase GitHub integration configuration page template.
django/library/jinja2/library/codebases/edit.jinja	Passes canUseGithubIntegration prop into codebase form mount.
django/library/github_integration.py	New core integration module: API wrapper, repo validator, importer, release listing, installation status helper.
django/curator/tests/test_dump_restore.py	Uses TEST_SAMPLES_DIR for archive fixture pathing.
django/curator/invoke_tasks/borg.py	Adds repository mirror directory into backup/restore rotation.
django/curator/fs.py	Skips imported releases in file consistency checks.
django/core/tests/base.py	Adds helper to clear repository root in tests.
django/core/settings/defaults.py	Adds GitHub integration app settings and secrets.
django/core/models.py	Adds github_username convenience property.
deploy/conf/.env.template	Adds environment variables for GitHub integration app configuration.
base.yml	Adds new docker secrets for GitHub app private key + webhook secret.
Makefile	Includes GitHub app secrets in external secrets list.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.