Skip to content

feat: Migrate package manager to uv #1553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FallenDeity wants to merge 7 commits into
base: master
Choose a base branch
from
Open

feat: Migrate package manager to uv #1553

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2427c6e
feat: Migrate package manager to uv
FallenDeity Jun 10, 2026
3bddc96
ci: Update test data step
FallenDeity Jun 10, 2026
a2637b0
ci: Add matrixes for build and test
FallenDeity Jun 11, 2026
d556e44
ci: Update circle ci matrix test naming to fan-in to test
FallenDeity Jun 11, 2026
08e2a71
fix: Update postgres job workflow to run in container, update dockerf…
FallenDeity Jun 17, 2026
8fd29b3
fix: remove workflow caching, and bump project version
FallenDeity Jun 30, 2026
5c44e0f
chore: update uv.lock
FallenDeity Jun 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 54 additions & 10 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
version: 2.1

jobs:
test:
static-checks:
docker:
- image: cimg/python:3.13.7
Comment thread
FallenDeity marked this conversation as resolved.
- image: cimg/python:3.14
steps:
- checkout
- run:
Expand All @@ -13,19 +13,45 @@ jobs:
git submodule update --remote
- run:
name: Install deps
command: make dev-install
command: make install
- run:
name: Linting
name: Format Check
command: make format-check
- run:
name: Generate OpenAPI schema
command: make openapi-generate

run-tests:
parameters:
python_version:
type: string
docker:
- image: cimg/python:<< parameters.python_version >>
steps:
- checkout
- run:
name: Pull Submodules
command: |
git submodule init
git submodule update --remote
- run:
name: Install deps
command: make install
- run:
name: Run tests
command: make test

test:
docker:
- image: cimg/base:stable
steps:
- run:
name: Generate OpenAPI schema
command: make openapi-generate
name: All tests passed
command: echo "All matrix tests completed successfully"

build:
docker:
- image: cimg/python:3.13.7
- image: cimg/python:3.14
steps:
- checkout
- run:
Expand All @@ -35,13 +61,14 @@ jobs:
git submodule update --remote
- run:
name: Install deps
command: make install
command: make install-base
- run:
name: Run migrations
command: make setup
- run:
name: Build database
command: make build-db

deploy:
machine:
image: ubuntu-2204:2024.11.1
Expand All @@ -64,10 +91,18 @@ workflows:
version: 2
test-and-deploy:
jobs:
- test
- static-checks
- run-tests:
matrix:
parameters:
python_version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
- test:
requires:
- run-tests
- build
- deploy:
requires:
- static-checks
- test
- build
filters:
Expand All @@ -77,11 +112,20 @@ workflows:
- staging
weekly:
jobs:
- test
- static-checks
- run-tests:
matrix:
parameters:
python_version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
- test:
requires:
- run-tests
- build
- deploy:
requires:
- static-checks
- test
- build
triggers:
- schedule:
cron: "0 0 * * 1"
Expand Down
51 changes: 32 additions & 19 deletions .github/workflows/database.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Lint
run: |
set -e
Expand All @@ -21,22 +21,37 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "0.11.19"
enable-cache: true
- name: Generate OpenAPI schema
run: |
make install
make install-base
make openapi-generate
sqlite:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
submodules: recursive
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "0.11.19"
enable-cache: true
python-version: ${{ matrix.python-version }}
- name: Start pokeapi
run: |
make install
make migrate
make install-base
make setup
make build-db
nohup make serve &
sleep 3
Expand All @@ -48,25 +63,23 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
submodules: recursive
- name: Build
- name: Build and start services

@Naramsim Naramsim Jun 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not building anymore, just starting the services.

@FallenDeity FallenDeity Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker compose up does build the image too if it dosent exist (which it dosent when this task first runs) but ill also add a --build flag to be explicit

run: |
docker compose -f docker-compose.yml -f docker-compose-dev.yml up -d --build
- name: Run migrations and build database
run: |
docker compose -f docker-compose.yml -f docker-compose-dev.yml up -d
make docker-migrate
make docker-build-db
- name: Dump DB
run: docker compose exec -T -u postgres db sh -c "cd /tmp && pg_dump -h localhost -Fc -U ash -N 'hdb_*' pokeapi > pokeapi.dump"
- name: Copy dump
run: docker compose cp db:/tmp/pokeapi.dump ./
- name: Down services
run: docker compose -f docker-compose.yml -f docker-compose-dev.yml down -v
- name: Start services
run: docker compose -f docker-compose.yml -f docker-compose-dev.yml up -d
- name: Import database
run: docker compose exec -T db pg_dump -U ash -Fc -N 'hdb_*' pokeapi > pokeapi.dump
- name: Drop and recreate database
run: |
docker compose cp ./pokeapi.dump db:/tmp/
docker compose exec -T -u postgres db sh -c "cd /tmp && pg_restore -h localhost -U ash -d pokeapi pokeapi.dump"
docker compose exec -T db psql -U ash -d postgres -c "DROP DATABASE pokeapi WITH (FORCE);"
docker compose exec -T db psql -U ash -d postgres -c "CREATE DATABASE pokeapi;"
- name: Import database
run: docker compose exec -T db pg_restore -U ash -d pokeapi < pokeapi.dump

@Naramsim Naramsim Jun 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put back -h localhost? I like the commands to be descriptive, so that anyone can see the pipeline and use it as a piece of documentation.

@FallenDeity FallenDeity Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default behavior for dump and restore for postgresql isnt localhost or the local loopback address instead it uses the unix socket (file based) which is faster without the tcp/ip overhead I can force it to use localhost if required though but imo there is no particular reason to go through localhost

previously the docker invocation was using -u postgres (os system user) and the the command was using -u ash which likely touched upon this auth permission rule with unix socket file perms vs tcp/ip bypass https://www.postgresql.org/docs/current/auth-trust.html due to differing users in the command and db role access

Local TCP/IP connections are not restricted by file-system permissions

docker compose exec -T -u postgres db sh -c "cd /tmp && pg_restore -h localhost -U ash -d pokeapi pokeapi.dump"

in the new workflow the commands run as the container default execution user so we dont run into any permission misalignments for file r/w

- name: Test data
run: curl -Ss http://localhost/api/v2/pokemon/1/ | grep -q 'bulbasaur'
4 changes: 2 additions & 2 deletions .github/workflows/docker-build-and-push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
submodules: recursive
- name: Docker meta
Expand Down Expand Up @@ -41,7 +41,7 @@ jobs:
echo "Platforms: ${{ steps.buildx.outputs.platforms }}"
- name: Build and push
id: docker_build
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: ./Resources/docker/app/Dockerfile
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/docker-k8s.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
submodules: recursive
- name: Docker meta
Expand Down Expand Up @@ -37,7 +37,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
submodules: recursive
- name: Docker meta
Expand Down Expand Up @@ -104,7 +104,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Build dev environment
run: |
docker compose -f docker-compose.yml -f docker-compose-dev.yml up -d
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
submodules: recursive
- name: Start pokeapi (docker)
Expand All @@ -26,8 +26,8 @@ jobs:
ls -larth
- name: Start pokeapi
run: |
make install
make migrate
make install-base
make setup
make build-db
nohup make serve &
sleep 3
Expand Down
1 change: 1 addition & 0 deletions .python-version
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.14
Comment thread
FallenDeity marked this conversation as resolved.
59 changes: 31 additions & 28 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,51 +4,54 @@ docker_config = --settings=config.docker-compose
gql_compose_config = -f docker-compose.yml -f Resources/compose/docker-compose-prod-graphql.yml
gqlv1beta_compose_config = -f docker-compose.yml -f Resources/compose/docker-compose-prod-graphql.yml -f Resources/compose/docker-compose-prod-graphql-v1beta.yml

# Auto-detect Python and pip commands
PYTHON := $(shell which python3 2>/dev/null || which python 2>/dev/null || echo python3)
PIP := $(shell which pip3 2>/dev/null || which pip 2>/dev/null || echo pip3)

.PHONY: help
.SILENT:

help:
@grep -E '^[a-zA-Z_-]+:.*?# .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?# "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

install: # Install base requirements to run project
$(PIP) install -r requirements.txt
check-uv:
@command -v uv >/dev/null 2>&1 \
|| { \
echo >&2 "uv is not installed. Install it from: https://docs.astral.sh/uv/getting-started/installation/"; \
exit 1; \
}

install: check-uv # Install requirements for local development
uv sync --locked --all-extras --dev

dev-install: # Install developer requirements + base requirements
$(PIP) install -r test-requirements.txt
install-base: check-uv # Install minimal requirements for runtime/pipeline environments
uv sync --locked --no-dev

setup: # Set up the project database
$(PYTHON) manage.py migrate ${local_config}
setup: check-uv # Set up the project database
uv run manage.py migrate ${local_config}

build-db: # Build database
echo "from data.v2.build import build_all; build_all()" | $(PYTHON) manage.py shell ${local_config}
build-db: check-uv # Build database
echo "from data.v2.build import build_all; build_all()" | uv run manage.py shell ${local_config}

wipe-sqlite-db: # Delete's the project database
rm -rf db.sqlite3

serve: # Run the project locally
$(PYTHON) manage.py runserver ${local_config}
serve: check-uv # Run the project locally
uv run manage.py runserver ${local_config}

test: # Run tests
$(PYTHON) manage.py test ${local_config}
test: check-uv # Run tests
uv run manage.py test ${local_config}

clean: # Remove any pyc files
find . -type f -name '*.pyc' -delete

migrate: # Run any outstanding migrations
$(PYTHON) manage.py migrate ${local_config}
migrate: check-uv # Run any outstanding migrations
uv run manage.py migrate ${local_config}

make-migrations: # Create migrations files if schema has changed
$(PYTHON) manage.py makemigrations ${local_config}
make-migrations: check-uv # Create migrations files if schema has changed
uv run manage.py makemigrations ${local_config}

shell: # Load a shell
$(PYTHON) manage.py shell ${local_config}
shell: check-uv # Load a shell
uv run manage.py shell ${local_config}

openapi-generate:
$(PYTHON) manage.py spectacular --color --file openapi.yml ${local_config}
openapi-generate: check-uv
uv run manage.py spectacular --color --file openapi.yml ${local_config}

docker-up: # (Docker) Create services/volumes/networks
docker compose up -d
Expand Down Expand Up @@ -86,11 +89,11 @@ docker-prod:

docker-setup: docker-up docker-migrate docker-build-db # (Docker) Start services, prepare the latest DB schema, populate the DB

format: # Format the source code
black . --extend-exclude '.+/scripts/.+'
format: check-uv # Format the source code
uv run black . --extend-exclude '.+/scripts/.+'

format-check: # Check the source code has been formatted
black . --check --extend-exclude '.+/scripts/.+'
format-check: check-uv # Check the source code has been formatted
uv run black . --check --extend-exclude '.+/scripts/.+'

pull:
git checkout master
Expand Down
6 changes: 4 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ A RESTful API for Pokémon - [pokeapi.co](https://pokeapi.co)

- Download this source code into a working directory, be sure to use the flag `--recurse-submodules` to clone also our submodules.

- Install the requirements using pip:

- Install [uv](https://docs.astral.sh/uv/getting-started/installation/) for Python environment management.

- Install the requirements using the Makefile commands:

```sh
make install
# This will install all the required packages and libraries for using PokeAPI
Expand Down
Loading
Loading