Skip to content

[PULP-2214] Add relative_path domains - #7959

Open
mdellweg wants to merge 1 commit into
pulp:mainfrom
mdellweg:relative_path_domain
Open

[PULP-2214] Add relative_path domains#7959
mdellweg wants to merge 1 commit into
pulp:mainfrom
mdellweg:relative_path_domain

Conversation

@mdellweg

Copy link
Copy Markdown
Member

This introduces a new shallow database type with a check constraint to maintain that all relative paths are sanitized.
This also consolidates on the idea what can be allowed as a relative path.

📜 Checklist

  • Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
  • A changelog entry or entries has been added for any significant changes
  • Follows the Pulp policy on AI Usage
  • (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

@mdellweg
mdellweg force-pushed the relative_path_domain branch 5 times, most recently from 4482b3f to 9000e8b Compare August 12, 2026 07:00
@mdellweg
mdellweg marked this pull request as ready for review August 12, 2026 07:30
Comment thread pulpcore/app/migrations/0155_create_rel_path_domains.py Outdated
@mdellweg
mdellweg force-pushed the relative_path_domain branch from b1b854f to 5aeec79 Compare August 12, 2026 13:56
@mdellweg
mdellweg enabled auto-merge (rebase) August 13, 2026 09:43
Comment thread pulpcore/app/models/publication.py Outdated
Comment on lines +664 to +666
indexes = [
SpGistIndex(
OpClass(models.expressions.RawSQL("\"base_path\" || '/'", ()), name="text_ops"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This index doesn't work with our current validate overlap ORM code. Unless the query contains "base_path" || '/' the index won't be used. AI recommended two options: 1. dropping the || '/' trick and just adding the special index on base_path or 2. we create a generated column with the / trick that the DB can index off and be used in our ORM code.

from django.contrib.postgres.indexes import OpClass, SpGistIndex
from django.db.models import GeneratedField, TextField, Value
from django.db.models.functions import Concat
class Distribution(...):
    base_path = RelativePathField()
    base_path_slash = GeneratedField(
        expression=Concat("base_path", Value("/")),
        output_field=TextField(),
        db_persist=True,
    )
    class Meta:
        indexes = [
            SpGistIndex(
                OpClass("base_path_slash", name="text_ops"),
                include=("pulp_domain",),
                name="core_distribution_base_path_slash",
            ),
        ]

Distribution.objects.filter(
    base_path_slash__startswith=f"{path}/",
    pulp_domain=get_domain(),
)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we hadf defined base_path to always include the trailing slash, The overlap would be a simple startswith symmetricised. But then there is no symmetric startswith operator in postgres and to declare one and it's operator class to be used in an exclusion constraint requires db-admin privileges.
So I guess the secon best thing we can do is what we already do. I'll rewrite the index (hoping that it is already an improvement today.).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had exactly that generated field in my code to base the index on until i realized that you don't need it to create the slashed index. But then again if you try .annotate(base_path_slash=Concat("base_path", Value("/")).filter(base_path_slash__startswith=".../"), the orm will use Coalesce inside the Concat and make the index unusable.
I think the current situation is a good compromise.

@mdellweg
mdellweg force-pushed the relative_path_domain branch from 5aeec79 to e337d3c Compare August 13, 2026 15:43
This introduces a new shallow database type with a check constraint to
maintain that all relative paths are sanitized.
This also consolidates on the idea what can be allowed as a relative
path.
@mdellweg
mdellweg force-pushed the relative_path_domain branch from e337d3c to fa01605 Compare August 13, 2026 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants