[PULP-2214] Add relative_path domains - #7959
Conversation
4482b3f to
9000e8b
Compare
b1b854f to
5aeec79
Compare
| indexes = [ | ||
| SpGistIndex( | ||
| OpClass(models.expressions.RawSQL("\"base_path\" || '/'", ()), name="text_ops"), |
There was a problem hiding this comment.
This index doesn't work with our current validate overlap ORM code. Unless the query contains "base_path" || '/' the index won't be used. AI recommended two options: 1. dropping the || '/' trick and just adding the special index on base_path or 2. we create a generated column with the / trick that the DB can index off and be used in our ORM code.
from django.contrib.postgres.indexes import OpClass, SpGistIndex
from django.db.models import GeneratedField, TextField, Value
from django.db.models.functions import Concat
class Distribution(...):
base_path = RelativePathField()
base_path_slash = GeneratedField(
expression=Concat("base_path", Value("/")),
output_field=TextField(),
db_persist=True,
)
class Meta:
indexes = [
SpGistIndex(
OpClass("base_path_slash", name="text_ops"),
include=("pulp_domain",),
name="core_distribution_base_path_slash",
),
]
Distribution.objects.filter(
base_path_slash__startswith=f"{path}/",
pulp_domain=get_domain(),
)There was a problem hiding this comment.
If we hadf defined base_path to always include the trailing slash, The overlap would be a simple startswith symmetricised. But then there is no symmetric startswith operator in postgres and to declare one and it's operator class to be used in an exclusion constraint requires db-admin privileges.
So I guess the secon best thing we can do is what we already do. I'll rewrite the index (hoping that it is already an improvement today.).
There was a problem hiding this comment.
I had exactly that generated field in my code to base the index on until i realized that you don't need it to create the slashed index. But then again if you try .annotate(base_path_slash=Concat("base_path", Value("/")).filter(base_path_slash__startswith=".../"), the orm will use Coalesce inside the Concat and make the index unusable.
I think the current situation is a good compromise.
5aeec79 to
e337d3c
Compare
This introduces a new shallow database type with a check constraint to maintain that all relative paths are sanitized. This also consolidates on the idea what can be allowed as a relative path.
e337d3c to
fa01605
Compare
This introduces a new shallow database type with a check constraint to maintain that all relative paths are sanitized.
This also consolidates on the idea what can be allowed as a relative path.
📜 Checklist
See: Pull Request Walkthrough