Option to rewrite endpoint URL - Fix local instance

[Didayolo]

Suggested reviewer: @ihsaan-ullah

Description

Add an option environment variable:

# Optional URL rewriting in compute worker, format: FROM | TO
#WORKER_BUNDLE_URL_REWRITE=http://localhost:9000|http://minio:9000

It is used inside compute_worker.py to rewrite the endpoint URL.

This solves cases of connection refused because backend signs URLs are different between the host and inside the docker container.

Typical case on macOS: backend signs URLs with host docker.for.mac.localhost:9000, inside the container we need minio:9000

Issues this PR resolves

• Closes Local setup - Compute worker cannot get submission from Minio - Connection refused #2003

Manual installation instructions

Add the WORKER_BUNDLE_URL_REWRITE variable in the .env file. For instance for local macOS setup:

# Optional URL rewriting in compute worker. Format: FROM | TO
WORKER_BUNDLE_URL_REWRITE=http://docker.for.mac.localhost:9000|http://minio:9000

Keep it commented when it is not useful (like it is in .env_sample)

Rebuild the compute worker to take the changes into account:

docker compose build compute_worker
docker compose up -d

A checklist for hand testing

• When the env variable is commented, the change should not do anything. Try to make submissions.
• On macOS, try to fill the env variable and make submissions

Checklist

• Code review by me
• Hand tested by me
• I'm proud of my work
• Code review by reviewer
• Hand tested by reviewer
• CircleCi tests are passing
• Ready to merge

[ihsaan-ullah]

Hi @Didayolo thank you for the great solution. I tested it and it works well. I can now process submissions.

As this problem is not related to codabench but just mac problem, I suggest that we put the fix in the docs and not as a PR. What do you think?

[Didayolo]

Hi @Didayolo thank you for the great solution. I tested it and it works well. I can now process submissions.

As this problem is not related to codabench but just mac problem, I suggest that we put the fix in the docs and not as a PR. What do you think?

I believe it can have some use cases depending on the setup. It is a simple and optional change. I am personally in favor of merging it to the main codebase.