Fix/configurable user agent #2620
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
Nice improvement! I'm good moving it forward once we add an integration test. @markphelps can you do that? |
Contributor
I can in a bit. @SharkFishDeveloper this looks like it also contains the fix in #2619 . Can we rebase this onto main instead without that change since its in another PR? |
SharkFishDeveloper
force-pushed
the
fix/configurable-user-agent
branch
from
e21492c to
81b4f06
Compare
Contributor
Author
|
Thanks for pointing that out. I’ve cleaned this up by rebasing onto main and keeping only the User-Agent change. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2554
Some external services reject requests made by Cog when fetching input URLs
because the default
python-requestsUser-Agent is blocked or disallowed.Previously, Cog used
requests.get()without any way for users to configurerequest headers. This caused URL fetches to fail for certain providers.
This change allows users to configure the User-Agent via an environment
variable (
COG_USER_AGENT). If the variable is set, it is included in therequest headers; otherwise, the existing behavior remains unchanged.
This provides a simple, backward-compatible way to avoid request rejections
without modifying Cog internals or re-hosting input files.