fix: add timeout and scheme validation when fetching remote topologies

[bunlongheng]

Summary

http_fetch_content() in netsim/cli/create.py downloads a remote topology with requests.get(url) using no timeout and without validating the URL scheme. This function is reached whenever a topology is supplied as a URL (netlab up <url> / netlab create <url>), and it is also reachable from the built-in netlab api server: a POST /jobs request with a topologyUrl field routes into netlab create/netlab up, which call this function.

Two problems:

1. No request timeout. requests.get() defaults to blocking forever. In the netlab api server the fetch runs inside a worker that holds the global RUN_LOCK (see netsim/cli/api.py), so a single request pointing at an unresponsive or slow-loris host makes the worker hang indefinitely and permanently wedges the job queue. Basic auth on the API server is optional and off by default, so this is triggerable without credentials.

2. No URL-scheme validation. The URL is passed straight to requests with no restriction, so unexpected schemes (for example file:// when a filesystem transport adapter is mounted) are accepted rather than rejected up front.

Fix

• Reject any topology URL whose scheme is not http/https before the request is made.
• Add a bounded HTTP_FETCH_TIMEOUT (20s) to the requests.get() call so a stuck download fails cleanly instead of blocking forever.

The change is confined to http_fetch_content() and preserves existing behavior for valid http/https topology URLs (including the GitHub ?raw=true rewrite).

Testing

• python3 -c "import ast; ast.parse(open('netsim/cli/create.py').read())" passes.
• Valid http/https URLs follow the unchanged download path; non-http(s) URLs now exit with a clear error; an unresponsive host now fails after the timeout instead of hanging.

[ipspace]

Nice fixes, thank you. However, being of the "every constant eventually becomes a variable" persuasion, it would be nice to have both parameters (timeout, allowed URL schemas) as variables defined in system settings, which creates a bit of a conundrum as we have to read the topology before we can download the topology.

I will add that bit to the code (might take a few days to find the time to do it) and then merge this.

Thanks again, Ivan

[ipspace]

Thanks again. I added the configurable parameters and documentation. Merging.

[Copilot]

Pull request overview

This PR hardens remote-topology fetching in netlab create/netlab up by adding URL-scheme allowlisting and a bounded HTTP request timeout, reducing the risk of hanging workers (notably in the netlab api job queue path).

Changes:

• Add URL scheme validation (allow http/https) before downloading remote topologies.
• Add a configurable requests.get(..., timeout=...) for remote topology downloads.
• Document the scheme restriction and how to change allowed schemes via system defaults.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
netsim/defaults/netlab.yml	Introduces default download settings (timeout + allowed schemes) for remote topology fetches.
netsim/cli/create.py	Implements scheme validation and request timeout in http_fetch_content().
docs/netlab/create.md	Documents the HTTP/HTTPS-only behavior and the defaults knob to alter allowed schemes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.