urllib.parse.urlsplit & parse_qs
URLs piece by piece
Lesson 11 of 20 · 10 min · Module 3: Ports and endpoints
A URL packs a protocol, a host, an optional port, a path, and a query string into one line. Python’s standard library can take it apart reliably.
Start this lesson →Free. No installation: you write and run the code in your browser, and it is checked automatically.
By the end, you’ll be able to…
- Name the parts of a URL
- Use urllib.parse instead of manual splitting
- Apply sensible defaults
The idea
urlsplit(url) returns parts such as scheme, hostname (lowercase), port (None if absent), path, and query. parse_qs(query) turns "q=linux&page=2" into {"q": ["linux"], "page": ["2"]}.
from urllib.parse import urlsplit
parts = urlsplit("https://codetemper.com/courses?level=beginner")
print(parts.scheme, parts.hostname, parts.path, parts.query)
01Anatomy of a URL
In https://codetemper.com:443/courses?level=beginner, https is the scheme, codetemper.com the host, 443 the port, /courses the path, and level=beginner the query.
02Let the library handle edge cases
URLs can contain user names, IPv6 hosts in brackets, encoded characters, and fragments after #. urlsplit handles all of them, while a hand-written split breaks on the first unusual case.
from urllib.parse import urlsplit
print(urlsplit("http://[::1]:8080/").hostname)::103Query values are lists
A key can appear twice, as in ?tag=a&tag=b, so parse_qs always returns lists. Decide explicitly which value you keep.
Your turn
describe_url(url) that returns a dict with scheme, host (lowercase), port (the explicit port, or 80 for http and 443 for https), path (use "/" when empty), and query (a dict where each key maps to its first value).What the checks look for
- Describes an https URL with a query
- Keeps an explicit port and lowercases the host
- Uses / for an empty path and port 80 for http
- Keeps the first value of a repeated key
Take this with you
A URL is structured data; parse it with a parser.
Go a little further
Also return the fragment (the part after #) when the URL has one.