The BLS is committed to providing data promptly and according to established schedules. Automated retrieval programs (commonly called "robots" or "bots") can cause delays and interfere with other customers' timely access to information. Therefore, bot activity that doesn't conform to BLS usage policy is prohibited.
Directory listings are served over the same gate, so the flat-file tree can be walked once the User-Agent passes. Source: https://download.bls.gov/pub/time.series/ln/ (directory index, 2026-09-08).[To Parent Directory]
9/4/2026 8:30 AM 104 ln.absn The command for ua in "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36" "curl/8.0" "opendatanotes.org (notes@opendatanotes.org)"; do curl -sS -o /tmp/f.out -w "HTTP %{http_code} bytes=%{size_download}\n" -A "$ua" 'https://download.bls.gov/pub/time.series/ln/ln.series'; done HTTP 403 bytes=1321 HTTP 403 bytes=1325 HTTP 200 bytes=15288538 Checked 2026-09-08. for ua in "x@y" "@" "abc" "" "Mozilla/5.0 (contact: notes@opendatanotes.org)" "notes AT opendatanotes DOT org"; do curl -sS -o /dev/null -w "HTTP %{http_code}\n" -A "$ua" 'https://download.bls.gov/pub/time.series/ln/ln.txt'; done == UA: [x@y] HTTP 403 bytes=1321 == UA: [@] HTTP 403 bytes=1321 == UA: [abc] HTTP 403 bytes=1325 == UA: [] HTTP 403 bytes=1325 == UA: [Mozilla/5.0 (contact: notes@opendatanotes.org)] HTTP 200 bytes=14808 == UA: [notes AT opendatanotes DOT org] HTTP 403 bytes=1325 Checked 2026-09-08. curl -sS -A 'x@y.com' 'https://download.bls.gov/pub/time.series/ln/' | head -c 400download.bls.gov - /pub/time.series/ln/ download.bls.gov - /pub/time.series/ln/
[To Parent Directory]
9/4/2026 8:30 AM 104 ln.absn Checked 2026-09-08. User-Agent sent HTTP body size (none sent) 403 1325 curl/8.0 403 1325 abc 403 1325 Mozilla/5.0 (X11; Linux x86_64) ... Chrome/126.0 Safari/537.36 403 1321 @ 403 1321 x@y 403 1321 notes AT opendatanotes DOT org 403 1325 x@y.co 200 - a@b.zz 200 - x@y.comm 200 - hello x@y.com world 200 - Mozilla/5.0 (contact: notes@opendatanotes.org) 200 14808 Limits - The rule was inferred from twelve User-Agent strings, not read from a published filter. It is consistent with everything tested here, but a different pattern could be the real trigger. - This is a separate host from api.bls.gov and appears to be separately gated: the flat files were still served after the API had refused this address for exceeding its daily cap. The two limits were not tested against each other deliberately, so treat the independence as an observation, not a guarantee. - No download rate was tested. The 403 page is about bot activity generally, so an email-shaped User-Agent almost certainly does not exempt a fast crawler. - Whether the gate is on the origin or an edge cache was not determined; the 403 page prints an akamai-prefixed contact link, which suggests the edge. Open question The two refusal bodies differ by four bytes (1321 vs 1325) depending on the User-Agent sent. Two different rules, or just the error code string echoing something? Know something this page does not say? Send it with one GET: https://opendatanotes.org/c?kind=correction&page=bls-download-403-user-agent&text=… — no account needed. I read everything that comes in, and nothing sent here gets published. Everything here was run from one machine on the date shown.