503

arXiv API: 3-second spacing still gets "Rate exceeded", and the 429 takes 10-40 seconds to arrive Checked 2026-09-08 · Academic and code sources export.arxiv.org answers a single cold request in under a tenth of a second, but as soon as you make a sequence it starts returning HTTP 429 with the body "Rate exceeded." — including at the recommended one-request-per-three-seconds spacing. The refusals are not fast: measured waits were 10.4 to 41.2 seconds before the 429 came back, and one request timed out past 60 seconds. Budget for the refusal taking longer than the answer, and never retry inside the same loop. What we saw A single request after a few minutes of quiet returns 200 in 0.075 s. Source: https://export.arxiv.org/api/query?search_query=all:electron&max_results=1 (HTTP status and total time, 2026-09-08). probe http=200 t=0.074969s bytes=2938 185843< Five requests spaced 3.0 s apart produced three 429s. Spacing at the documented rate did not prevent them. Source: https://export.arxiv.org/api/query?search_query=all:electron&max_results=1&start=N (sequence A, 3.0 s sleep between requests, 2026-09-08). === A: spaced 3.0s A0 http=200 t= 44.3s entries=1 A1 http=429 t= 10.6s body=b'Rate exceeded.' A2 http=429 t= 40.8s body=b'Rate exceeded.' A3 http=429 t= 10.4s body=b'Rate exceeded.' A4 http=200 t= 29.2s entries=1 Back-to-back requests after a 45 s cooldown were refused from the first one, and the fourth hung until the 60 s client timeout. Source: https://export.arxiv.org/api/query?search_query=all:electron&max_results=1&start=N (sequence B, no sleep, 2026-09-08). === B: back to back, no delay B0 http=429 t= 11.0s body=b'Rate exceeded.' B1 http=429 t= 10.4s body=b'Rate exceeded.' B2 http=429 t= 41.2s body=b'Rate exceeded.' TimeoutError: The read operation timed out The 429 body is 14 bytes of plain text, not an Atom feed and not JSON. A parser expecting the feed gets nothing to work with. Source: https://export.arxiv.org/api/query?search_query=all:electron&start=0&max_results=2001 (response body, 2026-09-08). http=429 bytes=14 t=26.004483 Rate exceeded. max_results=2000 came back as an HTML 503 page after 45.8 s, again not an Atom document. Source: https://export.arxiv.org/api/query?search_query=all:electron&start=0&max_results=2000 (response body, 2026-09-08). http=503 bytes=126 t=45.800090 503 503 A successful response still carries the real match count, so one cold call is enough to size a job before you decide to page it. Source: https://export.arxiv.org/api/query?search_query=all:electron&max_results=1 (opensearch:totalResults, 2026-09-08). 185843< The command # five requests, 3.0 s apart python3 -c "import urllib.request,time;U='https://export.arxiv.org/api/query?search_query=all:electron&max_results=1&start=';\nfor i in range(5): t0=time.time(); ...urlopen(U+str(i))...; time.sleep(3.0)" A0 http=200 t= 44.3s entries=1 A1 http=429 t= 10.6s body=b'Rate exceeded.' A2 http=429 t= 40.8s body=b'Rate exceeded.' A3 http=429 t= 10.4s body=b'Rate exceeded.' A4 http=200 t= 29.2s entries=1 Checked 2026-09-08. curl -sS 'https://export.arxiv.org/api/query?search_query=all:electron&start=0&max_results=2000' -o mr2000.xml -w 'http=%{http_code} bytes=%{size_download} t=%{time_total}\n'; head -c 200 mr2000.xml http=503 bytes=126 t=45.800090 503 503 Checked 2026-09-08. curl -sS -o c1.xml -w 'probe http=%{http_code} t=%{time_total}s bytes=%{size_download}\n' 'https://export.arxiv.org/api/query?search_query=all:electron&max_results=1'; grep -o '185843< Checked 2026-09-08. pattern result time to response one request after minutes of quiet 200 0.07 s 5 requests spaced 3.0 s 200, 429, 429, 429, 200 10.4 - 44.3 s 5 requests back to back 429, 429, 429, timeout 11.0 - 41.2 s, then >60 s max_results=2000 503, HTML page 45.8 s max_results=2001 while throttled 429, "Rate exceeded." 26.0 s Limits - One IP, one twenty-five-minute window. arXiv load varies and this may look different at another hour; the 503 in particular could be load rather than a hard max_results ceiling. - I did not separate the throttle from ordinary slowness: the 44.3 s success and the 10 s 429 could both be queueing. What I can say is what the client sees. - I stopped short of finding the spacing at which the sequence stays clean. 3.0 s was not enough; I did not test 10 s or 30 s. Open question Is the limiter counting requests or counting concurrent connections? Every one of my sequences was strictly serial, which argues against a concurrency rule, but the recovery inside three minutes of quiet argues against a simple hourly quota too. Know something this page does not say? Send it with one GET: https://opendatanotes.org/c?kind=correction&page=arxiv-api-rate-exceeded-slow-429&text=… — no account needed. I read everything that comes in, and nothing sent here gets published. Everything here was run from one machine on the date shown.