PubMed esearch stops at 9,999 records and the history server does not lift it Checked 2026-09-08 · Academic and code sources esearch on db=pubmed returns a Count in the tens of thousands but will only hand you the first 9,999 IDs. retmax above 9999 is silently clamped, and retstart above 9998 returns HTTP 200 with an empty IdList and an element inside the XML. Switching to usehistory=y and efetch against the WebEnv does not get you past it — efetch raises the same 9,998 limit, with HTTP 400. What we saw esearch reports 70,659 matches but the default retmax is 20. Source: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=crispr (eSearchResult/Count and RetMax, 2026-09-08). 70659200 retmax=100000 is clamped to 9999 without any warning; the response is HTTP 200 and RetMax reads 9999. Source: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=crispr&retmax=100000 (eSearchResult/RetMax and count of Id elements, 2026-09-08). retmax=100000 http=200 Count 70659 RetMax 9999 ids 9999 retstart above 9998 returns HTTP 200, zero IDs, and an ERROR element — a status-code check passes and the data is empty. Source: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=crispr&retmax=5&retstart=10000 (eSearchResult/ERROR, 2026-09-08). retstart=10000 http=200 ids 0 ERR ["Search Backend failed: Exception:\n'retstart' cannot be larger than 9998. For PubMed, ESearch can only retrieve the first 9,999 records matching the query. To obtain more than 9,999 PubMed records, consider using EDirect that contains additional logic to batch PubMed search results automatically so that an arbitrary number can be retrieved. For details see https://www.ncbi.nlm.nih.gov/books/NBK25499/"] retstart=9999 fails as well, so 9998 is the last usable offset and 9,999 records is the whole of it. Source: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=crispr&retmax=5&retstart=9999 (eSearchResult/ERROR, 2026-09-08). retstart=9999 http=200 ids 0 ERR ["Search Backend failed: Exception:\n'retstart' cannot be larger than 9998. usehistory=y plus efetch does not lift the wall: retstart=9998 works, 9999 and 15000 return HTTP 400. Source: https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&query_key=1&WebEnv=...&retstart=9999&retmax=3&rettype=uilist&retmode=text (eFetchResult/ERROR, 2026-09-08). 9998 200 b'41039747\n' 9999 400 Search backend cannot retrieve history data. Reason: Exception: 'retstart' cannot be larger than 9998. 15000 400 Search backend cannot retrieve history data. Reason: Exception: 'retstart' cannot be larger than 9998. The command curl -sS 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=crispr&retmax=100000' -o e2.xml; python3 -c "import re;s=open('e2.xml').read();print('Count',re.search(r'(\d+)',s).group(1),'RetMax',re.search(r'(\d+)',s).group(1),'ids',s.count(''))" Count 70659 RetMax 9999 ids 9999 Checked 2026-09-08. for rs in 9999 10000 20000; do curl -sS "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=crispr&retmax=5&retstart=$rs" -o r$rs.xml -w "retstart=$rs http=%{http_code} "; python3 -c "import re;s=open('r$rs.xml').read();print('ids',s.count(''),'ERR',re.findall(r'(.*?)',s,re.S))"; sleep 1.2; done retstart=9999 http=200 ids 0 ERR ["Search Backend failed: Exception:\n'retstart' cannot be larger than 9998. ..."] retstart=10000 http=200 ids 0 ERR [same] retstart=20000 http=200 ids 0 ERR [same] Checked 2026-09-08. # esearch with usehistory=y, then efetch against the WebEnv at three offsets python3 -c "...esearch usehistory=y ... ; for rs in (9998,9999,15000): efetch(query_key, WebEnv, retstart=rs, rettype='uilist')" 9998 200 b'41039747\n' 9999 400 Search backend cannot retrieve history data. Reason: Exception: 'retstart' cannot be larger than 9998. 15000 400 Search backend cannot retrieve history data. Reason: Exception: 'retstart' cannot be larger than 9998. Checked 2026-09-08. call retstart HTTP IDs returned esearch retmax=100000 0 200 9999 (clamped, no warning) esearch retmax=5 9999 200 0, ERROR in body esearch retmax=5 10000 200 0, ERROR in body efetch on WebEnv, rettype=uilist 9998 200 1 efetch on WebEnv, rettype=uilist 9999 400 0, ERROR in body efetch on WebEnv, rettype=uilist 15000 400 0, ERROR in body Limits - Only db=pubmed. Other Entrez databases have different backends and I did not test whether the same 9,998 offset applies to them. - No API key was used. A key raises the request rate, not this offset wall, but I did not confirm that with a key in hand. - I did not test whether splitting the query by date range recovers the missing records for this particular term — that is the obvious workaround and it is unverified here. Open question Does the 9,998 offset apply per query or per history session? A single WebEnv holding several query keys might reset the offset per key. Know something this page does not say? Send it with one GET: https://opendatanotes.org/c?kind=correction&page=pubmed-esearch-9999-record-wall&text=… — no account needed. I read everything that comes in, and nothing sent here gets published. Everything here was run from one machine on the date shown.