Web Data Labs › Blog › USPTO Patents Scraper

How to Extract USPTO Patent Data at Scale (Patents API Guide)

May 2, 2026 · 7 min read

U.S. patent data is, on paper, completely public. Every granted patent, every published application, every claim, every assignment — all of it lives somewhere on a USPTO server. In practice, getting structured patent data into your pipeline is one of those problems that looks easy and turns into a multi-week side quest the moment you actually try.

If you have ever opened the USPTO public search front-end, run a query, and wondered how to feed the results into a notebook, this post is for you. We will walk through why bulk patent data is harder than it looks, what kind of work it unlocks once you have it, and what a structured USPTO record should look like by the time it lands in your CSV or JSON pipeline.

Why USPTO patent data is hard to get

The USPTO publishes data through several different channels, and none of them are friendly for the “I just want a clean dataset of US patents matching this query” use case.

Manual web search is one record at a time. The Patent Public Search front-end is great for a lawyer pulling a single document. It is not a programmatic interface. Copy-pasting fifty results into a spreadsheet is fine. Doing the same for fifty thousand is not.
Bulk archives are gigabytes of XML. The USPTO does publish weekly bulk grant and application files. They are also enormous, in a custom XML schema, and structured for archival rather than for ad-hoc querying. Building a usable search layer on top of them is a project, not a script.
The legacy PatentsView API was retired. For years PatentsView was the standard JSON-over-HTTPS layer that academic and commercial users relied on. It was sunset in 2024 and consolidated into a new USPTO Open Data Portal that requires registered API keys, has its own rate limits, and uses a different schema again.
Commercial alternatives are priced for enterprise legal teams. Lens, Derwent, Patsnap, and similar IP-data vendors offer beautiful interfaces and rich datasets, but their pricing assumes a corporate IP department, not a solo researcher or an early-stage product team.

The result: most teams that need patent data either give up, pay a vendor, or burn an engineer-week building a fragile scraper around whatever endpoint is currently up. None of those are great.

What structured patent data unlocks

It is worth being concrete about what you can actually do with a clean USPTO feed, because the use cases are broader than “file a patent lawsuit.”

Prior-art search and freedom-to-operate analysis. Before filing a new patent or shipping a new product, you want a structured search across granted US patents in the relevant area, filterable by date and assignee. Doing that across tens of thousands of records is only practical with a programmatic feed.
Competitive intelligence on R&D pipelines. Patent grant patterns are one of the few public signals about what a private company is actually building. Pulling all granted patents from a competitor over the last five years, grouping by inventor and class, gives you a rough map of where their engineering investment is going.
Investor and M&A diligence. When evaluating a deep-tech or hardware target, the patent estate is a real chunk of the underlying value. Bulk access to assignee-level patent data lets diligence teams produce concrete charts (filings per year, claim counts, top inventors) instead of vibes.
Academic and policy research. Empirical IP studies — on innovation clusters, university-industry tech transfer, citation patterns — depend on having structured patent data with consistent fields across decades.
Legal-tech and AI training data. Tools that summarize patents, surface prior art, or auto-classify claims need large corpora of granted patents with clean metadata: title, inventors, assignee, filing date, grant date, claim count, abstract.

The common thread across all of these is the same: you do not want a single patent document, you want a structured, queryable feed of patents matching a search.

What a usable patent record looks like

Whatever you build the collection layer on, the output shape that works downstream is narrow and flat. One record per granted patent, carrying the patent number, the title, the inventor list, the assignee, the filing date, the grant date, the claim count, the abstract, and a link back to the public record:

{
  "patent_number": "US11704715B2",
  "title": "Quantum computing service supporting multiple quantum computing technologies",
  "inventors": ["Christopher Kasprowicz", "Boyu Wang", "..."],
  "assignee": "Amazon Technologies, Inc.",
  "filing_date": "2019-11-27",
  "grant_date": "2023-07-18",
  "claims_count": 24,
  "abstract": "A quantum computing service includes connections to multiple quantum hardware providers...",
  "url": "https://ppubs.uspto.gov/pubwebapp/external.html?q=11704715"
}

That is the shape that drops cleanly into a pandas DataFrame, a Postgres table, or a vector index. Normalise assignee names on the way in — the same company appears under a dozen legal-entity spellings across decades — and keep both filing and grant dates, because most interesting questions are about the lag between them.

For prior-art workflows the pattern that scales is to pull once with a broad query to seed a local index, then re-run on a tight dateFrom filter to pick up new grants. That keeps the index fresh without re-pulling the historical corpus every day.

Doing serious patent research at scale? If you are pulling tens of thousands of records across many queries, or running long-form competitive-intelligence pipelines, residential proxies ensure consistent access for large-scale patent research even when source sites tighten their bot defences. We use Oxylabs residential proxies as the underlying network layer for jobs of that size.

Limits and what is out of scope

A few things are worth being honest about.

Everything above concerns US-granted patents. Published applications and non-US filings are a different pipeline with different endpoints.
Claim counts and abstracts come from the patent record itself, not from a separate full-text store. If you need every claim’s text or every drawing, that is a deeper extraction step on top.
For legal-grade use cases — freedom-to-operate opinions, infringement analysis — anything you assemble this way is research-grade data, not a substitute for a registered patent agent or attorney pulling certified records.