How to use the NPI scraper

1. Overview

The NPI scraper pulls provider records from the free CMS National Plan & Provider Enumeration System (NPPES) API. It searches by taxonomy (specialty) and optionally filters by state and enrollment date. Results are saved to the PayScope leads database — duplicates are automatically skipped.

2. Step-by-step: Running a scrape

1. Go to the Leads page (/leads)
2. Click "Scraper Settings" to expand the configuration panel
3. Configure your filters:
   • States — select one or more states to target. Leave all unchecked to scrape nationwide.
   • Taxonomies — choose one or more provider specialties. Defaults to ENT + General Dentist.
   • Enrolled After / Before — optionally filter by the date the provider registered with CMS. Useful for targeting newer practices.
4. Click "Sync NPI Data" to start the scrape
5. Wait for the scrape to complete — the leads table will refresh with new results

3. Understanding the filters

• States: Narrows results to providers whose practice address is in the selected state(s). Running nationwide pulls significantly more records but takes longer.
• Taxonomies: Each taxonomy corresponds to a medical specialty. Multiple taxonomies can be selected — the scraper runs a separate pass for each one.
• Enrolled After / Before: Filters by the provider's CMS enumeration date. Newer enrollments (e.g. after 2020) may indicate practices that recently became independent and are still setting up their billing operations.

4. What data gets imported

Each lead includes:

• Provider name
• Specialty (mapped from taxonomy)
• Office phone number
• State
• Enrollment date (when they registered with CMS)

5. Tips

• Start with a specific state + one or two taxonomies before running a nationwide pull — it's faster and easier to QA.
• Use Enrolled After to target newer practices (e.g. "after Jan 2021") — they're often still optimizing their billing setup.
• The NPI Registry has ~7 million active providers. You can run multiple targeted pulls without hitting limits.
• Re-running the same filters is safe — duplicates are skipped automatically.