How to scrape Instagram profiles with an API (Node & Python)

Instagram profile data is one of the most requested data sources for marketing tools, influencer platforms and lead enrichment. Follower counts, bios, website links and recent posts tell you a lot about an account. Getting that data reliably is harder than it looks: Instagram's official APIs only cover accounts that authorize your app, and scraping the website yourself means dealing with login walls, rate limits, changing page structures and proxy management.

This guide shows how to fetch public Instagram profile data with a scraper API, using a single HTTP request per profile. The examples use the Scrape Everything Instagram API, but the patterns (timeouts, retries, caching, cost control) apply to any provider.

What data you can get from a public profile#

A public Instagram profile exposes the same information anyone sees when they open the profile in a browser without logging in. The profile endpoint returns it as typed JSON:

• Identity: id, username, full_name, is_verified.
• Bio: biography, external_url (the link in bio) and category_name for business and creator accounts.
• Counts: follower_count, following_count and post_count.
• Account type: is_private and is_business_account.
• Images: profile_pic_url and profile_pic_url_hd.

If you also need recent posts, the posts endpoint returns the profile plus up to 60 recent posts with type, caption, like and comment counts, thumbnail and timestamp. Private accounts only return what is public: their counts and bio, not their posts.

Step 1: get an API key#

1. Sign in with Google. New accounts receive 100 free credits, which is enough for 100 profile lookups.
2. Create a key on the API keys page. Optionally restrict it to your server's IP addresses.
3. Store it in an environment variable called SCRAPEEVERYTHING_API_KEY. Never put API keys in front-end code: the API does not accept browser requests, and a key in a browser bundle is a key anyone can copy.

Step 2: fetch a profile#

Every endpoint is a POST with a small JSON body. For a profile, the body is the username. You can also pass an @handle or a full profile URL; the API normalizes it.

curl -X POST https://api.scrapeeverything.com/v1/instagram/profile \
  -H "Authorization: Bearer $SCRAPEEVERYTHING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"username": "nasa"}'

The response always has the same envelope: success, the endpoint's data and a meta object that tells you what the request cost.

{
  "success": true,
  "data": {
    "username": "nasa",
    "full_name": "NASA",
    "is_verified": true,
    "is_private": false,
    "follower_count": 98000000,
    "post_count": 4300,
    "external_url": "https://www.nasa.gov/"
  },
  "meta": {
    "request_id": "req_...",
    "cached": false,
    "credits_charged": 1,
    "credits_remaining": 99,
    "duration_ms": 1840
  }
}

Node.js example with timeout and retries#

Most requests finish in a few seconds. When Instagram blocks a request, the API retries through other proxies on its side, so a small share of requests can take up to a minute. Set a generous client timeout (90 seconds) and only retry errors that are worth retrying: 429 (rate limited) and 502, 503, 504.

const API = "https://api.scrapeeverything.com/v1";

async function getInstagramProfile(username, attempt = 0) {
  const res = await fetch(`${API}/instagram/profile`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.SCRAPEEVERYTHING_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ username, cache_days: 1 }),
    signal: AbortSignal.timeout(90_000),
  });

  const retryable = [429, 502, 503, 504].includes(res.status);
  if (retryable && attempt < 3) {
    const wait = Number(res.headers.get("RateLimit-Reset")) || 2 ** attempt;
    await new Promise((r) => setTimeout(r, wait * 1000));
    return getInstagramProfile(username, attempt + 1);
  }

  const body = await res.json();
  if (!body.success) throw new Error(`${body.code}: ${body.message}`);
  return body.data;
}

const profile = await getInstagramProfile("nasa");
console.log(profile.full_name, profile.follower_count);

Python example#

The same logic with requests. For larger batches, use a thread pool or httpx with async, but keep the concurrency at or below your account's concurrency limit.

import os
import time
import requests

API = "https://api.scrapeeverything.com/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['SCRAPEEVERYTHING_API_KEY']}"}

def get_instagram_profile(username: str, attempt: int = 0) -> dict:
    res = requests.post(
        f"{API}/instagram/profile",
        headers=HEADERS,
        json={"username": username, "cache_days": 1},
        timeout=90,
    )
    if res.status_code in (429, 502, 503, 504) and attempt < 3:
        wait = int(res.headers.get("RateLimit-Reset") or 2 ** attempt)
        time.sleep(wait)
        return get_instagram_profile(username, attempt + 1)

    body = res.json()
    if not body["success"]:
        raise RuntimeError(f"{body['code']}: {body['message']}")
    return body["data"]

profile = get_instagram_profile("nasa")
print(profile["full_name"], profile["follower_count"])

Step 3: handle the answers that are not data#

A robust integration distinguishes between three kinds of non-success responses, because each one needs a different reaction:

The full list is in the errors reference. The important part for budgeting is the last column: you are not charged for failures on our side, so retries do not double your bill.

Step 4: control cost with caching#

Many workloads look up the same profiles again and again: a dashboard refreshing every hour, a CRM enrichment job that runs nightly, a user who opens the same creator page twice. Sending cache_days tells the API that a result up to that many days old is fine. If your account fetched the same profile within the window, you get the stored result back for 0 credits and meta.cached is true.

• Use cache_days: 1 for dashboards that do not need minute-level freshness.
• Use a larger value (up to 14) for enrichment, where a follower count from last week is good enough.
• Omit cache_days when you need live data, for example right before publishing a report.

Cached results belong to your account only; they are never shared with other customers. Details are in the caching guide.

Step 5: scale to thousands of profiles#

For bulk jobs, the limiting factor is your account's rate limit and concurrency, not the code. A simple pattern that works well:

1. Put the usernames in a queue (a database table, Redis list or even a file).
2. Run a fixed number of workers equal to your concurrency limit.
3. Each worker takes a username, calls the API with cache_days, and stores the result together with meta.request_id and the time you fetched it.
4. On 429, the worker sleeps until the reset time. On NOT_FOUND, it marks the username as invalid.

Free accounts can send 60 requests per minute with 5 in parallel; once your purchases total $39 or more (a Builder pack or larger does it at once) the limits rise to 300 per minute and 20 in parallel. At 1 credit per profile, 25,000 lookups fit in the Builder pack. See pricing for the current numbers.

Why not the official Instagram API?#

If you only need data about accounts that log in to your app and grant permissions, Instagram's official APIs are the right tool and you should use them. They do not let you look up arbitrary public profiles, though, which is what most research, marketing and enrichment use cases need. We compare the two approaches in detail in Instagram Graph API vs a scraper API.

Use the data responsibly#

A scraper API only returns data that is publicly visible without logging in, but public does not mean free of rules. Profiles belong to people, so data protection laws such as the GDPR may apply to how you store and use the data. Collect only what you need, keep it for as long as you need it, and respect deletion requests. Our overview of the legal landscape is a starting point, not legal advice.

Next steps#

• Try the request in the browser playground after signing in.
• Read the Instagram API reference for every field and parameter.
• Fetch recent posts with the Instagram posts endpoint or comments with the comments endpoint.