Buy fast & affordable proxy servers. Get 10 proxies today for free.
Download our Proxy Server Extension
Products
© Webshare Proxy
payment methods
If you have tried scraping Amazon and hit a wall, you are not alone. Amazon is one of the most aggressively protected sites on the web, and a plain Python script that works fine on smaller targets can start failing after just a handful of requests.
Still, people keep attempting Amazon scraping because the data is genuinely valuable. It offers more than 350 million products, along with live prices, ratings, availability, reviews, and other product information. That data supports price monitoring, competitor intelligence, market research, and AI training datasets. However, there’s no single unrestricted public API that provides all of the data a scraper might need.
So you try to scrape the ecommerce platform, and encounter 503 errors, CAPTCHAs instead of the product, and honeypot traps. Meanwhile, the pricing data you need is missing from the raw HTML because it's dynamically loaded through JavaScript.
This guide explains how to scrape Amazon in 2026 (with copyable code), what tools you need, and what actually holds up in practice.
Before writing any code, it helps to know which page you are targeting, because each Amazon page type holds different data and may require different selectors.
Most scraping projects fall into four page types, and the table below maps each one to the fields worth extracting and the selectors commonly used to find them.
Knowing this upfront lets you scrape Amazon product data, search results, and reviews without guessing at structure mid-run.
These page types also paginate differently. If you scrape Amazon search results, they use the page parameter, while to scrape Amazon reviews, you use pageNumber instead. Mixing up the two is a common early mistake. Amazon’s HTML can and has changed in the past, so it's important to validate that the selectors you’re using are still valid.
Amazon runs one of the most layered anti-bot stacks of any consumer site, and understanding what it checks tells you exactly what your scraper has to get right.
Here’s what trips up most scripts:
Another factor that can affect web scrapers is that Amazon changed its HTML in 2025.
Legacy selectors like #priceblock_ourprice and #priceblock_dealprice may no longer work, so newer selectors such as .a-price .a-offscreen for prices and #productTitle for product titles are generally more useful. However, because Amazon periodically A/B tests its markup, wrap your selectors in error handling so your scraper can handle changes without failing
Amazon’s APIs aren’t the best route for scraping because they weren’t built for data collection.
For many developers, the Product Advertising API was the go-to option, but Amazon deprecated it in May 2026 and replaced it with the Creators API.
The Creators API, like its predecessor, is gated behind Amazon’s Associates program. This means you must meet the program’s requirements for the marketplace you want to access. This includes generating at least 10 qualified sales in the past 30 days to access the PA API. If you don’t meet that target, your API access can be paused until your sales recover.
Reviews aren’t included either. The API only gives you a star rating and a count, not the actual review text.
Also, access is gated by your affiliate performance rather than your project's needs. Under the legacy PA-API, this meant starting at one request per second (capped at 8,640/day) and earning one more request per second for every $4,320 in shipped affiliate orders, up to 10/sec. As of the Creators API migration (PA-API fully retired May 15, 2026), the baseline requirement shifted to a flat 10 qualifying sales in the trailing 30 days just to unlock API access at all — with rate limits scaling from there. Either way, the ceiling is tied to your sales performance, not your scraping needs.
If you’re scraping Amazon for price monitoring or review data, you have no audience to sell to, so you’d have to build an entirely unrelated affiliate business first just to unlock access to the work you came to do.
As we’ll learn in this guide and see how to apply it in practice, the fix involves realistic headers and request timing, Webshare rotating residential proxies, and a Playwright headless browser for JavaScript content.
You’ll need Python 3.9 or newer. Check your version by running the command below:
python --versionThen, we’ll need the following libraries, so install them:
pip install requests beautifulsoup4 lxml playwright brotli
playwright install chromiumNext, sign up for a free Webshare account and copy your rotating residential proxy endpoint from the dashboard. The endpoint routes every request through an 80M+ IP pool and rotates you to a fresh IP on each request.
Paste it into your script like this:
http://username-rotate:password@p.webshare.io:80To pull IPs from a specific country, append the country code to the username, for example, username-rotate-us for the United States. This keeps your Amazon data consistent because product prices, availability, shipping options, and search results can vary between marketplaces and regions.
Webshare’s free plan includes 10 proxies with no credit card required, which is enough to wire up and test these scripts before you scale. For sustained Amazon scraping in production, the rotating residential plan is what you want, since its IP pool is what keeps you unblocked at volume.
This is the primary code path for scraping Amazon product data, and where most Amazon scraper builds start. It pairs requests with Beautiful Soup, the combination that nearly every Amazon web scraper in Python begins with.
If you’re new to scraping Amazon in Python, start here before reaching for anything heavier.
Below are the steps involved.
A plain request is the fastest way to see the problem. Send a plain request to a product page with no headers and no proxy, and check the status code.
There are two possible responses you’ll get:
One way to prevent requests from flagging is to make them look like they came from a real browser instead of scripts.
Real browsers send a set of headers that are absent from Python’s default request. These include:
With these headers, simple, low-volume scrapes will go through. However, Amazon’s blockers will still be triggered at high-volume workloads because headers don’t solve the IP.
Enough requests from one address, and the platform’s rate limiter will kick in, meaning you still need a rotating proxy like what Webshare offers.
Routing every request through the Webshare rotating endpoint gives each one a different residential IP, so no single address carries enough traffic to trip the rate limiter.
This way, the endpoint rotates automatically, and the proxy URL stays http:// even for HTTPS targets, since that refers to how you reach the proxy, not the site you are scraping.
A request to a product page returns the full HTML of that page as a single string. Pass that string to BeautifulSoup and let a parse_product helper pull out the title, price, rating, review count, and availability by their CSS selector.
Consider these factors regarding the selectors:
In real workflows, you’re rarely fetching one product. Instead, you’re pulling hundreds or thousands. To gather them at once, wrap the request and parse in a function, then loop your ASIN (Amazon Standard Identification Number) list.
To keep this stable at volume, you need two things:
Finally, the results are written out to both JSON, which keeps every row including errors, and CSV, which gives you a flat file for a spreadsheet:
Here's the complete code:
import requests
import time
import random
import json
import csv
from bs4 import BeautifulSoup
# Step 2: headers that make the request look like a real Chrome session
headers = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
),
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Referer": "https://www.amazon.com/",
}
# Step 3: Webshare rotating residential endpoint, a fresh IP per request
proxies = {
"http": "http://username-rotate:password@p.webshare.io:80",
"https": "http://username-rotate:password@p.webshare.io:80",
}
# Step 4: pull the product fields out of the HTML
def parse_product(html, asin):
soup = BeautifulSoup(html, "lxml")
def text(selector): # stripped text, or None if the selector is missing
el = soup.select_one(selector)
return el.get_text(strip=True) if el else None
# Rating lives in the #acrPopover title attribute, e.g. "4.5 out of 5 stars"
rating_el = soup.select_one("#acrPopover")
rating = rating_el["title"].split()[0] if rating_el and rating_el.has_attr("title") else None
return {
"asin": asin,
"title": text("#productTitle"),
"price": text(".a-price .a-offscreen"), # fragile: varies by product type, changes often
"rating": rating,
"review_count": text("#acrCustomerReviewText"),
"availability": text("#availability"),
}
# Steps 1-3: request a product page through the headers and rotating proxy
def scrape_product(asin):
url = f"https://www.amazon.com/dp/{asin}"
response = requests.get(url, headers=headers, proxies=proxies, timeout=30)
if response.status_code != 200: # record anything that is not a clean 200
return {"asin": asin, "error": response.status_code}
return parse_product(response.text, asin)
# Step 5: scrape multiple ASINs with a random delay between requests
asins = ["B0CX23V2ZK", "B08N5WRWNW", "B09B8V1LZ3"]
results = []
for asin in asins:
results.append(scrape_product(asin))
time.sleep(random.uniform(2, 5)) # human-like gap between requests
# Step 6: export results to JSON (everything) and CSV (successful rows only)
with open("products.json", "w", encoding="utf-8") as f:
json.dump(results, f, indent=2, ensure_ascii=False)
fieldnames = ["asin", "title", "price", "rating", "review_count", "availability"]
with open("products.csv", "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames, extrasaction="ignore")
writer.writeheader()
writer.writerows(row for row in results if "error" not in row)
Get your free Webshare proxy credentials → 10 proxies, no credit card, works immediately with the code above.
If you don’t have a list of ASINs, you can build one by scraping Amazon search results. Each search results page follows a predictable web address pattern, https://www.amazon.com/s?k=QUERY&page=N, where QUERY is your keyword, and N is the page number.
Every product in the results carries its ASIN in a data-asin attribute, so a single search page hands you a batch of them at once.
From each result, the scraper pulls:
You’ll need to paginate, because a single search can return hundreds of products spread across many pages. First, read the total result count off the first page, divide it by the results per page to get your page total, then loop through with &page=N.
Route each request through your Webshare rotating proxy, since a run of sequential requests to the same search from one IP is exactly the pattern the rate limiter watches for.
Here's the complete code:
import requests
import time
import random
import re
import math
from urllib.parse import quote_plus
from bs4 import BeautifulSoup
# Reuses the headers and proxies dict from Approach 1
def scrape_search(query, page=1):
url = f"https://www.amazon.com/s?k={quote_plus(query)}&page={page}"
response = requests.get(url, headers=headers, proxies=proxies, timeout=30)
soup = BeautifulSoup(response.text, "lxml")
results = []
for item in soup.select('[data-component-type="s-search-result"]'): # one block per product
asin = item.get("data-asin")
if not asin: # skip banners and non-product rows
continue
title_el = item.select_one("h2 a span") or item.select_one(".a-size-base-plus")
price_el = item.select_one(".a-price .a-offscreen")
rating_el = item.select_one("[aria-label*='out of 5 stars']")
sponsored = item.select_one(".puis-sponsored-label-text") is not None # sponsored flag
results.append({
"asin": asin,
"title": title_el.get_text(strip=True) if title_el else None,
"price": price_el.get_text(strip=True) if price_el else None,
"rating": rating_el["aria-label"].split()[0] if rating_el else None,
"sponsored": sponsored,
})
return results
def get_page_count(query, per_page=48):
url = f"https://www.amazon.com/s?k={quote_plus(query)}"
response = requests.get(url, headers=headers, proxies=proxies, timeout=30)
soup = BeautifulSoup(response.text, "lxml")
# Read "1-48 of X results" from the results bar, then divide to get the page total
# Fragile: Amazon changes this bar's wording and class, so guard it and cap the result
info = soup.select_one('[data-component-type="s-result-info-bar"]')
match = re.search(r"of(?:\s+over)?\s+([\d,]+)\s+results", info.get_text()) if info else None
if not match:
return 1
total = int(match.group(1).replace(",", ""))
return min(math.ceil(total / per_page), 20) # cap at 20 pages; Amazon rarely serves more
# Detect the page count, then loop through each page with &page=N
query = "wireless earbuds"
pages = get_page_count(query)
all_products = []
for page in range(1, pages + 1):
all_products.extend(scrape_search(query, page=page))
time.sleep(random.uniform(2, 5)) # pause between pages
print(f"Collected {len(all_products)} products across {pages} pages")
requests works when the data you need is sitting in the raw HTML. However, scraping Amazon with Playwright becomes necessary when the content is loaded dynamically through JavaScript, which on Amazon covers:
Playwright opens an actual Chromium browser in the background, so the JavaScript executes just like it would for a real visitor.
With Playwright, you can:
Here’s the complete code:
import asyncio
from playwright.async_api import async_playwright
async def scrape_product_js(asin):
async with async_playwright() as p:
browser = await p.chromium.launch(
headless=True,
proxy={ # Webshare rotating residential proxy
"server": "http://p.webshare.io:80",
"username": "username-rotate",
"password": "password",
},
args=[
"--no-sandbox",
"--disable-blink-features=AutomationControlled", # drop the automation fingerprint
],
)
context = await browser.new_context(
user_agent=(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
),
viewport={"width": 1366, "height": 768},
)
# Report navigator.webdriver as false, the value a real browser shows
await context.add_init_script(
"Object.defineProperty(navigator, 'webdriver', {get: () => false})"
)
page = await context.new_page()
await page.goto(f"https://www.amazon.com/dp/{asin}",
wait_until="domcontentloaded", timeout=60000)
# Dismiss the cookie banner if it appears (only shows in some regions)
try:
await page.click("#sp-cc-accept", timeout=5000)
except Exception:
pass
# Wait for the JS-rendered price to attach before reading anything
await page.wait_for_selector(".a-price .a-offscreen", timeout=15000)
data = {
"asin": asin,
"title": await page.locator("#productTitle").inner_text(),
"price": await page.locator(".a-price .a-offscreen").first.inner_text(),
"availability": await page.locator("#availability").inner_text(),
}
await browser.close()
return data
print(asyncio.run(scrape_product_js("B0CX23V2ZK")))
You also need to choose between sticky sessions and per-request rotation.
Sticky sessions are ideal when you’re driving a browser, since a single page load fires dozens of sub-requests, and holding one IP across all of them keeps the page from looking like it came from many visitors at once. Webshare lets you hold that IP for a window you set, from 1 minute to 24 hours.
Per-request rotation is the better fit for the stateless requests approach, where each request stands on its own, and a fresh IP address each time spreads the load.
Amazon keeps reviews on a separate page from the product listing.
To scrape Amazon reviews, target amazon.com/product-reviews/ASIN?pageNumber=N. Navigate through by iterating pageNumber from 1 to the calculated max (total reviews ÷ 10).
Note that review pages are slightly less protected than product pages, but still need proxies at scale.
The code below fetches various review-related details.
import requests
import json
import math
import re
from bs4 import BeautifulSoup
# Reuses the headers and proxies dict from Approach 1
def get_review_pages(asin):
url = f"https://www.amazon.com/product-reviews/{asin}?pageNumber=1"
response = requests.get(url, headers=headers, proxies=proxies, timeout=30)
soup = BeautifulSoup(response.text, "lxml")
# Read the review count off the page, then divide by 10 for the page total
# Fragile: Amazon changes this element, so guard it and cap the result
info = soup.select_one('[data-hook="cr-filter-info-review-rating-count"]')
match = re.search(r"([\d,]+)\s+with reviews", info.get_text()) if info else None
if not match:
return 1
total = int(match.group(1).replace(",", ""))
return min(math.ceil(total / 10), 50) # cap at 50 pages
def scrape_reviews(asin):
reviews = []
for page in range(1, get_review_pages(asin) + 1):
url = f"https://www.amazon.com/product-reviews/{asin}?pageNumber={page}"
response = requests.get(url, headers=headers, proxies=proxies, timeout=30)
soup = BeautifulSoup(response.text, "lxml")
# Each review card carries its fields in data-hook attributes
for review in soup.select('[data-hook="review"]'):
reviewer = review.select_one(".a-profile-name")
body = review.select_one('[data-hook="review-body"]')
rating = review.select_one('[data-hook="review-star-rating"] .a-icon-alt')
date = review.select_one('[data-hook="review-date"]')
helpful = review.select_one('[data-hook="helpful-vote-statement"]')
verified = review.select_one('[data-hook="avp-badge"]') is not None
reviews.append({
"asin": asin,
"reviewer_name": reviewer.get_text(strip=True) if reviewer else None,
"text": body.get_text(strip=True) if body else None,
"rating": rating.get_text(strip=True).split()[0] if rating else None,
"date": date.get_text(strip=True) if date else None,
"verified_purchase": verified,
"helpful_votes": helpful.get_text(strip=True) if helpful else None,
})
return reviews
# Scrape every review page and save the results to JSON
data = scrape_reviews("B0CX23V2ZK")
with open("reviews.json", "w", encoding="utf-8") as f:
json.dump(data, f, indent=2, ensure_ascii=False)
print(f"Collected {len(data)} reviews")
Once you’re pulling tens of thousands of pages, looping requests by hand gets slow and fragile. Scrapy handles that scale for you, with built-in concurrency and retries.
The only Scrapy-specific part is wiring in the proxy. One way is using a middleware that attaches your Webshare endpoint to every request. You can switch this on in a file (settings.py in the code below).
Here’s the whole thing, middleware and spider together:
# settings.py
# Turn on the proxy middleware (lower number = runs earlier)
DOWNLOADER_MIDDLEWARES = {
"amazon_scraper.middlewares.WebshareProxyMiddleware": 350,
}
DOWNLOAD_DELAY = 3 # wait a few seconds between requests
RANDOMIZE_DOWNLOAD_DELAY = True # vary that wait so it doesn't look robotic
# middlewares.py
class WebshareProxyMiddleware:
def process_request(self, request, spider):
# Send every request out through your Webshare endpoint
request.meta["proxy"] = "http://username-rotate:password@p.webshare.io:80"
# spiders/products.py
import scrapy
class ProductsSpider(scrapy.Spider):
name = "products"
asins = ["B0CX23V2ZK", "B08N5WRWNW"]
def start_requests(self):
for asin in self.asins:
yield scrapy.Request(f"https://www.amazon.com/dp/{asin}",
cb_kwargs={"asin": asin})
def parse(self, response, asin):
yield {
"asin": asin,
"title": response.css("#productTitle::text").get(default="").strip(),
"price": response.css(".a-price .a-offscreen::text").get(),
"review_count": response.css("#acrCustomerReviewText::text").get(),
}If you’d rather not write the middleware, the scrapy-rotating-proxies package does the same job: drop your endpoints into a ROTATING_PROXY_LIST and the package handles the rotation for you.
If you prefer SOCKS5 over HTTP proxies, Webshare supports both protocols at no additional cost, so you can use the same proxy pool with Scrapy without changing your proxy plan.
Each approach trades complexity for block resistance. Use the lightest one that clears your target:
There are some errors you’ll likely experience regardless of how well-built your Amazon scraper is:
Here are some best practices that can help keep your data extraction runs stable at scale:
Amazon’s Terms of Service restrict automated access, so scraping is done at your own risk, particularly for commercial use.
On the legal side, under U.S. case law, the Ninth Circuit's ruling in hiQ Labs v. LinkedIn found that scraping publicly accessible data is unlikely to violate the Computer Fraud and Abuse Act (CFAA) - a preliminary injunction ruling, not a final judgment, since the parties settled privately in 2022 before the case concluded. Separately, a district court found hiQ still breached LinkedIn's Terms of Service through its scraping, a contract claim independent of the CFAA question. In short: public-data scraping is on relatively solid CFAA footing, but ToS violations remain a distinct legal risk.
One best practice to follow is respecting Amazon's robots.txt; don’t use scraped data to deceive customers or violate Amazon’s brand guidelines. Also, for commercial use cases, it’s best to speak with a lawyer.