RealdataAPI Store - Browse tools published by our community and use them for your projects right away

Total Wine Scraper - Web Scraping Liquor Data

RealdataAPI / total-wine-scraper

Scrape wine product data, like brand name, images, descriptions, prices, stock information, and more, from the Total Wine website using Total Wine & More Scraper. The scraper is available in countries like the USA, UK, UAE, Canada, Australia, France, Germany, Spain, Italy, Mexico, Singapore, etc.

What is Total Wine & More Scraper?

Since the Total Wine & More platform has no official data API, this scraper will help you scrape the required product data.

The Total Wine & More scraper has the below features.

  • Extract lists: gather data for any list from the Total Wine & More platform.
  • Search from any query: you can use this scraper to search for any query to get the required results.

Updates, Bugs, Fixes & Changelogs

Our team is actively working on developing this scraper. If you want to suggest any feature, you can request us by creating an issue.

Setup and Usages To Scrape Total Wine & More Data

Enter the start URL or search query to use the Total Wine scraper. Presently, the scraper only supports the URL list as the start URL. Further, it would help to use a US-based proxy server to run the scraper. You can set the location or store using the ShippingInfo input field and collect the exact product listing.

Using Search

  • You can search the single keyword and set endPage, or maxItems if you want to.
  • The scraper goes through every search result page till it satisfies every condition.

Check out the output for this scraper example here.


{ "maxItems": 25, "endPage": 3, "proxy": { "useRealdataAPIProxy": true, "RealdataAPIProxyCountry": "US" }, "search": "sparkling", "shippingInfo": "ispStore~204:ifcStore~null@ifcStoreState~US-VA@method~INSTORE_PICKUP" }

Using Start URLs

  • Enter any start URL counts with endPage or maxItems according to your needs.
  • The scraper parses listed products for all URLs and go through the given start URL.

URL Query Parameters

  • The scraper uses the aty query parameter of the selected link if you provide it. If you don't, the tool uses the query parameter aty=1,1,0,1 to retrieve every existing product. Here is the sample URL:
  • If you provide it, the scraper keeps the query parameter pageSize for the selected pages. If you don't, it uses the parameter pageSize=24 to retrieve twenty-four products on each page. Here is the sample URL:

Check out the below result of the sample here.


{ "maxItems": 48, "endPage": 5, "proxy": { "useRealdataAPIProxy": true, "RealdataAPIProxyCountry": "US" }, "startUrls": [""], "shippingInfo": "ispStore~701:ifcStore~null@ifcStoreState~US-SC@method~INSTORE_PICKUP" }

Input Parameters

You must provide JSON Formatted input to this scraper with the list of Total Wine & More pages to visit. Here are the required input fields:

Field Type Description
search String It is an optional field for search queries you wish to search on the Total Wine platform.
startUrls Array It is an optional field for Total Wine & More web page URLs. The scraper only accepts product list links.
endPage Integer It is an optional input field to set the number of last web pages you wish to crawl with an infinite default. You can apply it to every start URL and search request separately.
shippingInfo String The field will set the location or store choice for the Total Wine website. The source website displays products for selected locations or stores based on these preferences. If you don't provide choices, the scraper uses the address of the first proxy server as the location to scrape the product data.
maxItems Integer You can restrict the collected products using this optional field. The field is valuable when you search big subcategories.
proxy Object Set proxy servers to execute the scraper successfully.
extendOutputFunction String It is an optional function to take the argument based on the JQuery handle ($) and give you the object with the required data.
customMapFunction String It is an optional function to take arguments based on every object and reflects objects with the function execution.

You've two options to set up proxies, like choosing them from our platform or using external proxies.


To scrape a particular listing link, only copy and paste the URL in the startURL field.

If you wish to extract only the first web page from the list, enter the link for that page and choose endPage as one.

As per the above explanation, you can retrieve any page. If you ask to scrape the 7th page and define the endPage parameter to 8, the scraper will only crawl the 7th and 8th pages.

If you want to extract all the products from the selected section, like white wine, red wine, and spirits, you must use the query parameter pageSize, which can increase the product counts for every request. The scraper may fail if the pageSize has a value greater than 200.

Use the EditThisCookie Chrome extension to scrape cookie value for the location or store choice. Check out the details to use the Chrome extension:

  • Visit any initial search or listing page to crawl.
  • Find the Pickup At section near the search bar, and edit it.
  • Use the EditThisCookie Chrome extension to find the twm-userStoreInformation cookie after selecting the location/store.
  • Copy the critical value, and paste it into the Shipping Info data field.

Input Example of Total Wine & More Scraper

{ "startUrls": [ ",0,0,1" ], "proxy": { "useRealdataAPIProxy": true, "RealdataAPIProxyCountry": "US" }, "endPage": 5, "maxItems": 100, "shippingInfo": "ispStore~701:ifcStore~null@ifcStoreState~US-SC@method~INSTORE_PICKUP" }

During the Execution

While executing, the scraper will send the message alert with the live scraping update. Every message includes a short label mentioning the current page it is scraping from the list. You have to read the message for the event with the total item count and loaded item count for every page after loading the items from the page.

The scraper will only run if you give the correct input. It will display the result with the failure reason in detail.

Export Total Wine & More Data

The Total Wine scraper saves the output in a customized dataset, with every item having a separate value and location.

You can handle the outputs in any coding language, like PHP, Python, NPM, or Node.js.

Properties of Scraped Total Wine & More Data

Check out the format of every item in the Total Wine & More:

Detailed Item Information

{ "bay": "", "brand": { "id": "Mascota Vineyards", "name": "Mascota Vineyards" }, "categories": [ { "id": "000284", "name": "Mendoza", "type": "REGION", "url": "", "storefrontUrl": "/wine/argentina/mendoza/c/000284" }, { "id": "000009", "name": "Red Wine", "type": "PRODUCT_TYPE", "url": "", "storefrontUrl": "/wine/red-wine/c/000009" }, { "id": "000011", "name": "Cabernet Sauvignon", "type": "VARIETAL_TYPE", "url": "", "storefrontUrl": "/wine/red-wine/cabernet-sauvignon/c/000011" } ], "containerType": "Bottle", "customerAverageRating": 4.4, "customerReviewsCount": 998, "department": "c0020", "directType": "Winery Direct", "id": "130007750", "images": [ { "imageType": "DEFAULT", "mobileOptimizedUrl": "", "thumbnailUrl": "", "url": "", "zoomImageUrl": "", "altText": "Mascota Vineyards Unanime, 2017" } ], "location": "Aisle 02, Left", "name": "Mascota Vineyards Unanime, 2017", "packageDescription": "750ml", "packageValue": "Single", "price": [ { "price": 25.99, "type": "EDLP" }, { "price": 19.99, "type": "LTSP" } ], "productUrl": "/wine/gift-center/deals/red-wine/cabernet-sauvignon/mascota-vineyards-unanime/p/130007750", "review": "James Suckling-Mendoza, Argentina - /"This is a rich, dense red with blackberry and blueberry aromas and flavors. Extremely well-crafted tannins. Hints of vanilla to the ripe fruit at the end. Incredible value...already beautiful to taste./"", "rating": 95, "ratingSource": "James Suckling", "salesStrategy": { "name": "Winery Direct" }, "shoppingOptions": [ { "eligible": false, "location": "US-TX", "message": ["Unavailable"], "type": "SHIPPING", "selected": false }, { "eligible": true, "location": "Oak Lawn, TX", "type": "INSTORE_PICKUP", "selected": true }, { "eligible": true, "location": "Oak Lawn, TX", "type": "DELIVERY", "selected": false } ], "skuId": "130007750-1", "stockLevel": [ { "purchaseLimit": 6, "stock": 211 } ], "storeDistance": 0.0001798902, "storeId": "531", "storeName": "Oak Lawn", "itemTasteProfile": "Blackberry, Chocolate", "itemStyle": "Elegant", "itemBody": "Full-bodied", "transactional": true, "type": "PRODUCT", "volume": "", "stockMessages": { "messages": [ { "shoppingMethod": "INSTORE_PICKUP", "stockMessage": "In stock", "addToCartMessage": "Add to cart", "addToCartStatus": true }, { "shoppingMethod": "DELIVERY", "stockMessage": "Available", "addToCartMessage": "Add to cart", "addToCartStatus": true }, { "shoppingMethod": "SHIPPING", "stockMessage": "Unavailable", "addToCartMessage": "View Product", "addToCartStatus": false } ], "digitalTransactional": true, "digitalInventoryQuantity": 211, "digitalSpecialOrder": false, "digitalLongTermOOS": false, "digitalLimitedStock": false, "digitalInStock": true, "digitalStoreQuantity": 213, "shippingTransactional": false, "shippingInventoryQuantity": 211, "shippingSpecialOrder": false, "shippingLongTermOOS": false, "shippingLimitedStock": false, "shippingInStock": true, "shippingStoreQuantity": 213, "digitalDeliveryEligible": true } }


Check out how industries use Total Wine & More Scraper worldwide.


E-commerce & Retail

You should have a Real Data API account to execute the program examples. Replace < YOUR_API_TOKEN> in the program using the token of your scraper. Read about the live APIs with Real Data API docs for more explanation.

import { RealdataAPIClient } from 'RealdataAPI-Client';

// Initialize the RealdataAPIClient with API token
const client = new RealdataAPIClient({
    token: '<YOUR_API_TOKEN>',

// Prepare actor input
const input = {
    "maxItems": 20,
    "endPage": 1,
    "startUrls": [
    "extendOutputFunction": ($) => { return {} },
    "customMapFunction": (object) => { return {...object} },
    "proxy": {
        "useRealdataAPIProxy": true,
        "RealdataAPIProxyCountry": "US"

(async () => {
    // Run the actor and wait for it to finish
    const run = await"oyildirim/totalwine-actor").call(input);

    // Fetch and print actor results from the run's dataset (if any)
    console.log('Results from dataset');
    const { items } = await client.dataset(run.defaultDatasetId).listItems();
    items.forEach((item) => {
from RealdataAPI_client import RealdataAPIClient

# Initialize the RealdataAPIClient with your API token
client = RealdataAPIClient("<YOUR_API_TOKEN>")

# Prepare the actor input
run_input = {
    "maxItems": 20,
    "endPage": 1,
    "startUrls": [""],
    "extendOutputFunction": "($) => { return {} }",
    "customMapFunction": "(object) => { return {...object} }",
    "proxy": {
        "useRealdataAPIProxy": True,
        "RealdataAPIProxyCountry": "US",

# Run the actor and wait for it to finish
run ="oyildirim/totalwine-actor").call(run_input=run_input)

# Fetch and print actor results from the run's dataset (if there are any)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
# Set API token

# Prepare actor input
cat > input.json <<'EOF'
  "maxItems": 20,
  "endPage": 1,
  "startUrls": [
  "extendOutputFunction": "($) => { return {} }",
  "customMapFunction": "(object) => { return {...object} }",
  "proxy": {
    "useRealdataAPIProxy": true,
    "RealdataAPIProxyCountry": "US"

# Run the actor
curl "$API_TOKEN" /
  -X POST /
  -d @input.json /
  -H 'Content-Type: application/json'

Search Query

search Optional String

It is the field to use search queries you want to search on the Total Wine website.

Cookie Value of Shipping Info

shippingInfo Optional String

You can set the value of the cookie and get the product listing for the selected location.

Maximum Listing Item Counts

maxItems Optional Integer

Set the maximum listing item counts in the result with all as the default.

List End Page

endPage Optional Integer

It is to list out the number of the last page. There is no last page by default. You can apply it to every start URL and search request separately.

Start URLs

startUrls Optional Array

Enter the URLs to start the scraping process. You have to enter the list of URLs.

Extend Output Function

extendOutputFunction Optional String

It takes an argument based on the JQuery handle ($) and reflects the output to merge with the default result.

Custom Map Function

customMapFunction Optional String

It takes the argument based on objects and gives the output data such that the function maps itself.

Proxy Configuration

proxy Optional Object

Choose the proxy server to help the scraper. Use US-based locations for the proxy server.

  "maxItems": 20,
  "endPage": 1,
  "startUrls": [
  "extendOutputFunction": "($) => { return {} }",
  "customMapFunction": "(object) => { return {...object} }",
  "proxy": {
    "useRealdataAPIProxy": true,
    "RealdataAPIProxyCountry": "US"