Scrape wine product data, like brand name, images, descriptions, prices, stock information, and more, from the Total Wine website using Total Wine & More Scraper. The scraper is available in countries like the USA, UK, UAE, Canada, Australia, France, Germany, Spain, Italy, Mexico, Singapore, etc.
Since the Total Wine & More platform has no official data API, this scraper will help you scrape the required product data.
The Total Wine & More scraper has the below features.
Our team is actively working on developing this scraper. If you want to suggest any feature, you can request us by creating an issue.
Enter the start URL or search query to use the Total Wine scraper. Presently, the scraper only supports the URL list as the start URL. Further, it would help to use a US-based proxy server to run the scraper. You can set the location or store using the ShippingInfo input field and collect the exact product listing.
Check out the output for this scraper example here.
{ "maxItems": 25, "endPage": 3, "proxy": { "useRealdataAPIProxy": true, "RealdataAPIProxyCountry": "US" }, "search": "sparkling", "shippingInfo": "ispStore~204:ifcStore~null@ifcStoreState~US-VA@method~INSTORE_PICKUP" }
Check out the below result of the sample here.
{ "maxItems": 48, "endPage": 5, "proxy": { "useRealdataAPIProxy": true, "RealdataAPIProxyCountry": "US" }, "startUrls": ["https://www.totalwine.com/wine/red-wine/c/000009"], "shippingInfo": "ispStore~701:ifcStore~null@ifcStoreState~US-SC@method~INSTORE_PICKUP" }
You must provide JSON Formatted input to this scraper with the list of Total Wine & More pages to visit. Here are the required input fields:
| Field | Type | Description |
|---|---|---|
| search | String | It is an optional field for search queries you wish to search on the Total Wine platform. |
| startUrls | Array | It is an optional field for Total Wine & More web page URLs. The scraper only accepts product list links. |
| endPage | Integer | It is an optional input field to set the number of last web pages you wish to crawl with an infinite default. You can apply it to every start URL and search request separately. |
| shippingInfo | String | The field will set the location or store choice for the Total Wine website. The source website displays products for selected locations or stores based on these preferences. If you don't provide choices, the scraper uses the address of the first proxy server as the location to scrape the product data. |
| maxItems | Integer | You can restrict the collected products using this optional field. The field is valuable when you search big subcategories. |
| proxy | Object | Set proxy servers to execute the scraper successfully. |
| extendOutputFunction | String | It is an optional function to take the argument based on the JQuery handle ($) and give you the object with the required data. |
| customMapFunction | String | It is an optional function to take arguments based on every object and reflects objects with the function execution. |
You've two options to set up proxies, like choosing them from our platform or using external proxies.
To scrape a particular listing link, only copy and paste the URL in the startURL field.
If you wish to extract only the first web page from the list, enter the link for that page and choose endPage as one.
As per the above explanation, you can retrieve any page. If you ask to scrape the 7th page and define the endPage parameter to 8, the scraper will only crawl the 7th and 8th pages.
If you want to extract all the products from the selected section, like white wine, red wine, and spirits, you must use the query parameter pageSize, which can increase the product counts for every request. The scraper may fail if the pageSize has a value greater than 200.
Use the EditThisCookie Chrome extension to scrape cookie value for the location or store choice. Check out the details to use the Chrome extension:
{ "startUrls": [ "https://www.totalwine.com/wine/red-wine/c/000009?page=3&viewAll=true&pageSize=100&aty=1,0,0,1" ], "proxy": { "useRealdataAPIProxy": true, "RealdataAPIProxyCountry": "US" }, "endPage": 5, "maxItems": 100, "shippingInfo": "ispStore~701:ifcStore~null@ifcStoreState~US-SC@method~INSTORE_PICKUP" }
While executing, the scraper will send the message alert with the live scraping update. Every message includes a short label mentioning the current page it is scraping from the list. You have to read the message for the event with the total item count and loaded item count for every page after loading the items from the page.
The scraper will only run if you give the correct input. It will display the result with the failure reason in detail.
The Total Wine scraper saves the output in a customized dataset, with every item having a separate value and location.
You can handle the outputs in any coding language, like PHP, Python, NPM, or Node.js.
Check out the format of every item in the Total Wine & More:
{ "bay": "", "brand": { "id": "Mascota Vineyards", "name": "Mascota Vineyards" }, "categories": [ { "id": "000284", "name": "Mendoza", "type": "REGION", "url": "https://www.totalwine.com/wine/argentina/mendoza/c/000284", "storefrontUrl": "/wine/argentina/mendoza/c/000284" }, { "id": "000009", "name": "Red Wine", "type": "PRODUCT_TYPE", "url": "https://www.totalwine.com/wine/red-wine/c/000009", "storefrontUrl": "/wine/red-wine/c/000009" }, { "id": "000011", "name": "Cabernet Sauvignon", "type": "VARIETAL_TYPE", "url": "https://www.totalwine.com/wine/red-wine/cabernet-sauvignon/c/000011", "storefrontUrl": "/wine/red-wine/cabernet-sauvignon/c/000011" } ], "containerType": "Bottle", "customerAverageRating": 4.4, "customerReviewsCount": 998, "department": "c0020", "directType": "Winery Direct", "id": "130007750", "images": [ { "imageType": "DEFAULT", "mobileOptimizedUrl": "https://www.totalwine.com/media/sys_master/twmmedia/h72/h2e/15949509165086.png", "thumbnailUrl": "https://www.totalwine.com/media/sys_master/twmmedia/h72/h2e/15949509165086.png", "url": "https://www.totalwine.com/media/sys_master/twmmedia/h72/h2e/15949509165086.png", "zoomImageUrl": "https://www.totalwine.com/media/sys_master/twmmedia/h72/h2e/15949509165086.png", "altText": "Mascota Vineyards Unanime, 2017" } ], "location": "Aisle 02, Left", "name": "Mascota Vineyards Unanime, 2017", "packageDescription": "750ml", "packageValue": "Single", "price": [ { "price": 25.99, "type": "EDLP" }, { "price": 19.99, "type": "LTSP" } ], "productUrl": "/wine/gift-center/deals/red-wine/cabernet-sauvignon/mascota-vineyards-unanime/p/130007750", "review": "James Suckling-Mendoza, Argentina - /"This is a rich, dense red with blackberry and blueberry aromas and flavors. Extremely well-crafted tannins. Hints of vanilla to the ripe fruit at the end. Incredible value...already beautiful to taste./"", "rating": 95, "ratingSource": "James Suckling", "salesStrategy": { "name": "Winery Direct" }, "shoppingOptions": [ { "eligible": false, "location": "US-TX", "message": ["Unavailable"], "type": "SHIPPING", "selected": false }, { "eligible": true, "location": "Oak Lawn, TX", "type": "INSTORE_PICKUP", "selected": true }, { "eligible": true, "location": "Oak Lawn, TX", "type": "DELIVERY", "selected": false } ], "skuId": "130007750-1", "stockLevel": [ { "purchaseLimit": 6, "stock": 211 } ], "storeDistance": 0.0001798902, "storeId": "531", "storeName": "Oak Lawn", "itemTasteProfile": "Blackberry, Chocolate", "itemStyle": "Elegant", "itemBody": "Full-bodied", "transactional": true, "type": "PRODUCT", "volume": "", "stockMessages": { "messages": [ { "shoppingMethod": "INSTORE_PICKUP", "stockMessage": "In stock", "addToCartMessage": "Add to cart", "addToCartStatus": true }, { "shoppingMethod": "DELIVERY", "stockMessage": "Available", "addToCartMessage": "Add to cart", "addToCartStatus": true }, { "shoppingMethod": "SHIPPING", "stockMessage": "Unavailable", "addToCartMessage": "View Product", "addToCartStatus": false } ], "digitalTransactional": true, "digitalInventoryQuantity": 211, "digitalSpecialOrder": false, "digitalLongTermOOS": false, "digitalLimitedStock": false, "digitalInStock": true, "digitalStoreQuantity": 213, "shippingTransactional": false, "shippingInventoryQuantity": 211, "shippingSpecialOrder": false, "shippingLongTermOOS": false, "shippingLimitedStock": false, "shippingInStock": true, "shippingStoreQuantity": 213, "digitalDeliveryEligible": true } }
You should have a Real Data API account to execute the program examples. Replace
< YOUR_API_TOKEN>
in the program using the token of your scraper. Read about the live APIs with Real Data API docs for more explanation.
import { RealdataAPIClient } from 'RealdataAPI-Client';
// Initialize the RealdataAPIClient with API token
const client = new RealdataAPIClient({
token: '<YOUR_API_TOKEN>',
});
// Prepare actor input
const input = {
"maxItems": 20,
"endPage": 1,
"startUrls": [
"https://www.totalwine.com/wine/red-wine/c/000009"
],
"extendOutputFunction": ($) => { return {} },
"customMapFunction": (object) => { return {...object} },
"proxy": {
"useRealdataAPIProxy": true,
"RealdataAPIProxyCountry": "US"
}
};
(async () => {
// Run the actor and wait for it to finish
const run = await client.actor("oyildirim/totalwine-actor").call(input);
// Fetch and print actor results from the run's dataset (if any)
console.log('Results from dataset');
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
})();
from RealdataAPI_client import RealdataAPIClient
# Initialize the RealdataAPIClient with your API token
client = RealdataAPIClient("<YOUR_API_TOKEN>")
# Prepare the actor input
run_input = {
"maxItems": 20,
"endPage": 1,
"startUrls": ["https://www.totalwine.com/wine/red-wine/c/000009"],
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useRealdataAPIProxy": True,
"RealdataAPIProxyCountry": "US",
},
}
# Run the actor and wait for it to finish
run = client.actor("oyildirim/totalwine-actor").call(run_input=run_input)
# Fetch and print actor results from the run's dataset (if there are any)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# Set API token
API_TOKEN=<YOUR_API_TOKEN>
# Prepare actor input
cat > input.json <<'EOF'
{
"maxItems": 20,
"endPage": 1,
"startUrls": [
"https://www.totalwine.com/wine/red-wine/c/000009"
],
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useRealdataAPIProxy": true,
"RealdataAPIProxyCountry": "US"
}
}
EOF
# Run the actor
curl "https://api.RealdataAPI.com/v2/acts/oyildirim~totalwine-actor/runs?token=$API_TOKEN" /
-X POST /
-d @input.json /
-H 'Content-Type: application/json'
search
Optional String
It is the field to use search queries you want to search on the Total Wine website.
shippingInfo
Optional String
You can set the value of the cookie and get the product listing for the selected location.
maxItems
Optional Integer
Set the maximum listing item counts in the result with all as the default.
endPage
Optional Integer
It is to list out the number of the last page. There is no last page by default. You can apply it to every start URL and search request separately.
startUrls
Optional Array
Enter the URLs to start the scraping process. You have to enter the list of URLs.
extendOutputFunction
Optional String
It takes an argument based on the JQuery handle ($) and reflects the output to merge with the default result.
customMapFunction
Optional String
It takes the argument based on objects and gives the output data such that the function maps itself.
proxy
Optional Object
Choose the proxy server to help the scraper. Use US-based locations for the proxy server.
{
"maxItems": 20,
"endPage": 1,
"startUrls": [
"https://www.totalwine.com/wine/red-wine/c/000009",
"https://www.totalwine.com/spirits/gin/c/000870"
],
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useRealdataAPIProxy": true,
"RealdataAPIProxyCountry": "US"
}
}
Disclaimer : Real Data API only extracts publicly available data while maintaining a strict policy against collecting any personal or identity-related information.
+1 424 2264664