Crawl the Pricing data from the Bigbakstet website using Python


Using the python code, we can easily crawl the pricing data in minutes.
Here we need some data like BigBasket product ID. This ID is available in your data or the webpage URL.

How to find it on URL?
Please check the below link carefully.

You can see the number after pd/ this number means that product ID. So, we got the product ID then we can build our simple crawler using python.We are using Google Colab for it this is very easy to use and you don’t need to install any library.

We need to import the below libraries

import requests
from bs4 import BeautifulSoup as bs

eanCodeLists = [40097083]
You can add multiple ID just add a comma between two ID's

ProductNameList = []
ProductPriceList = []
PriceList = []

for items in eanCodeLists:
urlopen = requests.get(''+str(items)).text
soup =bs(urlopen,'lxml')

PriceName = soup.find("td",{"class":"IyLvo"}).text.strip()

import pandas as pd
table_dict ={'Price':PriceList,'code':eanCodeLists}
df = pd.DataFrame(table_dict)


from google.colab import files'BigBasket.csv')

Previous articleCohort Analysis