< Back to Blog Overview

A Definitive Guide to Scrape Yelp Reviews for Businesses in 2022

27-08-2020

In this post we are going to scrape Yelp reviews. Yelp is a great source for public reviews for businesses.

We will code a scraper for that. Using this scraper, you will be able to scrape reviews of any company from yelp. To make things simple, we will use web scraping API.

scrape yelp reviews using python
Scaping Yelp Reviews With Python

Why Scrape Yelp?

Yelp is an American company that publishes reviews about businesses. The reviews they collect are crowd-sourced. It is the largest directory on the Internet available. 

Scraping yelp reviews will provide you with a large number of data trends and information. Using that data you can either improve your product or you can show it to your other free clients to convert them to your paid client. 

Since yelp is a business directory it has many businesses that can be in your target market. Yelp scraping allows you to extract valuable information like business names, contact information, location, and industry to help you create qualified leads a lot faster with a web scraper.

Read More: Scrape google search results using Python

Why Use Scrapingdog API to Scrape Yelp Reviews?

Scrapingdog can help you scrape yelp at scale without getting blocked. You just have to pass the target url and Scrapingdog will create an unbroken data pipeline for you, that too without any blockage.

Scrapingdog is fast and handles all the hassle of handling proxies and passing custom headers.

Requirements For Scraping Yelp

Generally, web scraping is divided into two parts:

  1. Fetching data by making an HTTP request
  2. Extracting important data by parsing the HTML DOM

Libraries & Tools

  1. Beautiful Soup is a Python library for pulling data out of HTML and XML files.
  2. Requests allow you to send HTTP requests very easily.
  3. Web scraping API extracts the HTML code of the target URL.

Read here: Learn web scraping with Python 101

Setup

Our setup is pretty simple. Just create a folder and install BeautifulSoup & requests. For creating a folder and installing libraries, type the below-given commands. I assume that you have already installed Python 3. x (The latest version is 3.9 as of April 2022).

mkdir scraper<br>pip install beautifulsoup4<br>pip install requests

Now, create a file inside that folder by any name you like. I am using scraping.py.

Firstly, you have to sign up for the scrapingdog API. It will provide you with 1000 FREE credits. Then just import Beautiful Soup & requests in your file. Like this.

from bs4 import BeautifulSoup<br>import requests

Why Use a Yelp Review API?

This tool will help us to scrape dynamic websites using millions of rotating proxies so that we don’t get blocked. It also provides a captcha clearing facility. It uses headerless chrome to scrape dynamic websites.

How to Use Yelp API? The Yelp API is used to access Yelp data in order to display Yelp business information on websites and applications.

What Are the Use Case Scenarios for Yelp Developer API?

There are a number of use case scenarios for Yelp Developer API, including retrieving business information, reviews, and ratings, searching for businesses by location, name, or category, and creating and managing user lists.

  • A business owner may use the Yelp Review API to programmatically monitor and respond to reviews of their business on Yelp.
  • A developer may use the Yelp Review API to create a tool for businesses to monitor and respond to reviews.
  • A Yelp user may use the Yelp Review API to get real-time notifications when new reviews are posted for businesses they are interested in.

Scraping Yelp Reviews for a Random Restaurant

We are going to scrape public reviews for this restaurant. We will create a yelp review scraper for that.

  1. Name of the person
  2. Location of the person
  3. Stars
  4. Review

Let’s Start Scraping Yelp

Now, since we have all the ingredients to prepare the scraper, we should make a GET request to the target URL to get the raw HTML data. If you are not familiar with the scraping tool, I urge you to review its documentation.

We will scrape Yelp for review data using the requests library below.

r = requests.get('<a href="https://api.scrapingdog.com/scrape?api_key=5ea541dcacf6581b0b4b4042&amp;url=https://www.yelp.com/biz/sushi-yasaka-new-york%27).text" rel="noreferrer noopener" target="_blank">https://api.scrapingdog.com/scrape?api_key=5ea541dcacf1b0b4b4042&amp;url=https://www.yelp.com/biz/sushi-yasaka-new-york').text</a>

This will provide you with an HTML code of that target URL.

Now, you have to use BeautifulSoup to parse HTML.

soup = BeautifulSoup(r,’html.parser’)

Now, all the reviews are in the form of a list. We have to find all those lists.

1 P9QEX001CFXam1N7Jk3DpA
allrev = soup.find_all(“li”,{“class”:”lemon — li__373c0__1r9wz margin-b3__373c0__q1DuY padding-b3__373c0__342DA border — bottom__373c0__3qNtD border-color — default__373c0__3-ifU”})

We will run for a loop to reach every reviewer. To extract name, place, stars, and reviews, we must first find the tags where this data is stored. For example “Name” is stored in “lemon — a__373c0__IEZFH link__373c0__1G70M link-color — inherit__373c0__3dzpk link-size — inherit__373c0__1VFlE”. Like this, using chrome developer tools, you can find the rest of the tags.

for i in range(0,len(allrev)): 

try:
                        l["name"]=allrev[i].find("a",{"class":"lemon--a__373c0__IEZFH link__373c0__1G70M link-color--inherit__373c0__3dzpk link-size--inherit__373c0__1VFlE"}).text
 except:
                        l["name"]=None

 try:
                        l["place"]=allrev[i].find("span",{"class":"lemon--span__373c0__3997G text__373c0__2Kxyz text-color--normal__373c0__3xep9 text-align--left__373c0__2XGa- text-weight--bold__373c0__1elNz text-size--small__373c0__3NVWO"}).text
 except:
                        l["place"]=None 

try:
                        l["stars"]=allrev[i].find("div",{"class":"lemon--div__373c0__1mboc i-stars__373c0__1T6rz i-stars--regular-5__373c0__N5JxY border-color--default__373c0__3-ifU overflow--hidden__373c0__2y4YK"}).get('aria-label')
 except:
                        l["stars"]=None 

try:
                        l["review"]=allrev[i].find("span",{"class":"lemon--span__373c0__3997G raw__373c0__3rKqk"}).text
 except:
                        l["review"]=None

u.append(l)
l={}

print({"data":u})

The output of the above code will be: –

{
 “data”: [
 {
 “review”: “If you’re looking for great sushi on Manhattan’s upper west side, head over to Sushi Yakasa ! Best sushi lunch specials, especially for sashimi. I ordered the Miyabi — it included a fresh oyster ! The oyster was delicious, served raw on the half shell. The sashimi was delicious too. The portion size was very good for the area, which tends to be a pricey neighborhood. The restaurant is located on a busy street (west 72nd) &amp; it was packed when I dropped by around lunchtimeStill, they handled my order with ease &amp; had it ready quickly. Streamlined service &amp; highly professional. It’s a popular sushi place for a reason. Every piece of sashimi was perfect. The salmon avocado roll was delicious too. Very high quality for the price. Highly recommend! Update — I’ve ordered from Sushi Yasaka a few times since the pandemic &amp; it’s just as good as it was before. Fresh, and they always get my order correct. I like their takeout system — you can order over the phone (no app required) &amp; they text you when it’s ready. Home delivery is also available &amp; very reliable. One of my favorite restaurants- I’m so glad they’re still in business !”,
 “name”: “Marie S.”,
 “stars”: “5 star rating”,
 “place”: “New York, NY”
 },
 {
 “review”: “My friends recommended for me to try this place for take out as I was around the area. I ordered the Miyabi, all the sushi and sashimi was very fresh and tasty. They also gave an oyster which was a bonus! The price is great for the quality and amount of fish. I was happily full.”,
 “name”: “Lydia C.”,
 “stars”: “5 star rating”,
 “place”: “Brooklyn, Brooklyn, NY”
 },
 {
 “review”: “Best sushi on UWS and their delivery is quicker than any I’ve seen! I ordered their 3 roll lunch special around 1:40pm and by 2, I was thoroughly enjoying my sushi! Granted, I live only a few blocks away but I was BLOWN away by the quick services. I had, spicy yellowtail, jalapeño yellowtail and tuna avocado roll. Great quality of fish for such a reasonable price. $16 for 3 rolls. This has certainly come by go-to place for amazing, fresh sushi on UWS.”,
 “name”: “Ella D.”,
 “stars”: “5 star rating”,
 “place”: “Manhattan, New York, NY”
 },
 ]
}

There you go! We have the yelp reviews ready to manipulate and maybe store somewhere like in MongoDB. But that is out of the scope of this tutorial.

Remember that if you aren’t using Python but other programming languages like Ruby, Nodejs, or PHP. You can easily find HTML parsing libraries to parse the results from Scrapingdog API.

We hope you enjoyed this tutorial, and we hope to see you soon in Scrapingdog. Happy Scraping!

Conclusion

In this article, we understood how you can scrape data using the data scraping tool & BeautifulSoup regardless of the type of website.

Feel free to comment and ask me anything. You can follow us on Twitter and Medium. Thanks for reading, and please hit the like button!

Additional Resources

And there’s the list! At this point, you should feel comfortable writing your first web scraper to gather data from any website. Here are a few additional resources that you may find helpful during your web scraping journey:

  1. Web Scraping With Python 101
  2. Scrape google search results using python
  3. Guide to web scraping
  4. Web Scraping for Lead Generation

Manthan Koolwal

My name Is Manthan Koolwal and I love to create web scrapers. I have been building them for the last 10 years now. I have created many seamless data pipelines for multiple MNCs now. Right now I am working on Scrapingdog, it's a web scraping API that can scrape any website without blockage at any scale. Feel free to contact me for any web scraping query. Happ Scraping!
Scrapingdog Logo

Try Scrapingdog for Free!

Free 1000 API calls of testing.

No credit card required!

DMCA.com Protection Status