Google Maps stands out as one of the most reliable and widely used platforms. Given that people rely on Google Maps for day-to-day navigation and location-related information, it naturally serves as a veritable treasure trove of geolocation data. Beyond its role as a simple navigational compass, this data available on Google Maps can unlock a realm of possibilities and applications in a diverse array of industries.
In this article, we will explore three specific practical use cases of Google Maps data that can provide substantial benefits to businesses and various industries.
However, it’s worth bearing in mind that gathering information from Google Maps takes time and effort. It requires not only excellent skills in automation with headless browsers such as Selenium and Playwright, but also extensive knowledge of web scraping libraries like Beautiful Soup and Scrapy, and even this may not be a foolproof solution as you’re liable to run into blocks.
For this article, we’ll be using Nimble’s Maps API that not only streams data seamlessly with a high degree of accuracy but also allows you to discover businesses in any geographic area, collect in-depth data about each one, and gather ratings. This makes it particularly well-suited for tackling the three practical use cases we’ll be looking at.
Nimble API | Collect Data From Any Website
Let’s dive in.
Use Cases of Data from Google Maps for Businesses and Various Industries
1. Finding New B2B Customers
With its comprehensive information of local business listings, including contact information, customer ratings, and website links, scraping Google Maps may help find new potential customers if you’re a B2B company. It is also a way to find complementary businesses and establish partnerships that could naturally lead to new customers.
For example, if you run a software development company that specializes in creating custom e-commerce solutions for online retailers, you can use Google Maps data to identify and connect with potential B2B customers, such as e-commerce businesses in need of your services.
Here are two specific ways that data from Google Maps can help with this:
- Identify Potential Leads: Web scraping from Google Maps allows B2B companies to identify potential leads with precision. Whether you’re in the business of providing software solutions, industrial equipment, or professional services, Google Maps provides a comprehensive directory of businesses that may benefit from your offerings. In the example we used above, you can scrape Google Maps to gather a comprehensive list of e-commerce stores in your area or within your target region. For instance, your web scraping efforts might reveal a range of e-commerce businesses, from small boutiques to larger online retailers. With this data, you can compile a list of potential leads that match your ideal customer profile.
- Explore Complementary Businesses: Beyond direct leads, Google Maps enables businesses to discover complementary enterprises that can form the basis for mutually beneficial partnerships. For instance, a company offering graphic design services might find local printing shops, marketing agencies, or web development firms as ideal partners, with the potential for referrals and collaborative projects. Such partnerships, naturally cultivated through Google Maps data, can significantly expand your customer base.
2. Location-Based Market Research
Data-driven marketing strategies can be enhanced with location-based data sourced from Google Maps. Here are some examples of how this scraped data can be used to gather insights for targeted marketing:
- Business Location: Google Maps offers extensive access to geolocation data, such as satellite imagery, street views and more. This data can be used, for instance, to analyze how much the surroundings affect a company’s success.
- Demography and people: When searching for a place on Google Maps, the very first information we have from that search is its demographics. Knowing how many people live in a particular location, their ages, cultures and religions, is also a great starting point for a company to make decisions.
- Customer proximity: Google Maps helps businesses understand how close the potential customers are to their establishments, and how the proximity to public transportation, parking areas and more, can influence the flow of customers.
3. Competitor Analysis
Competitor analysis using data from Google Maps can be achieved through different means. Location-based data, business information and reviews/ratings are some of the key sources.
- Competitor’s Location: With Google Maps you can identify where your competitors are located, and based on their traffic you can carefully choose where to establish your business.
- Business Information: Google Maps provides information about the competitors. Business hours, contact, address, description, images and videos. This information can be valuable for understanding your competitors’ operations. Regarding the images and videos, it helps to evaluate how much they’ve invested in branding and marketing.
- Customer Reviews/Ratings: Analyzing competitor’s reviews/ratings is a very powerful way to understand the strength of their business. Using sentiment analysis and other Natural Language Processing techniques on the reviews could bring important results that can be used to make your business overcome the competitor’s rank.
Challenges in Scraping Google Maps
We looked at some practical use cases of scraping Google Maps. However, the process of scraping Google Maps itself comes with several challenges. The available solutions tend to be either technically complex, expensive, or not 100% reliable. Here are some of the constraints associated with trying to scrape Google Maps:
- Official Google Maps API Cost: The official Google Maps API can be a powerful tool for accessing geospatial data, but it comes at a significant cost, especially for businesses with extensive scraping needs. Pricing is often based on the number of requests made to the API, and high-volume scraping can quickly accumulate substantial expenses. This financial burden can be a major drawback for businesses on a tight budget.
- In-house Infrastructure: Handling and storing large volumes of scraped data is a resource-intensive task, requiring you to invest in additional infrastructure to store, process, and manage the data efficiently. Establishing and maintaining this in-house infrastructure can be costly and time-consuming.
- IP Blocks and Rate Limits: Google often implements IP blocks and rate limits to prevent excessive requests and protect its platform from abuse. When scraping Google Maps, frequent IP blocks and rate limiting can disrupt data collection and require strategies for rotating IP addresses and managing request frequency.
- Captchas and Anti-Scraping Measures: Google may implement Captcha challenges or make use of dynamic loading of content to deter scrapers. Captchas require human verification and can be a significant obstacle for automated scraping scripts.
- Data Structure Changes: Google frequently updates its website structure, which can disrupt existing scraping scripts. These changes can alter the HTML layout and class names of elements, making it necessary to constantly maintain and adapt scraping scripts to accommodate these alterations, which can be time consuming.
- Geographical Restrictions: Google may restrict access to certain data based on geographical regions or countries. This can limit the scope of data collection, particularly if your scraping efforts are focused on specific areas. Adhering to geographical restrictions may require employing proxy servers or VPNs to access data from regions where access is limited.
Typically, this is the point at which you leverage headless browsers like Puppeteer or Playwright. However, as mentioned earlier, this is not a 100% foolproof solution as:
- You’re still likely to run into Captchas and other sophisticated anti-scraping measures like browser fingerprinting that can bring the scraping process to a screeching halt.
- You’ll have to take care of proxy management such as rotating proxies in order to get around Captchas and this will require reliance on third-party libraries.
- Developing and maintaining such complex browser-based scrapers is a difficult, resource-intensive task because of so many overlapping concerns to take care of.
This is precisely where Nimble’s API comes in as it’s able to comprehensively solve these challenges while making the data collection process seamless.
How to Make the Most of Google Maps Scraping with Nimble’s Maps API
Nimble’s Maps API offers a seamless and comprehensive solution for scraping and collecting real-time public data from Google Maps. With the Maps API, you can explore businesses in any geographic area, obtaining in-depth public data about each one, through a single, user-friendly API.
Unlike the official Google Maps API, Nimble’s Maps API comes with flexible pricing plans based on your use case scenario, as well as a free trial. But that’s not all. The Maps API also comes with a whole range of features that help ensure seamless and real-time collection of public data from Google Maps at scale:
- The Maps API lets you conduct precise searches and discover businesses with ease by simply using keywords, coordinates, zip codes and more. The information gathered is comprehensive, including business name and title, address and coordinates, contact information, hours of operation, ratings and reviews, images, peak business hours and more.
- Built on Nimble Browser, it leverages cutting-edge residential proxies from Nimble IP, which makes use of AI-driven fingerprinting technology to overcome anti-scraping measures such as rate limits, Captchas and IP blocks. Nimble offers the first premium residential proxy platform utilizing AI models which ensures a zero-downtime proxy solution for continuous data collection.
- With the Maps API, you can automate business discovery across up to 1,000 zip codes, collect reviews on hundreds of businesses in minutes, and obtain detailed data on multiple business types simultaneously. This is known as batch processing and lets you gather data in bulk effortlessly at scale.
- The Maps API is a managed service which means that the code remains updated on Nimble’s end, with zero effort on your part. This ensures that you do not have to update and change your code every time Google Maps updates its data structure.
- The integration process is painless as all you have to do is set your data source, parsing settings, and delivery method, send the request to Nimble APIs, and watch as structured web data seamlessly flows into your storage.
- In addition, Nimble offers comprehensive documentation, facilitates integration with multiple programming languages like Python and JavaScript, and includes a user-friendly Playground front-end feature for effortless information scraping using basic inputs.
👉 Explore and learn more about Nimble’s API and its functionalities
Now that we’ve looked at how the Maps API can help in extracting valuable insights from Google Maps data while bypassing the challenges we mentioned, let’s actually use it and see it in action.
However, to make use of Nimble’s Maps API, you must first create a Nimble account. You can sign up for an account here by selecting the plan of your choice, clicking on ‘Start Free Trial’ underneath it and then filling out your details. Once you’ve done that, you need to login with your account details.
Once done, let’s dive into the actual code.
Scraping Google Maps for Business Data: The Code
Below, we’ll dive into the Nimble’s Maps API while integrating it with Python. The steps will include:
- Search for a business in an industry of interest within a given geographical area/range. As an example, we’ll search for barbershops near Barcelona.
- Use the Python requests library to make an HTML request to Nimble’s API, and we’ll extract the information in .json format.
- Look for general information about the barbershops, their phone numbers, addresses and websites/emails.
- Scrape their ratings and reviews.
- Scrape which amenities and accessibility features they offer (wheelchair, parking, etc).
Let’s start by reproducing the first step on a Python script:
import requests
import base64
username = '<your_nimble_username'
password = '<your_nimble_password'
token = base64.b64encode(bytes(f"{username}:{password}", 'utf-8'))
token = token.decode('utf-8') # convert bytes to string
maps_general_url = 'https://www.google.com/maps/search/'
search_input = 'barber shop near barcelona'
search_input = search_input.replace(' ', '+').lower()
In the script above, we start by importing the necessary libraries. The requests library will serve to make an HTML request to the Nimble’s API and the base64 provides functions for encoding binary data to printable ASCII characters. The latter is important because Nimble APIs use Basic access authentication with a credential string (token) generated from the username and password.
base64(username:password)
In Python, the encoding is achieved with the function base64.b64encode() and the strings username and password are converted to bytes with the encoding utf-8. This output is then decoded to a string which will be the token used with the Basic access authentication.
Then variables maps_general_url and search_input are used to integrate with the API in a more readable way. The .replace() function mainly converts the spaces to +, because that’s how it is read by the URL. Let’s see how this information is grabbed by the API.
url = 'https://api.webit.live/api/v1/realtime/web'
headers = {
'Authorization': f"Basic {token}",
'Content-Type': 'application/json'
}
data = {
"parse": True,
"url": f"{maps_general_url}{search_input}",
"format": "json",
"render": False,
"country": "GB",
"locale": "en"
}
response = requests.post(url, headers=headers, json=data)
At the beginning of the script, we see the Maps API URL. The headers dictionary contains the Content-Type and the Authorization, which takes the token previously generated with base64.
The data dictionary has several keys to define the scraping process. We set parse to True at first, then the input url takes the maps_general_url and search_input variables. When it comes to the format we can select different types, such as list, table, object and json. In this case, we use json to facilitate the data processing step.
As for the country key, as mentioned before, Nimble uses residential proxies which allows the user to scrape from different geolocations. The locale is the language at the output.
Finally, we use the requests instance, to make a POST HTML request with the provided API URL and the dictionaries.
We’re now ready to run the script, we just need to save the .json file somewhere in our directory. For that, we can use the following:
import json
with open('data.json', 'w', encoding='utf-8') as f:
json.dump(response.json(), f, ensure_ascii=False, indent=4)
Once the script is launched, we’ll get a data.json filled with all the search results captured by the API. To easily navigate through the file, I recommend using a JSON extension. Let’s for instance get the information of the first search result:
import json
with open('data.json') as f:
data = json.load(f)
search_results = data["parsing"]["entities"]["SearchResult"]
print(search_results[0])
The output of the first result is the following:
{
'accessibility': [
{
'description': 'Has wheelchair-accessible entrance',
'display_name': 'Wheelchair-accessible entrance',
'full_path_name': '/geo/type/establishment_poi/has_wheelchair_accessible_entrance',
'is_available': True
},
{
'description': 'Has wheelchair-accessible toilet',
'display_name': 'Wheelchair-accessible toilet',
'full_path_name': '/geo/type/establishment_poi/has_wheelchair_accessible_restroom',
'is_available': True
},
{
'description': 'No wheelchair-accessible car park',
'display_name': 'Wheelchair-accessible car park',
'full_path_name': '/geo/type/establishment_poi/has_wheelchair_accessible_parking',
'is_available': False
}
],
'address': 'C/ de Ferlandina, 49, 08001 Barcelona, Spain',
'amenities': [{'description': 'Has toilets', 'display_name': 'Toilets', 'full_path_name': '/geo/type/establishment_poi/has_restroom', 'is_available': True}],
'business_category': ['Barber shop', 'Hairdresser'],
'business_status': 'Open ⋅ Closes 8\u202fpm',
'cid': '7114337547564718537',
'city': 'Barcelona',
'country': 'ES',
'csrf_token': 'cZNDZf-iCsOIptQPn9y-iAc',
'data_id': '0x12a4a2f544181a63:0x62bb344e5ed199c9',
'entity_type': 'SearchResult',
'latitude': '41.3817938',
'longitude': '2.1655197',
'number_of_reviews': '1026',
'page_index': 0,
'payments': [
{'description': 'Accepts debit cards', 'display_name': 'Debit cards', 'full_path_name': '/geo/type/establishment_poi/pay_debit_card', 'is_available': True},
{
'description': 'Accepts NFC mobile payments',
'display_name': 'NFC mobile payments',
'full_path_name': '/geo/type/establishment_poi/pay_mobile_nfc',
'is_available': True
}
],
'phone_number': '+34 722 69 89 92',
'place_id': 'ChIJYxoYRPWipBIRyZnRXk40u2I',
'place_information': {
'opening_hours': [
'Thursday: 11\u202fam-8\u202fpm',
'Friday: 11\u202fam-8\u202fpm',
'Saturday: 11\u202fam-8\u202fpm',
'Sunday: Closed',
'Monday: 11\u202fam-8\u202fpm',
'Tuesday: 11\u202fam-8\u202fpm',
'Wednesday: 11\u202fam-8\u202fpm'
],
'photos': [
{
'image_url': 'https://lh5.googleusercontent.com/p/AF1QipMvqNljABvkEXhAqTc-zx7boW42bFyYr_mNVTPf=w114-h86-k-no',
'latitude': '41.3818848',
'longitude': '2.1654470999999997',
'max_height': 3036,
'max_width': 4048,
'position': 0,
'source_type': 'photos:gmm_android'
},
{
'image_url': 'https://lh5.googleusercontent.com/p/AF1QipMvqNljABvkEXhAqTc-zx7boW42bFyYr_mNVTPf=w408-h306-k-no',
'latitude': '41.3818848',
'longitude': '2.1654470999999997',
'max_height': 3036,
'max_width': 4048,
'position': 1,
'source_type': 'photos:gmm_android'
}
],
'reviews_link': 'https://search.google.com/local/reviews?placeid=ChIJYxoYRPWipBIRyZnRXk40u2I&q=barber+shop+near+barcelona&authuser=0&hl=en&gl=GB',
'website_url': 'http://www.facebook.com/Jackbarberbcn'
},
'place_url': 'https://www.google.com/maps/search/?api=1&query=41.3817938%2C2.1655197&query_place_id=ChIJYxoYRPWipBIRyZnRXk40u2I',
'position': 1,
'rating': '4.7 stars',
'review_summary': {'overall_rating': 4.7, 'ratings_count': {'1': 28, '2': 17, '3': 16, '4': 87, '5': 878}, 'review_count': 1026},
'sponsored': False,
'street_address': 'C/ de Ferlandina, 49',
'title': 'Jack Barber Barcelona',
'zip_code': '08001'
}
If we want to grab general information, we can try and do the following:
def get_general_information(search_result):
return {
'business_name': search_result['title'],
'address': search_result['address'],
'zip_code': search_result['zip_code'],
'category': search_result['business_category'][0],
'status': search_result['business_status'],
'phone_number': search_result['phone_number'],
'website': search_result['place_information']['website_url']
}
result = get_general_information(search_results[0])
print(result)
By filtering with the function above, we get the general information structured this way:
{
'business_name': 'Jack Barber Barcelona',
'address': 'C/ de Ferlandina, 49, 08001 Barcelona, Spain',
'zip_code': '08001',
'category': 'Barber shop',
'status': 'Open ⋅ Closes 8\u202fpm',
'phone_number': '+34 722 69 89 92',
'website': 'http://www.facebook.com/Jackbarberbcn'
}
As for the reviews and ratings, we can grab general information with this function:
def get_reviews_information(search_result):
return {
'number_of_reviews': search_result['number_of_reviews'],
'rating': search_result['rating'],
'reviews_link': search_result['place_information']['reviews_link'],
'overall_rating': search_result['review_summary']['overall_rating'],
'review_count': search_result['review_summary']['review_count'],
'rating_1_count': search_result['review_summary']['ratings_count']['1'],
'rating_2_count': search_result['review_summary']['ratings_count']['2'],
'rating_3_count': search_result['review_summary']['ratings_count']['3'],
'rating_4_count': search_result['review_summary']['ratings_count']['4'],
'rating_5_count': search_result['review_summary']['ratings_count']['5'],
}
result = get_reviews_information(search_results[0])
print(result)
And we get the following ratings:
{
'number_of_reviews': '1026',
'rating': '4.7 stars',
'reviews_link': 'https://search.google.com/local/reviews?placeid=ChIJYxoYRPWipBIRyZnRXk40u2I&q=barber+shop+near+barcelona&authuser=0&hl=en&gl=GB',
'overall_rating': 4.7,
'review_count': 1026,
'rating_1_count': 28,
'rating_2_count': 17,
'rating_3_count': 16,
'rating_4_count': 87,
'rating_5_count': 878
}
In the case we want the actual reviews and not only the ratings, we need to make another request with the csrf_token and the data_id obtained in the first output. The structure of the request URL is different from the previous one and should be implemented this way:
main_url = 'https://api.webit.live/api/v1/realtime/serp?'
csrf_token = search_result['csrf_token']
data_id = search_result['data_id']
search_engine = 'search_engine=google_maps_reviews'
url_reviews = \
f"{main_url}{search_engine}&csrf_token={csrf_token}&place_id={data_id}"
As for the request instance, it is implemented just as before without the data dictionary:
headers = {
'Authorization': f"Basic {token}",
}
response = requests.get(url_reviews, headers=headers)
with open('reviews.json', 'w', encoding='utf-8') as f:
json.dump(response.json(), f, ensure_ascii=False, indent=4)
The reviews.json contains several reviews. Let’s grab the description of the first one in the file:
import json
with open('reviews.json') as f:
data = json.load(f)
reviews_results = data["parsing"]["entities"]["Review"]
print(reviews_results[0]["description"])
We obtain the following text:
Got lucky and managed to get a haircut here the same day. Simply called in and was booked within 2 hours.
The barbershop has a cool vibe and makes you feel part of some exclusive group. You almost know you'll get a good cut just from the aura. My barber, who I can't remember his name, was really
easygoing and asked me about every detail of the cut. He's got more tattoos on his body then tourists in Barcelona, I'll say that.
I wanted a classic cut and I think he nailed it. He even washed my hair, used a variety of products, and took his time and didn't rush.
Cut and beard was €21 and I'd say that's fair for
Concerning accessibility, we can simply grab the first key of the JSON dictionary accessibility. It should return a list like this:
[
{
description: "Has wheelchair-accessible entrance",
display_name: "Wheelchair-accessible entrance",
full_path_name:
"/geo/type/establishment_poi/has_wheelchair_accessible_entrance",
is_available: True,
},
{
description: "Has wheelchair-accessible toilet",
display_name: "Wheelchair-accessible toilet",
full_path_name:
"/geo/type/establishment_poi/has_wheelchair_accessible_restroom",
is_available: True,
},
{
description: "No wheelchair-accessible car park",
display_name: "Wheelchair-accessible car park",
full_path_name:
"/geo/type/establishment_poi/has_wheelchair_accessible_parking",
is_available: False,
},
];
Conclusion
This article explored three practical use cases for web scraping Google Maps data, focusing on finding B2B customers, location-based market research, and competitor analysis. While acknowledging the challenges of scraping Google Maps, we looked at why Nimble’s Maps API offers a highly scalable solution to seamlessly gather data from public listings in real-time, providing us with all the key pieces of information we need, from reviews and ratings to contact information to peak business hours and more.
Concerning our three use cases, we can see that:
- When it comes to finding new B2B customers, Nimble’s API can provide us with comprehensive information on local business listings.
- Concerning location-based market research, the general output provides keys for longitude and latitude, the address, the city, the country, the zip code and some information about the crowd. Extra location features can be accessed by tweaking the API request, according to the documentation as mentioned here.
- Finally, for the competitor analysis location-based and general information is useful, along with the reviews .json generated with the
csrf_tokenand thedata_id. In this case, the file contained the reviews for the first page, but the API can extract many more and provide a considerable amount of reviews to work with and integrate with Natural Language Processing techniques and Large Language Models, which consequently will help to evaluate the competitors’ strengths and weaknesses and act to perform better.
The gathered data is meticulously organized, and the resulting .json files are simple to navigate, leading to faster data manipulation and seamless integration. Furthermore, the separation of reviews and other information is a clever approach that enhances the readability of the outputs.
The Maps API also comes with a free trial so you can give it a spin at no extra cost and see if it suits your needs.
As a developer constantly involved in refining web scraping techniques, I’m impressed with the countless hours I could save by simply using Nimble’s API. The clarity of the data, the documentation and the ease of implementation makes it a fantastic tool for scraping and gathering data from Google Maps and to extract data-driven insights we need for our three use cases.