1. Installation

Monitoring changes in competitor pricing is essential for conducting accurate market analysis and boosting your brand’s competitive edge. Manually tracking prices can be time-consuming and prone to errors.

Therefore, automated price tracking methods have become powerful tools for optimizing business processes. In a competitive market, we explore four fundamental methods to implement automated competitor price tracking for your brand.

Installation of Scrapy

First, ensure that the latest version of Python is installed on your computer. Then, install the Scrapy library for your project using the following command. You can run this command in the Python terminal or command prompt to complete the installation

pip install scrapy

Creating a Scrapy Project

To start using Scrapy, you first need to create a project. You can initiate a new Scrapy project with the following command:

scrapy startproject fiyat_takibi

This command creates a folder named “fiyat_takibi” and includes the basic Scrapy project files within it.

Creating a Scrapy Spider

Now, we need to create a spider to extract product prices. In Scrapy, a spider is a class written to retrieve data from a specified web page.

Go to the fiyat_takibi/spiders directory within the project folder.

Create a new Python file, for example like fiyat_spider.py

Here is an example of a Scrapy crawler:

import scrapy

class FiyatSpider(scrapy.Spider):

    name = “fiyat_spider”

    # Define the URLs of your competitor’s product pages here;

    start_urls = [

        ‘https://www.ornek-site.com/urunler/urun-1’,

        ‘https://www.ornek-site.com/urunler/urun-2’,

        ‘https://www.ornek-site.com/urunler/urun-3’,

    ]

    def parse(self, response):

        # We are extracting the page title and product price;

        urun_adi = response.xpath(‘//h1[@class=”product-title”]/text()’).get()

        urun_fiyati = response.xpath(‘//span[@class=”price”]/text()’).get()

        # We are returning the data as output.

        yield {

            ‘urun_adi’: urun_adi,

            ‘urun_fiyati’: urun_fiyati.strip() if urun_fiyati else ‘Fiyat bulunamadı’,

        }

Explanation

The code snippet above is written for a sample e-commerce site. You should run it separately for each competitor’s website by assigning unique names.

start_urls: Each competitor’s website links should be stored in separate files. Here, you define the URL lists of the product pages you want to track for competitor price monitoring.

parse: Scrapy runs the parse function for each URL to extract data from the page. Here, we use XPath expressions to retrieve the product name and price information from the page.

xpath: It is a method used to locate elements on a page. XPath provides access to XML and HTML structures.

yield: This tells Scrapy to return the data as output. The extracted data is usually saved in a JSON or CSV file.

Back