How to Use Data Minor for Web Scraping: A Step-by-Step Guide

Web scraping has become an essential skill for data collection, and Data Minor offers a user-friendly approach to extract information from websites without coding knowledge. This tutorial walks you through the process of using Data Minor to efficiently scrape web data.

Getting Started with Data Minor

To begin using Data Minor, you’ll need to download it and add it as a browser extension. Once installed, you can access it directly from your browser’s extension menu. Before starting the scraping process, ensure you have the target website open in your browser.

Creating a Recipe

Data Minor works with “recipes” – custom configurations that tell the tool what data to extract from a website. You can either create your own recipe or use existing ones. The example used in this guide focuses on scraping the O1 tenderboard website, but the principles apply to any website you wish to scrape.

Selecting the Appropriate Scraping Method

Data Minor offers multiple options based on the structure of the website you’re scraping. For websites with tabular data organized in rows, the first option typically works best.

Selecting Elements to Scrape

The process involves several key steps:

1. Use the element finder tool to select the rows you want to scrape
2. Press Shift+C to select an element, which will appear with a green border
3. Choose “Grade A” to select all similar rows on the page

Adding Columns

To specify which data points to extract:

1. Select “Add new column”
2. Use the “Easy column finder” option
3. Press C to select the specific data element, which will appear with a blue border
4. Repeat this process to add multiple columns as needed

Handling Pagination

For websites with multiple pages:

1. Use the navigation finder to identify pagination elements
2. If the standard options don’t work properly, try the “addones navigation finder” option
3. Select the pagination button that allows moving to the next page

Testing and Saving Your Recipe

Before finalizing your recipe:

1. Test it on a single page to ensure it captures the intended data
2. Remember that Data Minor allows scraping up to 500 pages monthly
3. Verify that the automation for changing pages works correctly
4. Save your recipe for future use

Running the Scraper

When you’re ready to extract data:

1. Data Minor will start by scraping the first page
2. If you’ve enabled the “next page automation” option, it will automatically proceed through subsequent pages
3. The tool will extract only the columns you’ve specified in your recipe

With these steps, you can harness the power of Data Minor to extract structured data from websites without writing a single line of code, making web scraping accessible to everyone regardless of technical background.