Businesses often face two major challenges: 1) The slow, manual process of searching the web for new leads or market data, and 2) The difficulty of analyzing thousands of unstructured customer reviews to find actionable insights. I used web scraping to tackle both of these problems directly.

How I Used It

I applied two different tools to solve two unique challenges:

1. Lead Generation with Apify: At Nimbus Nordic, I was tasked with finding new potential customers for our Polish market. I wrote a script on the Apify platform that automatically searched Polish (.pl) websites for our specific business niche keywords. This identified new leads and also helped validate our existing markets. The automation saved a significant amount of manual research time.

2. Customer Insight with Octoparse & AI: For a jewelry business, I used Octoparse to scrape 3,000 of their Trustpilot reviews. Manually reading these would be impossible. Instead, I fed the scraped text into an AI model (LLM), which read, categorized, and counted each review. This process quickly showed us exactly which product categories received the most positive and negative feedback.

What This Involved

These projects required a combination of skills:

• Automated Data Extraction: Using both Apify and Octoparse to collect specific data from the web at scale.

• Custom Scripting: Writing targeted code on Apify to perform automated market research.

• AI-Powered Analysis: Combining scraped data with an LLM to perform large-scale sentiment analysis and text categorization.

• Data Structuring: Turning unstructured data (like customer reviews) into clear, organized insights.

The Result