Octoparse is a mostly-free web scraping program available to every major OS. All of the things that one has to account for when web scraping – proxies, IP addresses, scraping with precision, etc. – Octoparse simplifies with UI interface and a supposedly easy-to-use dashboard. They also provide a YouTube channel to help a first-time user get started.
Table of Contents
What is Octoparse?
Octoparse is a web scraping and proxy service tool that helps the beginner to an intermediate user with performing their intended tasks without running into problems.
Since it offers premium packages to industries that can afford them, the software is excellent at what it can do. Unlike other software that’s free for a very very limited amount of scraping power, Octoparse offers a generous package to its free users:
unlimited pages per crawl, 10 crawlers at a time, and 10,000 records per export. The number of records is the make-or-break limit to the free plan: depending on the project, 10,000 entries could be either more than enough or nowhere near enough.
Regardless, it’s as effective as the Python packages for web scraping, and perhaps even more so. Their product overview does not exaggerate its capabilities. Just be aware of the limitations.
Read more: Picking the Best Web Scraping Tools – A Complete Comparison!
Octoparse comes as a software program. Here’s a screenshot of the landing page when first logging into their software.
And there already lots of task templates for Amazon, eBay, Taobao, Rakuten, JD, BestBuy and more.
Why need to use proxies for Octoparse when using it for web scraping
Octoparse is nothing more than an interactive GUI and software tool designed to make web scraping easier. However, it does not run proxies by default, because for small-scale scraping tasks, proxies are not necessary. With larger and faster tasks, proxies are needed.
On the other hand: they must be used alongside Octoparse’s workflow in order to take full advantage of Octoparse while also using proxies. Using Octoparse alone does not replace the need for proxies in this case. Proxies are needed whenever proxies are needed, and Octoparse does not change that fact.
What’s the type of Octoparse proxies are needed?
Rotating Proxies for Octoparse
There is no doubt that the best proxies for Octoparse are rotating backconnect proxies When you’re web scraping or crawling online, usually the backconnect proxy provider, offer two types of IP rotation, The one is rotating by each session, another is rotating by time (sticky session), such as smartproxy, They offer Sticky & Random Endpoints, you can easily learn more from our guide.
So, If you want to choose the best rotating proxies for Octoparse, you have to choose the proxy provider that supports, IP rotation by every request, Here is some recommends for you.
- Smartproxy – <Editor Choice> – Rotating residential IPs & datacenter IP proxies
- Storm Proxies – <Budget Choice> – Offer Cheap Rotating Reverse Proxies
- NetNut – High Speed Auto-Rotating Proxy
- Geosurf – <Newbie friendly> – High rotation Gateway – residential IP Proxies
And note, Octoparse only support the IP as the proxy setting, not support “host:port”, So, if your proxy provider is using the “Host:port” as the proxy format, you need to change it to “IP:Port”.
Dedicated Proxies for Octoparse
Octoparse will assist with rotating proxies, however, and knows how to do so as needed when one IP address has exhausted its requests privileges in web scraping a website. Octoparse is designed to detect that and rotate to the next proxy or wait a certain amount of time until that IP address is cleared to scrape again.
Here are Top 3 dedicated proxies providers that you can use it for built-in IP rotation for Octoparse.
- Myprivateproxy – Offer fresh private proxies for web scraping
- Instantproxies – Budget Choice – Cheap private proxies
- Squidproxies – Money-back guarantee
How to Scrape Amazon reviews with Octoparse
This tutorial follows the guidelines found on Octoparse’s website here.
If it becomes necessary to acquire a list of items, Octoparse also provides a guide on how to do that.
Save and select the items from the list you actually want. While on this screen it is the best time to label or rename the fields to your preference. Simply double-click the field name to do this.
If you followed the extremely simple instructions on Octoparse’s homepage, you probably successfully completed this web crawling exercise. The tutorial in the link above is as succinct and clear of a walkthrough as any. There’s no need to elaborate or improve on it. These pictures show that you’re on the right track. Here’s what the final result should look like a table as output:
Which, with the file export options in the Octoparse menu, will save the extracted data into the format of your choosing. Octoparse made Amazon crawling incredibly simple.
If you much like Python, you can learn this easy guide also. Happy crawling!
English 
Chinese