Octoparse is a beginner-friendly web scraping tool that can fully automate data extraction from any website. Unlike most other scraping tools, it requires zero coding knowledge.
The tool can create datasets from unstructured or semi-structured data and export them. It supports data export in CSV, Excel, HTML, and other formats, including databases like SQL Server, MySQL, and Oracle.
Additionally, Octoparse is fully scalable. It offers a cloud service that enables users to run large-scale scraping projects simultaneously by using distributed computing.
How to Set Up IPRoyal Proxies With Octoparse
To get started with Octoparse, create an account on the website to start a free trial or purchase a plan.
Once you do, log into your account. Download (1) Octoparse for your OS (Windows or Mac) and install it.
Launch Octoparse, click the New (2) button, and select Custom task (3) to create a new scraping task.
For the URL Source, we’ll choose Enter manually (4) . We already have a list of IPRoyal’s location page URLs from a previous scraping task, so we’ll paste them in the URL Input (5) field and click Save (6) .
Octoparse will load the page, automatically detect elements you can scrape, and display them in the Data Preview section. In our case, it picked up almost everything - headings, text paragraphs, images, links, etc. However, it did not pick up the data we want to scrape, so we’ll have to do that manually. For now, click the Create workflow (7) button.
Next, we’d like to close the cookie information window and see the page’s content. Click the Allow all (8) button and select Click button (9) from the Tips window to close it.
Now, it’s time to select the data we need from the page. You can modify the auto-detected values in the Data Preview section until you’re satisfied. Since our data wasn’t recognized, we’ll select it manually from the page. For this task, we need the location name (Algeria) and the number of proxies (380,681). Click on Algeria (10) and select Text (11) to create a new extract data entry.
Do the same for the number of proxies. Click on 380,681 (12) and select Text (13) to create another data entry.
Clean up the data fields until you’re left with the ones you need and assign custom names (14) to each value if needed. Once you’re done, click the Proxy (15) button to set up your proxies for the task.
Tick the box next to Access websites via proxies (16) option and select Use my proxies (17) . Finally, click the Configure (18) button.
To add your IPRoyal proxies, make sure you enable IP whitelisting first since Octoparse doesn’t support using usernames and passwords for your proxies. Configure the proxy options for your whitelisted IP address (Country, state, city, protocol, IP rotation) and click the Save (19) button. We’ll use HTTP/HTTPS residential proxies from Washington, USA, which rotate on each request.
Go back to the IPRoyal dashboard and scroll down to the Formatted proxy list. Adjust the quantity field as needed and click Copy (20) to copy your credentials to the clipboard.
Return to Octoparse and paste your credentials (21) .
Click the Confirm button (22) to continue.
Finally, click the Save (23) button.
Our scraping task is ready, so click the Run (24) button to start it.
We’ll select the Standard Mode (25) since it’s a fairly simple web scraping task. You can use other options as necessary. The boost mode will split your tasks with multiple threads to speed things up, so you can safely use it for larger local tasks with IPRoyal because each request still gets a brand new IP address. In other words, your device’s resources are the only limitation.
And our web scraping job is underway. All we need to do is wait for it to finish.
Once it’s done, click the Export (26) button.
Select your preferred format and click the Confirm (27) button.
Check your saved file to confirm all the data is in its place, and you’re done.
