Scrapy Web Scraper

Estimated reading: 1 minute

The Scrapy Web Scraper helps to extract information from a specified webpage URL. It reads the page content and returns the data in multiple usable formats such as clean text, raw HTML, or structured data, allowing the information to be used directly in automation workflows.

Purpose:

1. Extract specific data from an individual webpage.

Parameter

Parameter	Description
Target URL	Specifies the URL of the webpage to scrape. The value can be hardcoded or provided as a variable, either as a credential or a generic value.
Timeout (seconds)	Specifies the maximum time to wait for the scraping process. The value must be at least 1 second.

Output

Choose any of the following output forms:

Output Type	Description
Clean Text	Plain, readable text extracted from the page.
Raw HTML	Complete HTML content of the page.
Data	Structured data extracted from the page.
DataFrame	Tabular representation of the extracted data.

Scrapy Web Scraper

Parameter

Output

Scrapy Web Scraper

CONTENTS