Unleash your potential – connect, collaborate, and automate with our powerful solutions.
I am trying to extract data from a travel website where I need to get a list of all hotels with URLs using the activity on the page. The problem is that this website does not have a next button link; instead, hotels are loaded as you keep scrolling. Is this possible with the “Extract Data” activity?
Sample website: www.booking.com
Hello, @abidurai No, you cannot achieve the above goal with the “Extract Data” activity. This activity will help you get the list of hotels with URLs that are currently loaded and visible on the page, but the results that load when scrolling down will not be included in the output.
Hello, @abidurai No, you cannot achieve the above goal with the “Extract Data” activity. This activity will help you get the list of hotels with URLs that are currently loaded and visible on the page, but the results that load when scrolling down will not be included in the output.
OK, but there is a button at the bottom of the webpage labeled “Load Results.” When I spy that as the next button, the activity throws an error: “Doesn’t have common element for Native and Surrounding element.” Please see the reference image below.
Hello, @abidurai No, you cannot achieve the above goal with the “Extract Data” activity. This activity will help you get the list of hotels with URLs that are currently loaded and visible on the page, but the results that load when scrolling down will not be included in the output.
OK, but there is a button at the bottom of the webpage labeled “Load Results.” When I spy that as the next button, the activity throws an error: “Doesn’t have common element for Native and Surrounding element.” Please see the reference image below.
The image above shows that there is no "Common" or "Static" element for the surrounding element required for the load results button.
Try to spy over the areas near the load results button and choose the surrounding element that has a common and static XPath on the page.
This should work, and all the hotels will be returned as output.
Hey, thanks for the solution. After spying over the load results button, I was able to execute the activity. However, the execution is not stopping and is still running.
Yes, the bot will execute the workflow until all the results from all pages on the website are extracted. This might take longer than expected based on the input provided. You can reduce the limit of extracted data in the properties. There is a property called "Number of items" in the extract data property where you can set the minimum limit of extractions. Please refer to the image below.
