How To Find Elements by Xpath in Selenium?

avoid IP bans while scraping amazon

In Selenium, you can find elements by XPath using the findElement or findElements methods.


  • find_element- This method is used to find the first matching element based on the provided XPath expression.

  • find_elements- This method is used to find all matching elements based on the provided XPath expression. It returns a list of elements.


Let's understand with an Example

In this code we are navigating to the home page of Scrapingdog and then extracting the h1 tag using findElement and all the h2 tags using findElements.



          from selenium import webdriver
          from selenium.webdriver.common.keys import Keys


          PATH = 'C:\Program Files (x86)\chromedriver.exe'

          # Set up the Selenium WebDriver
          options = webdriver.ChromeOptions()
          options.add_argument("--headless")
          driver = webdriver.Chrome(executable_path=PATH, options=options)

          # Navigate to the home page
          driver.get("https://www.scrapingdog.com")



          #will search for the first occurance of h1 tag
          h1_tag = driver.find_element_by_xpath("//h1")

          #will search for all the h2 tags
          h2_tag = driver.find_elements_by_xpath("//h2")

          print(h1_tag.text)
          # Extract data from the protected page
          print(h2_tag)




          # Close the browser
          driver.quit()

        

You can also refer to our guide on web scraping with Xpath and Python to learn more about Xpath.


Additional Resources:

How to select elements by text in Xpath?

How to select elements by class in XPath?

How to select Xpath selectors in Python?

Scrapingdog logo

Try Scrapingdog for Free!

Free 1000 API calls of testing.

No credit card required!