Web scraping

(Updated 8-8-2023)

• Links
• Python
  • xpath

Links

Harvard tutorial

William Marble

Stanford

MIT

Python

xpath

xpath with multiple layers:

driver.find_elements("xpath", "//pre[contains(@class, 'ma-0')]|//span[contains(@class, 'font-weight-bold')]")

xpath with and statement:

hidden_price_elements = driver.find_elements("xpath", "//*[contains(@class, 'click-to-see') and (contains(@class, 'category-slider__item__price category-slider__item__price--inline'))]")
driver.find_elements(By.XPATH, "//button[contains(@aria-label, 'Next page') and not(contains(@class, 'disabled'))]")

xpath with or statement:

driver.find_elements("xpath", "//button[contains(@class, 'class name')]//span[contains(@class, 'subclass name')] | //button[contains(@class, 'class name 2')]//span[contains(@class, 'subclass name')]")