I am trying to click the google store once the google webpage loads. I do not want to use time.sleep() for a few seconds for the google page to load in. I want the browser to click "store" once the page loads. Below is my code, what am I doing wrong?
from selenium import webdriver
import requests
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import pause
driver = webdriver.Chrome('/applications/chromedriver')
driver.set_window_size(1024, 600)
driver.maximize_window()
driver.get("https://www.google.com")
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.xpath, "/html/body/div[2]/div[2]/div[1]/a[2]")))
element.click()
The html for xpath is correct too since it works with driver.find_element_by_xpath("/html/body/div[2]/div[2]/div[1]/a[2]").click()
Related
I just need to click the load more button once to reveal a bunch more information so that I can scrape more HTML than what is loaded.
The following "should" go to github.com/topics and find the one and only button element and click it one time.
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
driver = webdriver.Edge()
driver.get("https://github.com/topics")
time.sleep(5)
btn = driver.find_element(By.TAG_NAME, "button")
btn.click()
time.sleep(3)
driver.quit()
I'm told Message: element not interactable so I'm obviously doing something wrong but I'm not sure what.
use
btn = driver.findElementsByXPath("//button[contains(text(),'Load more')]");
You are not finding the right element. This is the reason why it is not "interactable"
There are several issues with your code:
The "Load more" button is initially out of the view, so you have to scroll the page in order to click it.
Your locator is bad.
You need to wait for elements to appear on the page before accessing them. WebDriverWait expected_conditions explicit waits should be used for that, not hardcoded sleeps.
The following code works, it scrolls the page and clicks "Load more" 1 time.
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 20)
url = "https://github.com/topics"
driver.get(url)
load_more = wait.until(EC.presence_of_element_located((By.XPATH, "//button[contains(.,'Load more')]")))
load_more.location_once_scrolled_into_view
time.sleep(1)
load_more.click()
UPD
You can simply modify the above code to make it clicking Load more button while it presented.
I implemented this with infinite while loop making a break if Load more button not found. This code works.
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)
url = "https://github.com/topics"
driver.get(url)
while True:
try:
load_more = wait.until(EC.presence_of_element_located((By.XPATH, "//button[contains(.,'Load more')]")))
load_more.location_once_scrolled_into_view
time.sleep(1)
load_more.click()
except:
break
I have a python script. When it tries to click on "https://www.etsy.com/" site language change button using selenium It navigates to "href" URL. but I want it to show the pop-up.
Image
This is how the code look like
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("https://www.etsy.com/")
WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR,'a[id=locale-picker-trigger]'))).click()
driver.close()
What is the issue here?
Use element_to_be_clickable() instead, and js executor to click.
driver.get("https://www.etsy.com/")
lang=WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,'a[id=locale-picker-trigger]')))
driver.execute_script("arguments[0].click();", lang)
Update
driver.get("https://www.etsy.com/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH,"//button[contains(., 'Accept')]"))).click()
lang=WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,'a[id=locale-picker-trigger]')))
driver.execute_script("arguments[0].click();", lang)
browser snapshot
I am trying to get all the news links of articles by clicking the load more or next button from the given newspaper link but I am failing to do so in continuation...
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
# Here Chrome is used
# URL of website
url = "https://www.business-standard.com/search?q=economy"
# Opening the website
driver.get(url)
wait = WebDriverWait(driver, 20)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.btnno"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='next-colum']"))).click()
from selenium import webdriver
import time
driver = webdriver.Chrome()
driver.get('https://www.worldometers.info/coronavirus/country/canada/')
time.sleep(1)
button = driver.execute_script("window.scrollTo(0, 5500)")
button1 = driver.find_element_by_xpath('/html/body/div[4]/div[2]/div[1]/div[8]/div/div[9]/a/button')
button1.click()
Trying to click this button using xpath but it just doesn't seem to click the button to extend the webpage. I first tried to copy by xpath which didn't work and then I copied the full xpath which also didn't work. After that, I tried to find by name and enter the text on the button and click the button. But none of these methods actually worked and clicked the button. So how can I click this button so that it extends the webpage allowing me to scrape more of the webpage?
The problem was that the button wasn't at "clickable state" despite was rendered in page at moment.
To obviate this, you have to add an explicit wait to tell the driver to wait until it's state change to clickable.
Try to use this code instead :
from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get('https://www.worldometers.info/coronavirus/country/canada/')
time.sleep(1)
button = driver.execute_script("window.scrollTo(0, 5500)")
button1 = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '/html/body/div[4]/div[2]/div[1]/div[8]/div/div[9]/a/button')))
button1.click()
To click on View more News Induce WebdriverWait and wait for presence_of_element_located() and following css selector.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('https://www.worldometers.info/coronavirus/country/canada/')
button=WebDriverWait(driver,20).until(EC.presence_of_element_located((By.CSS_SELECTOR,"a.load-more__btn>.btn")))
button.location_once_scrolled_into_view
button.click()
Or Use following xpath
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('https://www.worldometers.info/coronavirus/country/canada/')
button=WebDriverWait(driver,20).until(EC.presence_of_element_located((By.XPATH,"//a[#class='load-more__btn']/button[#class='btn']")))
button.location_once_scrolled_into_view
button.click()
I am trying to scrape a website. Where in I have to press a link. for this purpose, I am using selenium library with chrome drive.
from selenium import webdriver
url = 'https://sjobs.brassring.com/TGnewUI/Search/Home/Home?partnerid=25222&siteid=5011&noback=1&fromSM=true#Applications'
browser = webdriver.Chrome()
browser.get(url)
time.sleep(3)
link = browser.find_element_by_link_text("Don't have an account yet?")
link.click()
But it is not working. Any ideas why it is not working? Is there a workaround?
You can get it done in several ways. Here is one of such. I've used driver.execute_script() command to force the clicking. You should not go for hardcoded delay as they are very inconsistent.
Modified script:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
url = 'https://sjobs.brassring.com/TGnewUI/Search/Home/Home?partnerid=25222&siteid=5011&noback=1&fromSM=true#Applications'
driver = webdriver.Chrome()
driver.get(url)
item = wait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "a[ng-click='newAccntScreen()']")))
driver.execute_script("arguments[0].click();",item)