I started learning python with selenium to automate some of my tasks. I have a problem which I'm unable to solve.
I'm trying to create a script which would automatically click the 'next' button.
I can't locate a webpage element (button) with ID 'next'. I tried every possible solution - by ID, by xpath etc.
It's possible that part of the webpage in which I'm trying to find the element is dynamically generated by jquery.
This is the error I get:
Unable to locate element: [id="next"]
Here is what I've done until this point (fragments):
from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
import time
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
browser = webdriver.Firefox(executable_path=r'C:\Users\technik
informatyk\Desktop\python\geckodriver.exe')
browser.get('https://it-szkola.edu.pl/usr')
browser.maximize_window()
browser.implicitly_wait(3)
zgoda = browser.find_element_by_xpath('//*[#id="cookieLayerBoxButton"]')
zgoda.click()
browser.implicitly_wait(10)
emailElem = browser.find_element_by_id('login-box-field-input')
emailElem.send_keys('mylogin')
passwordElem = browser.find_element_by_id('password')
passwordElem.send_keys('mypassword')
passwordElem.submit()
time.sleep(2)
aktualnosci = browser.find_element_by_id('hBLAkt').click()
grandtest = browser.find_element_by_xpath('//* [#id="New178"]/div[2]/div[1]/p[2]/a').click()
lista = browser.find_element_by_link_text('Lista testów').click()
test = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH,"/html/body/div[2]/div[1]/div[2]/table/tbody/tr[5]/td[5]/a"))).click()
#browser.refresh()
#time.sleep(2)
nastepne = browser.find_element_by_id("next")
#nastepne = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.LINK_TEXT, "Następne")))
#nastepne = browser.find_element_by_link_text('Następne').click()
nastepne.click()
Screenshots
,
,
,
,
,
Your help will be much appreciated.
Try xpath:
browser.find_element_by_xpath('//button[#id="next"]')
Try getting all buttons:
btns = browser.find_elements_by_xpath('//button')
for btn in btns:
txt = btn.text
if txt == 'Następne':
btn.click()
Related
This is the website I am trying to automate some clicks:
I have tried clicking the button using Xpath and FullXpath, but still got no luck.
This is the simple code:
w = webdriver.Chrome(executable_path='chromedriver.exe',
chrome_options=options)
w.get("https://quillbot.com/")
time.sleep(5)
pasteXpath = "//button[contains(#class,'outlinedPrimary') and .//span[contains(text(),'Paste Text')]]"
element = w.find_element_by_xpath(pasteXpath).click()
But it fails with this message in the console:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="inOutContainer"]/div[2]/div[2]/div/div[1]/div/div/div[1]/div/div/div[2]/div/div/button/span[1]/div"}
Please show me how to automate this click using selenium.
I recommend using By, WebDriverWait, and expected_conditions in the place of .find_element_by_xpath.
After you click the paste button you will receive a permissions prompt. See below to get past it.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.chrome.service import Service
import time
import pyautogui
service = Service('C:\\Path_To_Your\\chromedriver.exe')
driver = webdriver.Chrome(service=service)
driver.get('https://quillbot.com/')
paste_button = WebDriverWait(driver, 3).until(EC.visibility_of_element_located(
(By.XPATH, "//span[text()='Paste Text']")))
paste_button.click()
time.sleep(2)
pyautogui.press('tab')
pyautogui.press('tab')
pyautogui.press('enter')
This will work:
pasteXpath = "//button[contains(#class,'outlinedPrimary') and .//span[contains(text(),'Paste Text')]]"
element = w.find_element_by_xpath(pasteXpath).click()
Don't forget to add some wait / delay before it to make sure the page is fully loaded.
Try to use CSS selector instead:
element = w.find_element_by_css_selector('div[class*="MuiGrid-root"] > div[class="jss473"]').click()
You can find all the doc about css selector here
I am using selenium to try to scrape data from a website (https://www.mergentarchives.com/), and I am attempting to get the innerText from this element:
<div class="x-paging-info" id="ext-gen200">Displaying reports 1 - 15 of 15</div>
This is my code so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Firefox()
driver.maximize_window()
search_url = 'https://www.mergentarchives.com/search.php'
driver.get(search_url)
assert 'Mergent' in driver.title
company_name_input = '//*[#id="ext-comp-1009"]'
search_button = '//*[#id="ext-gen287"]'
driver.implicitly_wait(10)
driver.find_element_by_xpath(company_name_input).send_keys('3com corp')
driver.find_element_by_xpath(search_button).click()
driver.implicitly_wait(20)
print(driver.find_element_by_css_selector('#ext-gen200').text)
basically I am just filling out a search form, which works, and its taking me to a search results page, where the number of results is listed in a div element. When I attempt to print the text of this element, I simply get a blank space, there is nothing written and no error.
[Finished in 21.1s]
What am I doing wrong?
I think you may need explicit Wait :
wait = WebDriverWait(driver, 10)
info = wait.until(EC.visibility_of_element_located((By.XPATH, "//div[#class = 'x-paging-info' and #id='ext-gen200']"))).get_attribute('innerHTML')
print(info)
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You may need to put a condition by verifying if search results loaded or not and once its loaded you can use below code
print(driver.find_element_by_id('ext-gen200').text)
I've written a script in Python in association with selenium to keep clicking on MORE button to load more items until there are no new items left to load from a webpage. However, my below script can click once on that MORE button available on the bottom of that page.
Link to that site
This is my try so far:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
print(elems.get_attribute("href"))
try:
loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
driver.execute_script("arguments[0].scrollIntoView();", loadmore)
loadmore.click()
except Exception:break
driver.quit()
How can I keep clicking on that MORE button until there are no such button left to click and parse the links as I've already tried using for loop.
I've managed to solve the problem pursuing sir Andersson's logic within my exising script. This is what the modified script look like.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
try:
loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
driver.execute_script("arguments[0].click();", loadmore)
wait.until(EC.staleness_of(loadmore))
except Exception:break
for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
print(elems.get_attribute("href"))
driver.quit()
why not just?
while (driver.FindElements(By.ClassName("more")).Count > 0)
{
driver.FindElement(By.ClassName("more")).Click();
//Some delay to wait lazyload to complete
}
c# example. pretty sure that it can be done with python as well
This is the webpage -- link. I am going to crawl (In Persian). I have a problem when I am going to click on the next page button. The XPath is:
nextpage = '//*[#id="ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom"]/a[1]/img'
page = driver.find_element_by_xpath(nextpage)
page.click()
After the page.click() I got the following error:
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
Some answers say there might be a duplicate XPath, but I could not find such a thing in the source of the webpage.
The complete code:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
driver = webdriver.Chrome(executable_path='./chromedriver')
url = 'http://hmi.mrud.ir/sabaa/SABAA/Home/Default.aspx?strTownship=0101&g=false'
driver.get(url)
time.sleep(10)
nextpage = '//*[#id="ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom"]/a[1]/img'
page = driver.find_element_by_xpath(nextpage)
page.click()
Thanks.
Use the following before clicking on the element:
driver.execute_script("arguments[0].scrollIntoView(true);", page)
This is scrolling to the element.
Hope it helps you!
As per the Website if you want to click on the Next Page Button you have to induce WebDriverWait for the WebElement to be clickable following either of the code block as follows :
nextpage = WebDriverWait(driver, 10).until(EC.element_to_be_clickable(By.XPATH,"//div[#id='ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom']/a[#class='dxp-lead dxp-button dxp-bi']/img[#class='dxWeb_pPrev_Aqua']"))
nextpage.click()
Or
nextpage = WebDriverWait(driver, 10).until(lambda x: x.find_element_by_xpath("//div[#id='ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom']/a[#class='dxp-lead dxp-button dxp-bi']/img[#class='dxWeb_pPrev_Aqua']"))
nextpage.click()
I am trying to press this "New Search" button. It appears on the top of the screen after entering a search on http://www.lexisnexis.com/hottopics/lnacademic/
I have looked at the Xpath and Unique Selector.
What I have tried:
browser.find_element_by_css_selector('#restoreButtons > a:nth-child(3)').click()
browser.find_element_by_xpath(id('restoreButtons')/x:a[3])
browser.find_element_by_xpath(/x:a[3])
For all three I get an "unable to locate element error"
To build on alecxes answer,
once you're inside the iframe, you can find the clickable using it's xpath:
new_search_xpath = '/html/body/div[2]/table/tbody/tr[2]/td[2]/div[1]/table/tbody/tr/td/span/a[3]'
new_search = driver.find_element(By.XPATH, new_search_xpath)
new_search.click()
You should consider installing firebug firefox addon to grab Xpaths:
https://addons.mozilla.org/en-US/firefox/addon/firebug/
This is because the element is inside an iframe. You have to be in the context to search elements inside. Use .switch_to.frame():
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
wait = WebDriverWait(driver, 10)
frame = wait.until(EC.presence_of_element_located((By.ID, "mainFrame")))
driver.switch_to.frame(frame)
FYI, here is the complete working code:
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('http://www.lexisnexis.com/hottopics/lnacademic/')
actions = ActionChains(driver)
wait = WebDriverWait(driver, 10)
frame = wait.until(EC.presence_of_element_located((By.ID, "mainFrame")))
driver.switch_to.frame(frame)
driver.find_element_by_id("terms").send_keys("Test")
driver.find_element_by_id("srchButt").click()