I am trying to accept cookies and search in google. But I am facing a problem that I never faced before.
Webdriver cant find the accept cookies button element. I looked everything. I tried to see if something changes the xpath(Is there any thing triggers any other element so the xpath will change) but I have zero in my hand. I tried using accept.send_keys(Keys.RETURN) and accept.click(). Nothing seems to work.
What I have for now;
def GoogleIt():
path = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(path)
driver.get("https://google.com")
wait(5)
accept = driver.find_element_by_xpath("/html/body/div/c-wiz/div[2]/div/div/div/div/div[2]/form/div/div[2]")
accept.send_keys(Keys.RETURN)
accept.click()
search = driver.find_element_by_xpath("/html/body/div/div[2]/form/div[2]/div[1]/div[1]/div/div[2]/input")
search.click()
search.send_keys(talk)
Note: wait() is a different name of time.sleep()
Error;
Traceback (most recent call last):
File "C:\Users\Teknoloji\AppData\Local\Programs\Python\Python38-32\lib\tkinter\__init__.py", line 1883, in __call__
return self.func(*args)
File "C:/Users/Teknoloji/Desktop/Phyton/Assistant V1/Assistant.py", line 20, in GoogleIt
accept = driver.find_element_by_xpath("/html/body/div/c-wiz/div[2]/div/div/div/div/div[2]/form/div/div[2]")
File "C:\Users\Teknoloji\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users\Teknoloji\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\Teknoloji\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\Teknoloji\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div/c-wiz/div[2]/div/div/div/div/div[2]/form/div/div[2]"}
(Session info: chrome=85.0.4183.121)
Process finished with exit code -1
If you need more information I am here.
Thanks...
To click on accept cookies button which is inside an iframe you need to switch to iframe first.
Induce WebDriverWait() and frame_to_be_available_and_switch_to_it() and following css selector.
Induce WebDriverWait() and element_to_be_clickable() and following xpath.
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src^='https://consent.google.com']")))
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//div[#id='introAgreeButton']"))).click()
You need to import following libraries.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
In case KunduK's solution doesn't work for some of those reading this, use "By.ID" parameter instead of the "By.CSS_SELECTOR" in the frame_to_be_available_and_switch_to_it() method -
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.ID, "name_of_the_iframe")))
After the latest update of Selenium, the above solutions did not work for me.
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium import webdriver
from selenium.webdriver.common.by import By
firefox_binary = FirefoxBinary()
browser = webdriver.Firefox(firefox_binary=firefox_binary)
Let's say you want to get this page from Google:
browser.get('https://www.google.com/search?q=cats&source=lnms&tbm=isch')
To continue, you have to click on "Accept all"
Then, you use Selenium to click the button:
browser.find_element(By.XPATH,"//.[#aria-label='Accept all']").click()
Then, you can continue to Google page!
Related
I got all the right modules that I need and my code looks pretty good.I am trying to click the add shortcut button using selenium, it is my first time using selenium, but I am pretty sure I did all of the code right. This is all of my code:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get('chrome://newtab')
add_button = driver.find_element_by_id('addShortcut')
add_button.click()
This is the chrome elements:
https://i.stack.imgur.com/m0DJR.png
This is my error:
Traceback (most recent call last):
File "C:\Users\User\Desktop\Srikar's Stuff\Programming\Python\WebScraper.py", line 7, in <module>
add_button = driver.find_element_by_id('addShortcut')
File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="addShortcut"]"}
(Session info: chrome=89.0.4389.90)
Is it an error with my code or something else?
Help!!??
So Google is using the feature of Shadow Root. It hides what is under it as an Iframe. I found the following workaround:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get('chrome://newtab')
delay = 1 # seconds
js_query = "return document.getElementsByTagName('ntp-app')[0].shadowRoot.getElementById('mostVisited').shadowRoot"
try:
_ = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.TAG_NAME, 'ntp-app')))
print("Page is ready!")
except TimeoutException:
print("Loading took too much time!")
most_visited = driver.execute_script(js_query)
add_button = most_visited.find_element_by_id('addShortcut')
add_button.click()
This should work though it is a little bit hacky
The icons aren't on that page. They're in an <iframe> called chrome-untrusted://new-tab-page/custom_background_image?url=, and I don't think you can bring that up on its own. Why would you want to? Unless you login, this Chrome isn't connected to your desktop Chrome.
I trying to click on the attribute class container in the tag div with a librarie Selenium. Here is code :
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('https://www.flashscore.com/')
driver.find_element(By.CSS_SELECTOR, ".header__button header__button--search").click()
And here is error display :
>>> driver.find_element(By.CSS_SELECTOR, ".header__button header__button--search").click();
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\avis\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\avis\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\avis\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py",
line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css
selector","selector":".header__button header__button--search"}
(Session info: chrome=85.0.4183.102)
the code is inspired of the doc of selenium : https://www.selenium.dev/documentation/en/getting_started_with_webdriver/performing_actions_on_the_aut/
I did some research and find a function which makes it possible to overcome the exception and it'is :element_to_be_clickable()
it is used to wait as the element is display and be clickable, according to the doc.
And I used it as this :
> > element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "header__button
> header__button--search"))
> element.click();
But this syntaxe error is display in console :
File "", line 2
element.click()
^ SyntaxError: invalid syntax
while I not see of syntax error.
From where can arise from the error ?
And function use is good ?
In HTML, you specify multiple classes with a space:
<span class="class1 class2">....</span>
In selenium, when searching for a single class, just use one of the class names:
driver.find_element(By.CSS_SELECTOR, ".class1")
This will code will open the site with Selenium and click the Search button:
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
# prevent version errors and plugin warning, may not be needed for you
options = webdriver.ChromeOptions()
options.add_argument("disable-extensions")
options.add_argument("disable-plugins")
options.experimental_options["useAutomationExtension"] = False # prevent load error - Error Loading Extension - Failed to load extension from ... - Could not load extension from ... Loading of unpacked extensions is disabled
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
# main code
driver.get('https://www.flashscore.com/')
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".header__button--search")))
driver.find_element(By.CSS_SELECTOR, ".header__button--search").click()
To find an element using multiple classes, check this post:
Find div element by multiple class names?
I am doing a side project for fun to automate the website https://10fastfingers.com/ in order for Selenium to complete all achievements of the website automatically. You can find my current code in my repository: https://github.com/jasperan/pyfastfingers
However, I have encountered a problem with the login page:
https://10fastfingers.com/login
Selenium does not let me locate the following HTML tags, whose corresponding xpath values are presented below, right away:
//*[#id="UserEmail"]
[//*[#id="UserPassword"]
It seems like the website does not load them properly the first time, because even when I inspect these manually (with my own web client, Firefox, Chrome, or even Chromium but launched by myself...), I get automatically redirected to its grandparent:
/html/body
When I've located this element manually, after a second inspection I can redirect to my desired email and password elements.
However, programatically, I can't do that. No matter how many times I try to locate the element, it doesn't properly locate, throwing me the following exception every time:
File "pyfastfingers.py", line 112, in <module>
main()
File "pyfastfingers.py", line 100, in main
do_login(driver)
File "pyfastfingers.py", line 74, in do_login
password = driver.find_element_by_xpath('[//*[#id="UserPassword"]')
File "/home/j/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "/home/j/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
'value': value})['value']
File "/home/j/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/j/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: Unable to locate an element with the xpath expression [//*[#id="UserPassword"] because of the following error:
SyntaxError: Failed to execute 'evaluate' on 'Document': The string '[//*[#id="UserPassword"]' is not a valid XPath expression.
(Session info: chrome=74.0.3729.169)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Linux 4.18.0-21-generic x86_64)
Here is some code that corresponds to my login function:
driver.get('https://10fastfingers.com/login')
placeholder = driver.find_element_by_xpath('/html/body')
email = driver.find_element_by_xpath('//*[#id="UserEmail"]')
password = driver.find_element_by_xpath('[//*[#id="UserPassword"]')
email.send_keys(os.environ['FINGERS_EMAIL'])
password.send_keys(os.environ['FINGERS_PASSWORD'])
login_button = driver.find_element_by_id('login-form-submit')
login_button.click()
# Login complete
You can find the complete code in my repository.
The issue is in
driver.find_element_by_xpath('[//*[#id="UserPassword"]')
Remove first '[' in the xpath.
driver.find_element_by_xpath('//*[#id="UserPassword"]')
Have you tried the wait until function.
wait = WebDriverWait(driver, 10)
men_menu = wait.until(ec.visibility_of_element_located((By.XPATH, "//*[#id="UserPassword"]")))
Your issue is xpath expression:
[//*[#id="UserPassword"]
It should be:
//*[#id="UserPassword"]
But it seem like you can use .find_element_by_id instead .find_element_by_xpath, looks better.
driver.get('https://10fastfingers.com/login')
email = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.ID, 'UserEmail')))
email.send_keys(os.environ['FINGERS_EMAIL'])
password = driver.find_element_by_id('UserPassword')
login_button = driver.find_element_by_id('login-form-submit')
password.send_keys(os.environ['FINGERS_PASSWORD'])
login_button.click()
Following import:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I run the script and it gives me one of two errors:
selenium.common.exceptions.ElementNotInteractableException: Message: Element <a class="grid_size in-stock" href="javascript:void(0);"> could not be scrolled into view
or
Element not found error
Right now this is the error it's giving. Sometimes it works, sometimes it doesn't. I have been trying to change the timing around to get it working right but to no avail. It will not work.
Code:
import requests
from selenium.webdriver.support import ui
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
def get_page(model, sku):
url = "https://www.footlocker.com/product/model:"+str(model)+"/sku:"+ str(sku)+"/"
return url
browser = webdriver.Firefox()
page=browser.get(get_page(277097,"8448001"))
browser.find_element_by_xpath("//*[#id='pdp_size_select_mask']").click()
link = ui.WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#size_selection_list")))
browser.find_element_by_css_selector('#size_selection_list').click()
browser.find_element_by_css_selector('#size_selection_list > a:nth-child(8)').click()
browser.find_element_by_xpath("//*[#id='pdp_addtocart_button']").click()
checkout = browser.get('https://www.footlocker.com/shoppingcart/default.cfm?sku=')
checkoutbutton = browser.find_element_by_css_selector('#cart_checkout_button').click()
The website automatically opens the size_selection_list div, so you don't need to click on it. But you do need to wait for the particular list element that you want to select. This code worked for me on this site a couple of times consistently.
from selenium.webdriver.support import ui
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
def get_page(model, sku):
url = "https://www.footlocker.com/product/model:"+str(model)+"/sku:"+ str(sku)+"/"
return url
browser = webdriver.Firefox()
page=browser.get(get_page(277097,"8448001"))
browser.find_element_by_xpath("//*[#id='pdp_size_select_mask']").click()
shoesize = ui.WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'a.grid_size:nth-child(8)')))
shoesize.click()
browser.find_element_by_xpath("//*[#id='pdp_addtocart_button']").click()
checkout = browser.get('https://www.footlocker.com/shoppingcart/default.cfm?sku=')
checkoutbutton = browser.find_element_by_css_selector('#cart_checkout_button').click()
I don't have enough reputation to comment, so: can you provide more of the stack trace for the "element not found" error? I stepped through the live Foot Locker site and didn't find the cart_checkout_button, though I may have made a mistake.
I'm also not sure (in your specific example) that you need to be clicking the size_selection_list the first time, before clicking the child, but I'm not as concerned about that. The syntax overall looks OK to me.
edit with the provided stack trace:
Traceback (most recent call last): File "./footlocker_price.py", line 29, in browser.find_element_by_css_selector('#size_selection_list > a:nth-child(8)').click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 501, in _execute
return self._parent.execute(command, params)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element could not be scrolled into view
What this means is that the click on #size_selection_list > a:nth-child(8) is failing because it can't be interacted with directly. Some other element is in the way. reference
Due to the way the particular page you're interacting with works (which is here for others reading this), I believe the size selection list is simply hidden when the page loads, and is displayed after you click on the Size button.
It seems like the ui.WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#size_selection_list"))) line should do what you want, though, so I'm at a little bit of a loss. If you add in a time.sleep(5) before you click on the size element, does it work? It's not ideal but maybe it will get you moving on to other parts of this.
I am trying to browse facebook via selenium in python.
Here is my script so far.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
usr = ""# I have put 3 different accounts and tested it. Same error
pwd = ""
driver = webdriver.Chrome('E:\python_libs\chromedriver.exe')
driver.get("https://www.facebook.com")
assert "Facebook" in driver.title
elem = driver.find_element_by_id("email")
elem.send_keys(usr)
elem = driver.find_element_by_id("pass")
elem.send_keys(pwd)
elem.send_keys(Keys.RETURN)
elem = driver.find_element_by_css_selector(".input.textInput")
elem.send_keys("Posted using Python's Selenium WebDriver bindings!")
elem = driver.find_element_by_css_selector(".selected")
elem.click()
time.sleep(5)
driver.close()
This script when run on windows with chromedriver obtained and correctly installed returns an error.
Traceback (most recent call last):
File "C:\Users\Home\Desktop\facebook_post.py", line 28, in <module>
elem.click()
File "C:\Python27\lib\site-packages\selenium-2.42.0- py2.7.egg\selenium\webdriver\remote\webelement.py", line 60, in click
self._execute(Command.CLICK_ELEMENT)
File "C:\Python27\lib\site-packages\selenium-2.42.0- py2.7.egg\selenium\webdriver\remote\webelement.py", line 370, in _execute
return self._parent.execute(command, params)
File "C:\Python27\lib\site-packages\selenium-2.42.0-py2.7.egg\selenium\webdriver\remote\webdriver.py", line 172, in execute
self.error_handler.check_response(response)
File "C:\Python27\lib\site-packages\selenium-2.42.0-py2.7.egg\selenium\webdriver\remote\errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: u'unknown error: Element is not clickable at point (481, 185). Other element would receive the click: <input type="file" class="_n _5f0v" title="Choose a file to upload" accept="image/*" name="file" id="js_0">\n (Session info: chrome=35.0.1916.114)\n (Driver info: chromedriver=2.10.267521,platform=Windows NT 6.1 x86)'
I am unable to make head or tail of this.
Any help would be appreciated.
The Graph API is not feasible here as I want to browse as myself and not as some application.
If however browsing as myself can be done by graph API or some other means, please do tell.
If you're doing this process several times, Selenium has some options in WebDriverWait to not waste too much time and check to see if elements are visible.
Here is the documentation on Waits in Selenium and
this is a section of the Python documentation of Selenium that talks about the Expected Conditions classes. It seems that "clickable" is one conditions that you can check for.
In my case, I wrote a simple function that took care of visibility and clicking and called it every time I needed to click on something dynamic.
My example code:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Browser = webdriver.Chrome()
def wait_until_visible_then_click(element):
element = WebDriverWait(Browser,5,poll_frequency=.2).until(
EC.visibility_of(element))
element.click()
EDIT:
The links above seem to be nerfed. This is the new documentation on waits and here are the expected conditions docs.