Develop Reference

Python, selenium - why do i get this error?

I want to access google maps with python, but at first you have to click accept cookies button and I don't know why but I keep getting this error:
File "file", line 12, in
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,
'//*[#id="introAgreeButton"]'))).click() File
"C:\Users\gassp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\selenium\webdriver\support\wait.py",
line 80, in until
raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:
Here is my code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('PATH')
url = 'https://www.google.com/maps/'
driver.get(url)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]'))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
driver.submit()

According to https://selenium-python.readthedocs.io/api.html :
exception selenium.common.exceptions.TimeoutException(msg=None, screen=None, stacktrace=None)
Bases: selenium.common.exceptions.WebDriverException
Thrown when a command does not complete in enough time.
Hence you either need to increase the time in WebDriverWait(driver, 10) or check if the condition EC.element_to_be_clickable is even fulfillable.
Furthermore, try the following implementation:
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]')))
element.click()
That fixed a few problems at my end with a similar code structure.

When I tried to access the site I did not found the xpath locator //*[#id="introAgreeButton"] in chrome dev tool. Not sure what element it refers to.
However when I commented that line and added the click event on the search button, I was able to get the results.
Also there is no submit() method available on driver instance.
Below was my trial using the code you have provided -
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//button[#id='searchbox-searchbutton']"))).click()
Output :-

The problem was that it didn't change between the main frame and the accept cookie frame. Here is the soulution code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('C:\\Users\\gassp\\OneDrive\\Namizje\\Python.projects\\chromedriver.exe')
url = 'https://www.google.com/maps/'
driver.get(url)
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//*[#id="consent-bump"]/div/div[1]/iframe')))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
driver.submit()

How to extract data with selenium webdriver

Hello I am trying to extract the odds of this webpage : https://www.netbet.fr/derniere-minute?filter=13
Here is my python script :
#!/usr/bin/python3
# -*- coding: utf­-8 ­-*-
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import os
options = Options()
options.headless = True
options.add_argument("window-size=1400,800")
options.add_argument("--no-sandbox")
options.add_argument("--disable-gpu")
options.add_argument("start-maximized")
options.add_argument("enable-automation")
options.add_argument("--disable-infobars")
options.add_argument("--disable-dev-shm-usage")
driver = webdriver.Chrome(options=options)
driver.get('https://www.netbet.fr/derniere-minute?filter=13')
odds = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, '//div[contains(#class, "nb-odds_amount")]')))]
print(odds, '\n')
driver.close()
driver.quit()
The output gives me that :
Traceback (most recent call last):
File "./azerty.py", line 31, in <module>
odds = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, '//div[contains(#class, "nb-odds_amount")]')))]
File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
This script run perfectly with other webpage but not in this case. Some help, thanks

Some elements are hidden and that's where the issue occurs. You wait until ALL elements are visible visibility_of_all_elements_located while some elements are hidden, so you will wait infinitely. Try waiting for presence instead of visibility to go around that issue presence_of_all_elements_located
odds = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH, '//div[contains(#class, "nb-odds_amount")]')))]

AttributeError("move_to requires a WebElement") AttributeError: move_to requires a WebElement error using Action Class in Selenium Webdriver

Link to website # https://www.2dehands.be/a/zakelijke-goederen/landbouw-veevoer/m1527886462-kleine-pakken-hooi.html?c=c41b759082ce0aa7411a74e54f8dbd13&previousPage=home
I want click this button red in this image:
.
Here is the code:
act = ActionChains(driver)
add_button = driver.find_elements_by_xpath("//button[#class='mp-Button mp-Button--secondary']")
subDiv = driver.find_elements_by_xpath("//div[#class='phone-number-container']")
sleep(5)
actions = ActionChains(driver)
actions.move_to_element(add_button)
actions.click(subDiv)
actions.perform()
But got this error:
raise AttributeError("move_to requires a WebElement")
AttributeError: move_to requires a WebElement

Please find below working solution:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as Wait
from selenium.webdriver.common.action_chains import ActionChains
#utilise chrome driver to open specified webpage
driver = webdriver.Chrome(executable_path=r"\chromedriver.exe")
driver.maximize_window()
driver.get("https://www.2dehands.be/a/zakelijke-goederen/landbouw-veevoer/m1527886462-kleine-pakken-hooi.html?c=c41b759082ce0aa7411a74e54f8dbd13&previousPage=home")
actionChains = ActionChains(driver)
element = WebDriverWait(driver, 10).until(
EC.visibility_of_element_located((By.CSS_SELECTOR, ".contact-options-mobile:nth-child(2) span:nth-child(2)")))
actionChains.move_to_element(element).click().perform()
element1 = WebDriverWait(driver, 10).until(
EC.visibility_of_element_located((By.XPATH, "//span[#class='phone-number-bubble']")))
print element1.text
output : 0494383160

or you could have just given find_element_by_xpath instead of find_elements_by_xpath

Pycharm: Selenium with Python: Unable to find web elements using headless chrome

When I run the below code with headless chrome I'm facing issues while identifying the elements. The same code runs just runs fine by commenting the below lines with head full mode.
# chrome_options.add_argument('--headless')
# chrome_options.add_argument('--disable-gpu')
Test Details:
Operating system: Windows10
ChromeDriver: 75.0.3770.8
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
driver = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)
wait = WebDriverWait(driver,30)
driver.maximize_window()
driver.get('https://learn.letskodeit.com/')
print(driver.title)
wait.until(EC.element_to_be_clickable((By.LINK_TEXT, 'Login'))).click()
wait.until(EC.visibility_of_element_located((By.ID, 'user_email'))).send_keys("test#email.com")
wait.until(EC.visibility_of_element_located((By.ID, 'user_password'))).send_keys("abcabc")
wait.until(EC.visibility_of_element_located((By.NAME, 'commit'))).click()
print(driver.title)
driver.close()
driver.quit()
Output:
"C:\Program Files (x86)\Python37-32\python.exe" C:/PycharmProjects/seleniumwd2/basics/RunHeadlessChromeTests.py
Checking for win32 chromedriver:75.0.3770.8 in cache
Driver found in C:\Users\vishr\.wdm\chromedriver\75.0.3770.8\win32/chromedriver.exe
Home | Let's Kode It
Traceback (most recent call last):
File "C:/PycharmProjects/seleniumwd2/basics/RunHeadlessChromeTests.py", line 15, in <module>
wait.until(EC.element_to_be_clickable((By.LINK_TEXT, 'Login'))).click()
File "C:\Users\vishr\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\support\wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Process finished with exit code 1

for headless browser you have to set the window size to fire on event.Because headless browser can't recognise where to click without window size.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('window-size=1920x1080')
driver = webdriver.Chrome(executable_path='path/to/chrome driver',options=chrome_options)
wait = WebDriverWait(driver,30)
driver.maximize_window()
driver.get('https://learn.letskodeit.com/')
print(driver.title)
wait.until(EC.element_to_be_clickable((By.LINK_TEXT, 'Login'))).click()
wait.until(EC.visibility_of_element_located((By.ID, 'user_email'))).send_keys("test#email.com")
wait.until(EC.visibility_of_element_located((By.ID, 'user_password'))).send_keys("abcabc")
wait.until(EC.visibility_of_element_located((By.NAME, 'commit'))).click()
print(driver.title)
driver.close()
driver.quit()
Printed output on console on headless mode.
Home | Let's Kode It
Let's Kode It

You're clicking a button that loads another page.
You have to wait for the other page to load. This is tricky to get perfect.
The easiest solution:
sleep(3)
The better solution using implicit waiting:
driver.implicitly_wait(3)
An even better solution using explicit waiting:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(driver, 3).until(EC.presence_of_element_located((By.ID, "user_email")))
An even better better solution would be to catch exceptions, and layer all these solutions.

To click() on the element with text as Login you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following solutions:
Using PARTIAL_LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "Login"))).click()
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.navbar-link.fedora-navbar-link[href='/sign_in']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='navbar-link fedora-navbar-link' and #href='/sign_in']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

VISHVAMBRUTHJAVAGALTHIMMEGOWDA,
I tried your code and was getting the same exception, First I thought that username was in Frame or iframe but it is not.
Then I tried to introduce webdriver wait and it worked just fine :
Code :
wait = WebDriverWait(driver,10)
driver.maximize_window()
driver.get("https://learn.letskodeit.com/")
print(driver.title)
driver.find_element_by_xpath("//a[contains(text(),'Login')]").click()
wait.until(EC.visibility_of_element_located((By.ID, 'user_email'))).send_keys("test#email.com")
#driver.find_element_by_id("user_email").
driver.find_element_by_id("user_password").send_keys("abcabc")
driver.find_element_by_name("commit").click()
print(driver.title)
Remember to imports these :
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
EDIT 1 :
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu') # Last I checked this was necessary.
driver = webdriver.Chrome("C:/Users/XXXX/Downloads/BrowserDriver/chromedriver_win32/chromedriver.exe", chrome_options=options)
wait = WebDriverWait(driver,10)
driver.get("https://learn.letskodeit.com/")
print(driver.title)
wait.until(EC.element_to_be_clickable((By.LINK_TEXT, 'Login'))).click()
wait.until(EC.visibility_of_element_located((By.ID, 'user_email'))).send_keys("test#email.com")
wait.until(EC.visibility_of_element_located((By.ID, 'user_password'))).send_keys("abcabc")
wait.until(EC.visibility_of_element_located((By.NAME, 'commit'))).click()
print(driver.title)

Finding element with explicit wait using selenium webdriver in python

I am trying to find the element with the link text, I am using following code
handle = driver.window_handles
#handle for windows
driver.switch_to.window(handle[1])
#switching to new window
link = wait.until(EC.presence_of_element_located((By.LINK_TEXT, "Followers ")))
And I am getting following traceback
Traceback (most recent call last):
File "<pyshell#28>", line 1, in <module>
link = wait.until(EC.presence_of_element_located((By.LINK_TEXT, "Followers ")))
File "C:\Python27\lib\site-packages\selenium\webdriver\support\wait.py", line 71, in until
raise TimeoutException(message)
TimeoutException: Message: ''
HTML of the element I am trying to select is
Followers <span class="profile_count">43,799</span>
How can I solve this problem??

If you use By.LINK_TEXT, there should be a link with exactly that text: Followers, but you have Followers 43,799.
In your case, you should use By.PARTIAL_LINK_TEXT instead:
wait.until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, 'Followers')))
UPDATE Here's working example:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome() # CHANGEME
driver.get('http://www.quora.com/Kevin-Rose')
element = WebDriverWait(driver, 2).until(
EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, "Followers"))
)
element.click()

Cheers
from selenium import webdrivercode
from selenium.webdriver.common.by import By
import selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("http://somedomain/url_that_delays_loading")
#Follow below syntax
element = WebDriverWait(driver, 10)
element.until(EC.presence_of_element_located((By.ID, '--webElement--')))