Develop Reference

Python Selenium impossible, tried id, class can't find XPath

I am trying to enter a field with text (email), in the username box. However, it is not interacting with it. I have it in a try & except statement but it only spits out excepts.
Website is account.protonmail.com/signup
I have tried everything: xpath, class, id, but nothing works. I am losing my mind!
Password works fine:
Does not work:
driver.find_element_by_xpath('//*[#id="username"]').send_keys(password)
Does work:
driver.find_element_by_xpath('//*[#id="repeat-password"]').send_keys(password)
And yes, I do wait until JavaScript finishes running.

it is in iframe, and needs explicit waits :-
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.get("https://account.protonmail.com/signup")
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[sandbox^='allow-scripts']")))
ele = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#username"))).send_keys('someemail#gmail.com')
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Element you are trying to access is inside an iframe. First you need to switch to the frame like this
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[contains(#src,'account-api.protonmail.com')]"))

It's very difficult for you to write everything, make it easier
click = driver.find_element_by_xpath('//*[#id="username"]').click()
click.send_keys('')
click.send_keys(password)

Selenium: Element Click Intercepted while submitting a form

So I'm trying to submit a form but something is either preventing me from accessing the box or I'm using a wrong element but I think I'm using the correct one.
Here is my code:
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path = 'mypath/chromedriver.exe')
driver.maximize_window()
#driver.implicitly_wait(50)
driver.get("https://ai.fmcsa.dot.gov/SMS")
wait = WebDriverWait(driver, 20)
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[#title='Close']"))).click()
wait = WebDriverWait(driver, 20)
driver.find_element_by_xpath('//*[#id="home-body"]/div[1]/div/div[1]/form/label').click()
driver.find_element_by_xpath('//*[#id="home-body"]/div[1]/div/div[1]/form/label').send_keys('1818437')
driver.find_element_by_xpath('/html/body/div[3]/div[2]/article/section[2]/div[1]/div/div[1]/form/input[2]').click();
What I'm getting on the output is
ElementClickInterceptedException: Message: element click intercepted:
Element ... is
not clickable at point (553, 728). Other element would receive the
click:
(Session info: chrome=93.0.4577.63)
What might be the issue?

Things to noted down in this scenario :-
When you define an explicit waits wait = WebDriverWait(driver, 20), you can always use wait reference in the scope. you do not need to create again and again in same class.
Try to avoid absolute xpath /html/body/div[3]/div[2]/article/section[2]/div[1]/div/div[1]/form/input[2], try with relative xpath/xpath axes.
When we try to send keys to some element, in general it should be a input tag, not label
You may have to scroll, may be not in this case but when you scroll manually to interact with elements in UI, same has to automated with Selenium as well.
Also I observed to this webapp that search and input tags are duplicated, so I have used xpath indexing [2] to handle.
Sample code :-
driver = webdriver.Chrome(executable_path = 'mypath/chromedriver.exe')
driver.maximize_window()
#driver.implicitly_wait(50)
driver.get("https://ai.fmcsa.dot.gov/SMS")
wait = WebDriverWait(driver, 20)
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[#title='Close']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "(//input[#name='MCSearch'])[2]"))).send_keys('1818437')
wait.until(EC.element_to_be_clickable((By.XPATH, "(//input[#name='search'])[2]"))).click()

You can use below xpaths too.
driver.find_element_by_xpath("//div[#class='sms-search-box']//input[1]").send_keys('1818437')
driver.find_element_by_xpath("//div[#class='sms-search-box']//input[2]").click()

Xpath you are using is not right. Your xpath for the input field should be like this.
driver.find_element_by_xpath("//input[#name='MCSearch' and #placeholder='Type Name or U.S. DOT#']").send_keys("1818437")
driver.find_element_by_xpath("//input[#placeholder='Type Name or U.S. DOT#']//following::input[#value='Search']").click();

python selenium - Cant find the element of sign in button

I'm trying to create an automation test in Asos (for practice purpose only) however I'm having a hard time locating this sign-in element...
I need to click on that sign-in button.
these are the element I got in inspect:
a class="_1336dMe _1uUU2Co _1336dMe _1uUU2Co" href="https://my.asos.com/my-account?
lang=en-GB&store=COM&country=GB&keyStoreDataversion=3pmn72e-27"
data-testid="signin-link" tabindex="-1">Sign In

I had the same problem when trying to find this button on Google Maps. Is the sign in button on a pop up window? Then the problem is becuse you have to change between frames.
Here is a code sample:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('driver path')
url = 'url'
driver.get(url)
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//*[#id="consent-bump"]/div/div[1]/iframe')))
agree = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="introAgreeButton"]/span/span')))
agree.click()
#back to the main page
driver.switch_to_default_content()
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchboxinput"]'))).send_keys('gostilne')
search = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[#id="searchbox-searchbutton"]')))
search.click()
Make shure that the xpath for frames and buttons is correct.

Having problems clicking an element using Selenium

I'm working on a program which will trade stocks for me in the future. I have run into some problems after Logging In, typing the symbols of the stock, and now I need to click in some kind of way to be able to go on and buy the stock. I'm pretty confident that i know how to make a click with selenium but this tricks me out. I will give away the full code if anyone wants to try the program themself, just change the path to the browser! The account is fake, so don't worry.
Code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
browser = webdriver.Chrome('/Users/larskvist/downloads/chromedriver')
browser.get('https://www.forex.com/en-uk/account-login/')
username_elem = browser.find_element_by_name('Username')
username_elem.send_keys('kebababdulaziz#gmail.com')
password_elem = browser.find_element_by_name('Password')
password_elem.send_keys('KEbababdulaziz')
password_elem.send_keys(Keys.ENTER)
time.sleep(5)
search_elem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable(
(By.CSS_SELECTOR, "input.market-search__search-input")))
search_elem.click()
search_elem.send_keys('FB')
time.sleep(2)
search_click_elem = WebDriverWait(browser, 20).until(
EC.element_to_be_clickable((By.XPATH, "//app-market-table[#class='search-results-element ng-
star-inserted']//div[#class='price--buy clickable-price arrows-flashing']")))
search_click_elem.click
The IMG shows what i want to click, when clicked manually a pop up buy option appears.
Thanks in advance!

Seems like webdriver click is not working.
Induce JS executor to click on .
search_click_elem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//app-market-table[#class='search-results-element ng-star-inserted']//div[#class='price--buy clickable-price arrows-flashing']")))
browser.execute_script("arguments[0].click();", search_click_elem)
Browser snapshot:

Selenium crashes when I'm trying to parse the next page (and seven after it) on a website. Any way to tackle this?

I want to parse an IMDb film rating located here on around 8 pages. In order to do that I'm using Selenium, and I'm having trouble with clicks, proceeding algorithm to next page. In the end I need 1000 titles when I'll continue using BeautifulSoup. Code below isn't working, I need to use button 'NEXT' with this HTML:
<a class="flat-button lister-page-next next-page" href="/list/ls000004717/?page=2">
Next
</a>
This is the code:
from selenium import webdriver as wb
browser = wb.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
field = browser.find_element_by_name("flat-button lister-page-next next-page").click()
Error is the following:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".flat-button lister-page-next next-page"}
(Session info: chrome=78.0.3904.108)
I suppose I lack knowledge of syntax needed, or maybe I mixed it up a little. I tried searching on SO, though every example is pretty unique and I don't possess the knowledge to extrapolate these cases fully. Any way Selenium can handle that?

You could try using an XPath to query on the Next text inside the button. You should also probably invoke WebDriverWait since you are navigating across multiple pages, then scroll into view since this is at the bottom of the page:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
from time import sleep
browser = wb.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
# keep clicking next until we reach the end
for i in range(0,9):
# wait up to 10s before locating next button
try:
next_button = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(#class, 'page') and contains(text(), 'Next')]")))
# scroll down to button using Javascript
browser.execute_script("arguments[0].scrollIntoView(true);", next_button)
# click the button
# next_button.click() this throws exception -- replace with JS click
browser.execute_script("arguments[0].click();", next_button)
# I never recommend using sleep like this, but WebDriverWait is not waiting on next button to fully load, so it goes stale.
sleep(5)
# case: next button no longer exists, we have reached the end
except TimeoutException:
break
I also wrapped everything in a try / except TimeoutException block to handle the case where we have reached the end of pages, and Next button no longer exists, thus breaking out of the loop. This worked on multiple pages for me.
I also had to add an explicit sleep(5) because even after invoking WebDriverWait on element_to_be_clickable, next_button was still throwing StaleElementReferenceException. It seems like WebDriverWait was finishing before page was fully loaded, causing the status of next_button to change after it had been located. Normally adding sleep(5) is bad practice, but there did not seem to be another workaround here. If anyone else has a suggestion on this, feel free to comment / edit the answer.

There are a couple of ways that could work:
1. Use a selector for the next button and loop until the end:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
browser = webdriver.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
selector = 'a[class*="next-page"]'
num_pages = 10
for page in range(pages):
# Wait for the element to load
WebDriverWait(browser, 10).until(ec.presence_of_element_located((By.CSS_SELECTOR, selector)))
# ... Do rating parsing here
browser.find_element_by_css_selector(selector).click()
Instead of clicking on the element, the other option could be to navigate to the next page using broswer.get('...'):
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
# Set up browser as before and navigate to the page
browser = webdriver.Chrome()
browser.get('https://www.imdb.com/list/ls000004717/')
selector = 'a[class*="next-page"]'
base_url = 'https://www.imdb.com/list/ls000004717/'
page_extension = '?page='
# Already at page = 1, so only needs to loop 9 times
for page in range(2, pages + 1):
# Wait for the page to load
WebDriverWait(browser, 10).until(ec.presence_of_element_located((By.CSS_SELECTOR, selector)))
# ... Do rating parsing here
next_page = base_url + page_extension + str(page)
browser.get(next_page)
As a note: field = browser.find_element_by_name("...").click() will not assign field to a webelement, as the click() method has no return value.

You could try a partial css selector.
browser.find_element_by_css_selector("a[class*='next-page']").click()

To click on the element with text as NEXT till the 901 - 1,000 of 1,000 page you have to:
scrollIntoView() the element once the visibility_of_element_located() is achieved.
Induce WebDriverWait for the element_to_be_clickable()
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.imdb.com/list/ls000004717/')
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.pagination-range"))))
while True:
try:
WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.CSS_SELECTOR, "div.row.text-center.lister-working.hidden")))
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.pagination-range"))))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.flat-button.lister-page-next.next-page"))).click()
print("Clicked on NEXT button")
except TimeoutException as e:
print("No more NEXT button")
break
driver.quit()
Console Output:
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
Clicked on NEXT button
No more NEXT button