Develop Reference

Python selenium web-driver: selecting value from the dropdown list (No such Element Exception)

I am trying to extract the user reviews from google app using selenium webdriver. I loaded all reviews on the web-browser. Now I want to change the 'Most relevant' option of review page to 'Newest' option as shown in the picture.
Here is my code:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
baseurl = 'https://play.google.com/store/apps/details?id=com.mapmyrun.android2&showAllReviews=true'
driver.get(baseurl)
driver.find_element_by_xpath("//div[#class='OA0qNb ncFHed']//div[#class='MocG8c UFSXYb LMgvRb KKjvXb']//span[#class='vRMGwf oJeWuf']").click()
When I ran the code, it throws following error:
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[#class='OA0qNb ncFHed']//div[#class='MocG8c UFSXYb LMgvRb KKjvXb']//span[#class='vRMGwf oJeWuf']"}

To click on google review options as Newest Induce WebDriverWait() and wait for element_to_be_clickable() and following xpath option.
driver.get("https://play.google.com/store/apps/details?id=com.mapmyrun.android2&showAllReviews=true")
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//div[./span[text()='Most relevant']]"))).click()
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//div[#role='option'][./span[text()='Newest']]"))).click()
Browser snapshot:

If you want to simply click the div and click newest. Your class is dynamic and will change it's better to get an indentification that won't change.
driver.get("https://play.google.com/store/apps/details?id=com.mapmyrun.android2&showAllReviews=true")
driver.find_element_by_xpath("//span[text()='Most relevant']/parent::div").click()
time.sleep(3)
driver.find_elements_by_xpath("//span[text()='Newest']")[1].click()

How can I crawl Dynamic website with Selenium? There's error 'NoSuchElementException'

I've newly started to study Python and encountered NoSuchElementException while I'm doing crawling with SELENIUM.
I think target site is 'Dynamic webpage' hence I searched various way and tried but..... failed.
Would you help me? Thank in advance
Target site : http://www.kofiabond.or.kr/index_en.html
What I need : Click area on below picture and move to that page
enter image description here
Below is the code which I made:
from selenium import webdriver
driver=webdriver.Chrome('C:/Users/SMH/chromedriver.exe')
url='http://www.kofiabond.or.kr/index_en.html'
driver.get(url)
driver.maximize_window()
xpath = "//*[#id='genLv1_0_imgLv1']"
open_btn=driver.find_element_by_xpath(xpath)
driver.execute_script(open_btn)

To click on Final Quotation Yields link which is present inside an iframe you need to switch iframe first then hover on the main menu to element become visible on the page and then click on the element.
Induce WebDriverWait() and wait for frame_to_be_available_and_switch_to_it()
Induce WebDriverWait() and wait for visibility_of_element_located() and hover on the element
Induce WebDriverWait() and wait for element_to_be_clickable()
Code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
driver=webdriver.Chrome("C:/Users/SMH/chromedriver.exe")
driver.get('http://www.kofiabond.or.kr/index_en.html')
#Switch to iframe
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.NAME, 'fraAMAKMain')))
element=WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH,"//a[./img[#alt='bond Interest Rates']]")))
#Hover main menu
ActionChains(driver).move_to_element(element).perform()
#Click on the element once become clickable
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//a[./div[text()='Final Quotation Yields']]"))).click()
Browser snapshot

Selenium (Python) find_element_by_id does not work on schwab.com

Using Selenium (via Python), I am trying to locate the "Login" button of http://schwab.com.
The button is an element of type BUTTON and id='loginSubmitButton'.
I am using the following code:
from selenium import webdriver
driver = webdriver.Chrome()
driver.maximize_window()
driver.get("http://schwab.com")
driver.implicitly_wait(10)
driver.find_element_by_id("loginSubmitButton")
driver.close()
The browser correctly opens the page and the button is verifiably there (using Chrome dev tools), however Selenium fails to locate it.
I have tried many variations of this code, including using WebDriverWait, but nothing seems to work.
Suggestions are quite appreciated.

You are unable to click on that button because login form is under iFrame.
So first you need to switch into iframe, then only you'll be able to access those elements
username_frame = driver.find_element_by_id('LoginComponentForm')
driver.switch_to.frame(username_frame)
driver.find_element_by_id("loginSubmitButton").click()

As the the desired element is within an <iframe> so to invoke click() on the element you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get('https://www.schwab.com/')
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#LoginComponentForm")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span#LoginText"))).click()
Using XPATH:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get('https://www.schwab.com/')
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='LoginComponentForm']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//span[#id='LoginText']"))).click()
Reference
Here you can find a relevant discussion on Ways to deal with #document under iframe

Using find_element_by_xpath is more efficient than using find_element_by_id.
You can try using this
Also the login form is displayed on the iframe.
You need to switch to the iframe to click the button.
and then switch back to the default to be able to click or do actions on other elements outside the frame.
For switching to iframe you can use the following code :
driver.switch_to.frame(iframe)
Then use your click button code here.
For switching back to default content, use this line :
driver.switch_to.default_content()
This should work.

JavaScript works on chrome console but doesn't work in firefox or selenium code

I am trying to automate the login to the following page using selenium:
https://services.cal-online.co.il/Card-Holders/SCREENS/AccountManagement/Login.aspx?ReturnUrl=%2fcard-holders%2fScreens%2fAccountManagement%2fHomePage.aspx
Trying to find the elements of username and password using both id, css selector and xpath didn't work.
self._web_driver.find_element_by_xpath('//*[#id="txt-login-username"]')
self._web_driver.find_element_by_id("txt-login-password")
self._web_driver.find_element_by_css_selector('#txt-login-username')
For all three I get NoSuchElement exception
I tried the following JS script: document.getElementById('txt-login-username')
when I run this script in selenium or in firefox it returns Null
but when I run it in chrome console I get a result I can use.
Is there any way to make it work from the python code or to run this on the chrome console itself and not from the python execute_script?

To automate the login to the page using Selenium as the the desired elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://services.cal-online.co.il/Card-Holders/SCREENS/AccountManagement/Login.aspx?ReturnUrl=%2fcard-holders%2fScreens%2fAccountManagement%2fHomePage.aspx")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='calconnectIframe']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='txt-login-username']"))).send_keys("ariel6653")
driver.find_element_by_xpath("//input[#id='txt-login-password']").send_keys("ariel6653")
Browser Snapshot:
Here you can find a relevant discussion on Ways to deal with #document under iframe

found a solution to the problem. the problem really was that the object is inside an iframe.
I tried to use the solution suggested in Get element from within an iFrame
but got a security error. the solution is to switch frame the follwoing way:
driver.switch_to.frame("iframe")
and now you can use the normal find elment

Selenium can't locate the element, a search button using xpath

I can't find a solution how this element cannot be found by using a selenium xpath. Other button on other websites always working just fine. Usually what I would normally do is, just open up a page and inspect the element and I would right click it and copy the xpath. Done.
But this website, www.gsc.com.my (a malaysian cinema booking site). Seems not able to find the button. Is it protected by another security layer?
Lets see the code below,
from selenium import webdriver
chromedriver_path = './chromedriver.exe'
driver = webdriver.Chrome(chromedriver_path)
driver.get('https://www.gsc.com.my')
driver.find_element_by_xpath("""//*[#id="btnNext"]""").click()
The error Message:
no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="btnNext"]"}

Button is located inside an iframe, so you need to switch to that frame before clicking the button:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
driver.switch_to.frame('getQuickFilter_ctrl_ifrBox')
wait(driver, 10).until(EC.element_to_be_clickable((By.ID, "btnNext"))).click()

Because there are two elements with id btnNext, you'll have to specify which of them using an index, 1 for the first, 2 for the second.
driver.find_element_by_xpath("""//*[#id="btnNext"][1]""").click()

You can try with this css selector :
div.container input#btnNext
Note that you will need to switch to iframe first , cause the check button is in a iframe.
For switching to iframe :
driver.switch_to.frame(driver.find_element_by_id("getQuickFilter_ctrl_ifrBox"))
and for clicking on check Button
wait = WebDriverWait(driver, 10)
check_button = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.container input#btnNext")))
check_button.click()