Develop Reference

Selenium only locates element when I open developer tools in chrome [duplicate]

This question already has answers here:
Click on ember.js enabled element using Selenium
(5 answers)
Closed 1 year ago.
At this stage, all the script aims to do is to locate the search bar in a digital library, send the name of the resource I am looking for, click on it and get the current url. The first bit (sending keys to search bar) works fine. But I have noticed that the second try/finally block only executes when I right click and inspect any element on the page. If I don't, I get the following error message :
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="ember767"]"}
My code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
PATH = "/Applications/chromedriver"
ser = Service(PATH)
driver = webdriver.Chrome(service=ser)
driver.maximize_window()
driver.implicitly_wait(20)
driver.get("https://browzine.com/libraries/1374/subjects")
print("Enter targeted Journal name:")
targeted_journal = input()
wait = WebDriverWait(driver, 10)
try:
button = driver.find_element(By.ID, "ember648")
driver.implicitly_wait(10)
ActionChains(driver).move_to_element(button).click(button).perform()
button.send_keys(targeted_journal)
button.send_keys(Keys.RETURN)
finally:
pass
# step 2 : click targeted journal and get current url
try:
driver.implicitly_wait(20)
click_journal = driver.find_element(By.ID, "ember767")
ActionChains(driver).move_to_element(click_journal).click(click_journal).perform()
targeted_url = driver.current_url
finally:
pass
print(targeted_url)
driver.quit()
Update - answer found
The ID initially collected for the targeted element was a dynamic one. It was only as such when the developer tools window was open. Therefore the script was only able to detect that particular ID when "Inspect element" was clicked as it was the only case in which the targeted element was assigned this particular ID ("ember767" in my case).
To avoid this issue, an Xpath was used to locate the element instead. See Mayank Shukla's answer below.

In some cases, element's ID or other attributes changes or attribute's value appears on mouse hover. So it's preferred to use xpath for locating such elements
In this scenario, I think the element's id is changing after you are hovering the mouse. Try using xpath and let me know if that works. Because might be after browser refresh this is happening i.e. element's id is changing.

StaleElementReferenceException on FirefoxWebElement.get_attribute even after WebDriverWait

I am trying to get the inner text of a specific element on a website (which runs on React production build) using Selenium. I am doing the following:
driver = webdriver.Firefox()
driver.get('some site using React')
WebDriverWait(driver, 10).until(
expected_conditions.text_to_be_present_in_element(
(By.CSS_SELECTOR, '.parent p'),
'some text here'
)
)
This all works - in fact, I can even access the element afterwards:
elem = driver.find_element(By.CSS_SELECTOR, '.parent p')
print(elem)
# -> <selenium.webdriver.firefox.webelement.FirefoxWebElement (session="57b82b73-0c6a-40c5-97e1-b5861b3d43e6", element="736edc49-cef0-4723-b69e-afabd80f1e9d")>
However, as soon as I try to do anything with this, I get a StaleElementReferenceException. For instance,
print(elem.get_attribute('innerText'))
or
print(elem.text)
both throw the following error:
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <p> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
I have also tried replacing expected_conditions.text_to_be_present_in_element with expected_conditions.presence_of_element_located, but the issue still persists. I am at a loss for why I am able to successfully await the element's loading, even recognising that the text in said element matches my format (i.e. we are able to read the text, somehow), and even being able to print the element itself - but when I attempt to get its text, even just read a property (not call a method), I get this exception. Sometimes, on perhaps every third attempt or so, it works - but then it goes back to not working again.
How can I get around this error, in order to access my text as soon as it matches my intended format (i.e. includes 'some text here')?
As per #PDHide's request, here is my full code:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium import webdriver
driver = webdriver.Firefox()
while True: # Running repeatedly, to ensure that the solution works consistently over multiple executions
driver.get('url')
WebDriverWait(driver, 10).until(
EC.text_to_be_present_in_element(
(By.CSS_SELECTOR, '.parent p'),
'some text here'
)
)
print(driver.find_element(By.CSS_SELECTOR, '.parent p').get_attribute('innerText'))
This usually works for one or two executions, before breaking and reverting to throwing a StaleElementReferenceException.

Stale element state that the DOM (Page HTML) was modified after the element was located.
so for example:
print(elem.get_attribute('innerText'))
clicksomebutton.click()
print(elem.get_attribute('innerText'))
It fail if click refreshes or loads something on the screen. FOr this to work you have to find element again.
print(elem.get_attribute('innerText'))
clicksomebutton.click()
print(driver.find_element_by_xpath('blabla').get_attribute('innerText'))

Can't click on dropdown button selenium Python

I would like to click on the element "Project" to display the dropdown list (see image below)
Using the selenium library in python elsewhere i get the error :
could not be scrolled into view
which is obtained, for instance, using this code:
driver = webdriver.Firefox()
driver.get(url)
driver.find_element_by_xpath('//div[#class="multiselect-container"]').click()
or using some code where I wait the element to be displayed, for instance like described here:
Message: Element <option> could not be scrolled into view while trying to click on an option within a dropdown menu through Selenium
mySelectElement = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.ID, "edit-projects")))
mySelectElement.click()
But I can't get it working.
Any would be appreciated.
The html source code may be found here:
https://rpidejr.hopto.org/f/8a909b51dcc34d09a00a/

I saved the source code file to my drive as test.html and opened it and click on the Project box with the below code. Just edit the paths to your local machine.
from selenium import webdriver
import time
driver = webdriver.Firefox(executable_path=r'C:\\Path\\To\\geckodriver.exe')
driver.get("file:///C:/Path/To/test.html")
time.sleep(1)
#project = driver.find_element_by_xpath("//select[#id='edit-projects']")
#project.click()
project_elements = driver.find_elements_by_xpath("//select[#id='edit-projects']")
for element in project_elements:
try:
element.click()
except Exception as e:
print(e)

selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable with Selenium and Python

I am currently working on a project which fills a form automatically. And the next button appears when the form is filled, that's why it gives me an error.
I have tried:
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,"//input[#type='button' and #class='button']")))
Next = driver.find_element_by_xpath("//input[#type='button' and #class='button']")
Next.click()
HTML:
<span class="btn">
<input type="button" value="Next" class="button" payoneer="Button" data-controltovalidate="PersonalDetails" data-onfieldsvalidation="ToggleNextButton" data-onclick="UpdateServerWithCurrentSection();" id="PersonalDetailsButton">
</input>
<div class="clearfix"></div>
</span>
ERROR:
selenium.common.exceptions.ElementClickInterceptedException: Message:
element click intercepted: Element is not clickable at point (203, 530).
Other element would receive the click: ... (Session info: chrome=76.0.3809.132)

If the path of the xpath is right, maybe you can try this method to solve this problem. Replace the old code with the following code:
button = driver.find_element_by_xpath("xpath")
driver.execute_script("arguments[0].click();", button)
I solved this problem before, but to be honestly, I don't know the reason.

This error message...
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (203, 530). Other element would receive the click: ... (Session info: chrome=76.0.3809.132)
...implies that the click() on the desired element was intercepted by some other element and the desired element wasn't clickable.
There are a couple of things which you need to consider as follows:
While using Selenium for automation using time.sleep(secs) without any specific condition to achieve defeats the purpose of automation and should be avoided at any cost. As per the documentation:
time.sleep(secs) suspends the execution of the current thread for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.
You can find a detailed discussion in How to sleep webdriver in python for milliseconds
As WebDriverWait returns the WebElement you can invoke the click() method directly.
Solution
To click on the button with value as Next you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.button#PersonalDetailsButton[data-controltovalidate='PersonalDetails']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='button' and #id='PersonalDetailsButton'][#data-controltovalidate='PersonalDetails']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

i faced similar issues, the .click() always returns a Not clickable exception. the
driver.execute_script('arguments[0].click()', button)
does the magic. You can also use it to execute any other js script this way
script = 'your JavaScript goes here'
element = driver.find_element_by_*('your element identifier goes here')
driver.execute_script(script, element)

I looked at the exact element that was causing it and it was a banner about consent/cookies. So at first, I made sure it clicked "OK" on the consent banner and then I clicked the other button that I needed. Hope it helps someone.

It look's like there are some other elements which are having the same xpath try changing the xpath something like this
Next = driver.find_element_by_xpath("//input[#id='PersonalDetailsButton']");
Next.Click();
or
Next = driver.find_element_by_xpath(//input[#value='Next' and #id='PersonalDetailsButton']);
Next.Click();
Try first xpath if that doesn't work go with the second one . If that also doesn't work try using sikuli. I am pretty sure that first xpath will work

I faced a similar issue and I observed something that might help to understand the root cause of the issue. In my case, I was able to click at an element being in PC view mode of the website but failed to do so in mobile view (in which I needed my script to run). I found out that in mobile view, ordering of elements (li in my case) changed in view while they remained same in the html document. That's why I was not able to click on it without actually scrolling to it first. It might also explain why this works: -
driver.execute_script("arguments[0].click();", button)

I don't have enough rep to comment but the common reason for this error might be Selenium locates the element from DOM on screen and locate the x-y coordinates (300, 650) then clicks on them but if some changes takes place on screen in between the click duration, for example google ads or some pop-up then it's unable to click on it resulting in this exception
I'm just guessing if anyone has a proper explanation to pls share

I had the same problem too. But my problem was not with the element. The button was activated with href. I changed the code from
<a class="services-button" href="desired url">
To
<a class="services-button" onclick="location.href='{% url "desired url" %}'";">

This solution worked for me :
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
driver = webdriver.Firefox(executable_path="")
driver.get("https://UrlToOpen")
action = ActionChains(driver)
firstLevelMenu = driver.find_element_by_id("menu")
firstLevelMenu.click()
source : http://allselenium.info/mouse-over-actions-using-python-selenium-webdriver/

"selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element is not clickable ... "
This exception occurs when element is not found on a web page (When the element we are looking for is at bottom part of the page which is not loaded yet)
So we you can scroll the page using javascript and load complete page and
from selenium.webdriver.common.by import By
from selenium import webdriver
url = "YOUR URL"
SCROLL_PAUSE_TIME = 0.5
driver = webdriver.Chrome()
driver.maximize_window()
driver.get(url)
def scroll_page():
i = 0
while i < 5:
# Scroll down to 500 pixel
driver.execute_script("window.scrollBy(0, 500)", "")
# Wait to load page
time.sleep(SCROLL_PAUSE_TIME)
# Will scroll only for 4 increments of 500px
i += 1

You could try:
driver.execute_script("arguments[0].click();", button)
This solution solved my problems when I faced similar issues.

Click and view more pages with Selenium in Python

Problem
I need to go into all the top user profile in this page using Selenium.
The Top users profile are located in the right of the page.
What i've done
self.driver.get(response.url)
user_list = self.driver.find_elements_by_xpath('//table[contains(#class,"W-100 Bc-c")]/tbody/tr')
for single_user in user_list:
single_user.find_element_by_xpath('.//td/a').click()
time.sleep(3)
But I get this error message:
WebDriverException: Message: unknown error: Element is not clickable
at point (865, 685). Other element would receive the click:
<div id="MouseoverMask" class="End-0 Start-0 T-0 B-0"></div>
Info
Python 2.7.10
Selenium 2.48
Pycharm
EDIT+
I try to make a print of the name and it works:
print(str( single_user.find_element_by_xpath('.//td/a').text ) )
But the click() no.

if you sure that the object you get is the right one, often the problem is:
The object is not visible
Page was not fully loaded when you try to click on the object.
So just take a look on the Wait method provided by Selenium and be sure your object is visible
In order to wait an element to be clickable :
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.ID,'someid')))
In your case, you can try to find each element with the id you got and click on it :
self.driver.get(response.url)
user_list = self.driver.find_elements_by_xpath('//table[contains(#class,"W-100 Bc-c")]/tbody/tr')
for single_user in user_list:
id = single_user.find_element_by_xpath('.//td/a').get_attribute("id")
self.driver.find_elements_by_id(id).click()
time.sleep(3)

I don't see any error at my end but after first click web elements are changed so you will not get the next web element as captured earlier in xpath. By the way try below code-
from selenium import webdriver
import time
driver = webdriver.Firefox()
driver.get('https://answers.yahoo.com/dir/index/discover?sid=396545663')
user_list = driver.find_elements_by_xpath('//table[contains(#class,"W-100 Bc-c")]/tbody/tr')
lnks = [i.find_element_by_xpath('.//td/a').get_attribute('href') for i in user_list]
for single_user in lnks:
driver.get(single_user)
time.sleep(3)