Develop Reference

Selenium not extracting info using xpath

I am trying to extract some information from the amazon website using selenium. But I am not able to scrape that information using xpath in selenium.
In the image below I want to extract the info highlighted.
This is the code I am using
try:
path = "//div[#id='desktop_buybox']//div[#class='a-box-inner']//span[#class='a-size-small')]"
seller_element = WebDriverWait(driver, 5).until(
EC.visibility_of_element_located((By.XPATH, path)))
except Exception as e:
print(e)
When I run this code, it shows that there is an error with seller_element = WebDriverWait(driver, 5).until( EC.visibility_of_element_located((By.XPATH, path))) but does not say what exception it is.
I tried looking online and found that this happens when selenium is not able to find the element in the webpage.
But I think the path I have specified is right. Please help me.
Thanks in advance
[EDIT-1]
This is the exception I am getting
Message:

//div[class='a-section a-spacing-none a-spacing-top-base']//span[class='a-size-small a-color-secondary']
XPath could be something like this. You can shorten this.
CSS selector could be and so forth.
.a-section.a-spacing-none.a-spacing-top-base
.a-size-small.a-color-secondary

I think the reason is xpath expression is not correct.
Take the following element as an example, it means the span has two class:
<span class="a-size-small a-color-secondary">
So, span[#class='a-size-small') will not work.
Instead of this, you can ues xpath as
//span[contains(#class, 'a-size-small') and contains(#class, 'a-color-secondary')]
or cssSelector as
span.a-size-small.a-color-secondary

Amazon is updating its content on the basis of the country you are living in, as I have clicked on the link provided by you, there I did not find the element you are looking for simply because the item is not sold here in India.
So in short if you are sitting in India and try to find your element, it is not there, but as you change the location to "United States". it is appearing there.
Solution - Change the location

To print the Ships from and sold by Amazon.com of an element you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and get_attribute():
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.a-section.a-spacing-none.a-spacing-top-base > span.a-size-small.a-color-secondary"))).get_attribute("innerHTML"))
Using XPATH and text attribute:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='a-section a-spacing-none a-spacing-top-base']/span[#class='a-size-small a-color-secondary']"))).text)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
Outro
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

How can I get the text in Selenium?

I want to get the text of an element in selenium. First I did this:
team1_names = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CSS_SELECTOR, ".home span"))
)
for kir in team1_names:
print(kir.text)
It didn’t work out. So I tried this:
team1_name = driver.find_elements_by_css_selector('.home span')
print(team1_name.getText())
so team1_name.text doesn’t work either.
So what's wrong with it?

You need to take care of a couple of things here:
presence_of_element_located() is the expectation for checking that an element is present on the DOM of a page. This does not necessarily mean that the element is visible.
Additionally, presence_of_element_located() returns a WebElement, not a list. Hence, iterating through for won't work.
Solution
As a solution you need to induce WebDriverWait for the visibility_of_element_located(), and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and text attribute:
print(WebDriverWait(browser, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".home span"))).text)
Using XPATH and get_attribute():
print(WebDriverWait(browser, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[#class='home']//span"))).get_attribute("innerHTML"))
Note: You have to add the following imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
Outro
Link to useful documentation:
The get_attribute() method Gets the given attribute or property of the element.
The text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

Python selenium cannot find element even with wait

I am trying to send text to an input field, but selenium is not able to find the element.
element = WebDriverWait(b, 10).until(EC.presence_of_element_located((By.XPATH, '/html/body/table/tbody/tr[1]/td/form/div/table/tbody/tr[2]/td/table[2]/tbody/tr/td[4]/table/tbody/tr/td[1]/input')))
element.send_keys("Customer Care", Keys.ENTER)
I've tried using the XPATH, the full XPATH and the ID to locate it, but it keeps giving me an error that indicates that it cannot find the element
selenium.common.exceptions.TimeoutException
A snippet of the HTML element
<input class="iceInpTxt testBox" id="headerForm:jumpto" maxlength="40" name="headerForm:jumpto" onblur="setFocus('');iceSubmitPartial(form, this, event);" onfocus="setFocus(this.id);" onkeyup="iceSubmit(form,this,event);" onmousedown="this.focus();" type="text" value="">

Element has ID, use it as locator. Check if element is inside a iframe:
wait = WebDriverWait(b, 10)
element = wait.until(EC.element_to_be_clickable((By.ID, 'headerForm:jumpto')))
element.send_keys("Customer Care", Keys.ENTER)
How to switch to iframe:
wait = WebDriverWait(b, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe_locator")))
element = wait.until(EC.element_to_be_clickable((By.ID, 'headerForm:jumpto')))
element.send_keys("Customer Care", Keys.ENTER)
# How to go back to default content
b.switch_to.default_content()

it is a good idea to check whether or not you installed and imported selenium or other necessary packages. Use pip to check your version and see if there is a bug online. Please let me know what python version you are using. It is likely the XPATH you provided was incorrect or maybe try increasing the amount of time in the 2nd parameter of WebDriverWait(1st,2nd). It would be much more helpful if you had a link to this html page so I could check your XPATH. If you'd like further help, please provide your html page.
Edit:
This is something that needs to be reproduced so that it can be checked. If you have tried the above, I am unable to help unless I see the html document. You should remove all sensitive information before sharing it. The other elements of your code seem to be correct.

If your usecase involves invoking click() or send_keys() while inducing WebDriverWait instead of presence_of_element_located() you need to use the expected_conditions as element_to_be_clickable() as follows:
So effectively, you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(b, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.iceInpTxt.testBox[id^='headerForm'][name$='jumpto']"))).send_keys("Customer Care", Keys.ENTER)
Using XPATH:
WebDriverWait(b, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='iceInpTxt testBox' and #id='headerForm:jumpto'][#name='headerForm:jumpto']"))).send_keys("Customer Care", Keys.ENTER)
References
You can find a couple of detailed discussion about the different expected_conditions in:
WebDriverWait not working as expected

InvalidElementStateException invalid element state: Element must be user-editable in order to clear it" error while sending text with Selenium Python

I have an input HTML element like this in Django
<input id="id" type="number" maxlength="50">
When I want to find and clear it
elm_input = self.wait.until(EC.presence_of_element_located((By.ID, elm_id)))
elm_input.clear()
elm_input.send_keys(value)
It's got error InvalidElementStateException
InvalidElementStateException invalid element state: Element must be user-editable in order to clear it"
We cannot send key clear because selenium know CLEAR or DELETE Keys is an Charactics Keys not an Number Keys, It's don't send Keys to element input. So how can I we fix it, I had try ActionChains but It's not working with as well

This error message...
InvalidElementStateException invalid element state: Element must be user-editable in order to clear it"
...implies that the WebDriver instance was unable to clear the existing contents of the element.
A bit more of the outerHTML of the element would have helped us to analyze the issue in a better way. However you need to take care of a couple of things as follows:
While sending a character sequence instead of using presence_of_element_located() you have to induce WebDriverWait for the element_to_be_clickable().
Ensure the Locator Strategy uniquely identifies the WebElement and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
elm_input = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#id[type='number'][maxlength='50']")))
elm_input.clear()
elm_input.send_keys("1234567890")
Using XPATH:
elm_input = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#id='id' and #type='number'][#maxlength='50']")))
elm_input.clear()
elm_input.send_keys("1234567890")
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Reference
You can find a relevant discussion in:
Invalid element state: Element must be user-editable in order to clear it error trying to click and insert a date on a dropdown-toggle using Selenium

Element is visible but cannot be found by webdriver

I'm using selenium and python for automated test and have an issue when try to click on an element in web.
I'm using find_element_by_xpath, provide the correct path (I have try on browser and it return 1 of 1).
driver = webdriver.Chrome()
driver.get('page_url')
driver.find_element_by_xpath(//button[#class="aoc-end md-button md-ink-ripple"])
Here is html:
<div class="_md md-open-menu-container md-whiteframe-z2 md-active md-clickable" id="menu_container_77" style="top: 499px; left: 866px; transform-origin: right top;" aria-hidden="false">
<md-menu-content class="agent-dropdown-menu-content new-menu-content" width="3" role="menu"><!----><md-menu-item>
<button class="aoc-end-work md-button md-ink-ripple" type="button" ng-transclude="" ng-disabled="!agent.capabilities.canSupervisorLogout" ng-click="logOutAgent(agent.userHandle)" role="menuitem">
The element should be found but actual result is that selenium.common.exceptions.ElementNotVisibleException: Message: element not visible

From the exception, it seems the element is present of the page but currently not visible.
Possible reason for invisibility of element can be (element is masked by another element, element can be in a drop - down list etc. ). If element is in drop-down list, then first open the list and then find the element.
Hope it will help.

Finding locators in web application developed in angular is quite tricky, especially when the developers don't follow any of guidelines useful from an automation perspective, like adding unique attributes for every web elements and etc...
In your case, it seems that the same thing is happening.
Following locator should work for you (by the way, you missed "" after and before locator in driver.find_element_by_xpath() method):
driver.find_element_by_xpath("//button[#ng-click='logOutAgent(agent.userHandle)']");

Since the element you are trying to click is a dynamic element, you will need to wait for it to become clickable before clicking on it. Use Expected conditions and WebDriver wait:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.XPATH, "//button[#class='aoc-end md-button md-ink-ripple']"))).click()
Also, notice the " " and ' ' quotes on the outer and inner xpath selector.