I have already accessed a webpage, populated text boxes and drop downs with required parameters, and submitted (button click) this information so the webpage can perform a calculation. I am trying to access the results (value of text). Results should be a list of calculations listed least to greatest, and I only want the lowest value. I am not sure if I am having a timing issue or a CSS Selector issue.
I have tried:
e = driver.find_elements_by_css_selector("span[data-bind='calc']")
new = e[0].text
print(new)
Error: IndexError: list index out of range
I wanted to make sure the data table was being completely populated before I tried to access it's calculated elements, and I have also tried:
output_by_css = WebDriverWait(driver, 10).until(
lambda driver : driver.find_elements_by_css_selector('span.calc')
)
for output in (output_by_css):
print(output.text)
Error: raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Can someone please help me determine if this is an issue with the CSS Selector by span or if it's a timing issue, or something else I have not yet thought of? How can I fix it?
You can try applying the following selector:
driver.find_elements_by_css_selector('span[data-bind*=calc]')
Here we are getting all the span tags having data-bind attribute having "calc" inside the value.
Related
I try to use Selenium to get specific users' information (e.g., num of followers) by entering their page using ids. The thing is, though I find the needed information in the INSPECT, I cannot position it using Selenium, even with the help of ChroPath, which tells you the Xpath or CssSelector that you can use to position. It keeps saying: No such element...I'm quite confused. I'm not even trying to automatically log in or anything.
here are the codes:
from selenium import webdriver
driver = webdriver.Chrome(executable_path='E:/data mining/chromedriver.exe')
driver.get('https://twitter.com/intent/user?user_id=823730524426477568')
ele = driver.find_element_by_class_name('css-901oao css-16my406 r-poiln3 r-bcqeeo r-qvutc0').text
print(ele)
Error:
Message: no such element: Unable to locate element: {"method":"css selector","selector":".r-poiln3 r-bcqeeo r-qvutc0"}
(Session info: chrome=88.0.4324.104)
It's so strange, because everything is right on the first page, and I don't even need to scroll down to see the information, but it won't be scraped...
To fix the issue with your code, try this:
ele = driver.find_element_by_css_selector('.css-901oao.css-16my406.r-poiln3.r-bcqeeo.r-qvutc0').text
However, because of how the site is formatted, it won't get you the result you want. It appears that not only do they rotate the class name, but there isn't enough variability in how the elements are labelled to make anything but XPath a viable option for getting specific data. (Unless you want to go through a list of all the elements with the same class to find what you need). After some initial site interactions, these XPaths worked for me:
following = driver.find_element_by_xpath('//*[#id="react-root"]/div/div/div[2]/main/div/div/div/div[1]/div/div[2]/div/div/div[1]/div[2]/div[4]/div[1]/a/span[1]/span').text
followers = driver.find_element_by_xpath('//*[#id="react-root"]/div/div/div[2]/main/div/div/div/div[1]/div/div[2]/div/div/div[1]/div[2]/div[4]/div[2]/a/span[1]/span').text
Problem is, there are over 100 elements with classes 'css-901oao css-16my406 r-poiln3 r-bcqeeo r-qvutc0' on that page.
To avoid absolute xpath you could use something like this for e.g. number of followers:
Find span with descendant span with text 'Followers'. Above that, on same level (sibling) there is span with child - number of followers for text
ele = driver.find_element_by_xpath("//span[descendant::span[text()='Followers']]/preceding-sibling::span/span").text
The selectors which you are using are very much fragile. chropath gives fragile xpath which is changing in next run so script is failing. You might like to use relative xpath generated by selectorshub which is much robust.
I am trying to scrape the target website for product_links. The program should open the required URL in the browser and scrape all the links with a particular class name. But for some reason, I am getting a NoSuchElementException for this piece of code
links = driver.find_elements_by_class_name("styles__StyledTitleLink-mkgs8k-5")
for link in links:
self.driver.implicitly_wait(15)
product_links.append(link.find_element_by_css_selector('a').get_attribute('href'))
I tried printing out the text in each link with link.text in the for loop. The code is actually selecting the required elements. But for some reason is not able to extract the href URL from each link. I am not sure what I am doing wrong.
This is the entire error message
NoSuchElementException: Message: no such element: Unable to locate
element: {"method":"css selector","selector":"a"} (Session info:
chrome=83.0.4103.106)
Error seems there is no css element with 'a' so you need to try with other locators to identify elements. try with xpath=//a[contains(text(),'text of that element')]
You are looking for a class name generated by a builder, check the random string at the end of the class name, these classes won't be found in every web.
if you want to scrape them, find a different generic class or find all classes with a subset string "StyledTitleLink"
Here's how to do it with JQuery
You should try and find a different solution to your problem
New to python here. I am trying to have my script click a checkbox with a given input value. Unfortunately there is no input ID. Using find_element returns with error:
Unable to locate element
Using find_elements() returns a list and therefore I get the error:
'list' object has no attribute 'click'
This is the code I have had trouble with.
OldJob = driver.find_elements_by_xpath("//input[#value='0 ']")
OldJob.click()
Here is the html:
Any thoughts on how I can get around this?
To locate the <input> element you can use either of the following Locator Strategies as follows:
cssSelector:
driver.find_element_by_css_selector("input[name='RowNum'][value^='0'][type='checkbox']")
xpath:
driver.find_element_by_xpath("//input[#name='RowNum' and contains(#value, '0')][#type='checkbox']")
I'm trying to crawl review data on an app from a Google Play website page using Python and Selenium. What I'm trying to do here is to click the "Full Review" button to see the entire text of each review.
But I keep running into these errors each time:
ElementNotInteractableException: Message: element not interactable
or
AttributeError: 'list' object has no attribute 'click'
or
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".LkLjZd ScJHi OzU4dc "}
The element for the button is this:
<button class="LkLjZd ScJHi OzU4dc " jsaction="click:TiglPc" jsname="gxjVle">Full Review</button>
The xpath for the button is this:
//*[#id="fcxH9b"]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div/div[26]/div/div[2]/div[2]/span[1]/div/button
And this is the code I'm using for xpath:
full_review = driver.find_elements_by_xpath('//*[#id="fcxH9b"]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div/div[26]/div/div[2]/div[2]/span[1]/div/button')
full_review.click()
I can't find the button by class name, xpath or anything.
Could anyone help me figure out this problem? In addition, is it possible to find the element by jsname?
I would really appreciate if you could help.
Avoid using xpath whenever possible, it's the most brittle selector.
id > CSS > Xpath
For your button, this css selector should work:
button[jsname='gxjVle']
You'll probably need to specify the child as that probably won't be unique
Your XPath selector is a little bit flaky, you can simplify it to be something like:
//button[text()='Full Review']
You're using the wrong function, if you're looking for a single unique element you should be using WebDriver.find_element_by_xpath one, mind that element, not elements
It's better to use Explicit Wait just in case element is not immediately available in DOM (for example if you're testing application built using AJAX technology)
Assuming all above:
full_review = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Full Review']")))
I am trying to get user details from each block as given
driver.get("https://www.facebook.com/public/karim-pathan")
wait = WebDriverWait(driver, 10)
li_link = []
for s in driver.find_elements_by_class_name('clearfix'):
print s
print s.find_element_by_css_selector('_8o._8r.lfloat._ohe').get_attribute('href')
print s.find_element_by_tag_name('img').get_attribute('src')
it says:
unable to find element with css selector
any hint appreciable.
Just a mere guess based on assumption that you are not logged in. You are getting exception cause for all class clearfix, element with ._8o._8r.lfloat._ohe does not exists. So your code isn't reaching the required elements. Anyhow, if you are trying to fetch href and img source of results, you need not iterate over all clearfix cause as suggested by #leo.fcx, your css is incorrect, trying the css provided by leo, you can achieve the desired result as:
driver.get("https://www.facebook.com/public/karim-pathan")
for s in driver.find_elements_by_css_selector('._8o._8r.lfloat._ohe'): // there didn't seemed to iterate over each class of clearfix
print s.get_attribute('href')
print s.find_element_by_tag_name('img').get_attribute('src')
P.S. sorry for any syntax, never explored python binding :)
Since you are using all class-names that the element applies, adding a . to the beginning of your CSS selector should fix it.
Try this:
s.find_element_by_css_selector('._8o._8r.lfloat._ohe')
instead of:
s.find_element_by_css_selector('_8o._8r.lfloat._ohe')
Adding to what #leo.fcx pointed about the selector, wait for search results to become visible:
wait.until(EC.visibility_of_element_located((By.ID, "all_search_results")))