Develop Reference

I found a span on a website that is not visible and I can't scrape it! Why?

Currently I'm trying to scrape data from a website. Therefore I'm using Selenium.
Everything is working as it should. Until I realised I have to scrape a tooltiptext.
I found already different threads on stackoverflow that are providing an answer. Anyway I did not manage to solve this issue so far.
After a few hours of frustration I realised the following:
This span has nothing to do with the tooltip I guess. Because the tooltip looks like this:
There is actually a span that I can't read. I try to read it like this:
bewertung = driver.find_elements_by_xpath('//span[#class="a-icon-alt"]')
for item in bewertung:
print(item.text)
So Selenium finds this element. But unfortunatly '.text' returns nothing. Why is it always empty ?
And what for is the span from the first screenshot ? Btw. it is not displayed at the Website as well.

Since you've mentioned Selenium finds this element, I would assume you must have print the len of bewertung list
something like
print(len(bewertung))
if this list has some element in it, you could probably use innerText
bewertung = driver.find_elements_by_xpath('//span[#class="a-icon-alt"]')
for item in bewertung:
print(item.get_attribute("innerText"))
Note that, you are using find_elements which won't throw any error instead if it does not find the element it will return an empty list.
so if you use find_element instead, it would throw the exact error.
Also, I think you've xpath for the span (Which does not appear in UI, sometime they don't appear until some actions are triggered.)
You can try to use this xpath instead:
//i[#data-hook='average-stars-rating-anywhere']//span[#data-hook='acr-average-stars-rating-text']
Something like this in code:
bewertung = driver.find_elements_by_xpath("//i[#data-hook='average-stars-rating-anywhere']//span[#data-hook='acr-average-stars-rating-text']")
for item in bewertung:
print(item.text)

Element not found when using webdriver with Selenium in Python

got a problem with my code and I need your help. What I'm trying to do is the following:
1- access a website;
2- fill the registration form: name, email, password, etc.
Step 1 works; after clicking the sign up button, the form will pop up in a new tab.
Step 2; when trying to find the elements, by, id or name, I get the error "selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element"
My code is the following:
driver.find_element_by_link_text('Sign Up').click()
time.sleep(3)
driver.find_element_by_id("signup_title").send_keys(signup_tile)
driver.find_element_by_id("signup_name").send_keys(signup_name)
Attached you can find the html. Thank you in advance, much appreciated your help.
Note
In console I tried to find the element searching the id using
$x("//*[#id='signup_title']") and it was found: [input#signup_title.sc-AxirZ.kzqQJb.invalid]. Also the element is loaded.
HTML

Try using:
driver.find_element_by_xpath("<XPath>")
It seems to work better. To get the XPath of an element, find the element in the inspector, right-click it then Copy -> Copy XPath. If that doesn't work, select Copy Full XPath instead

The problem was related to the second tab; because the form opens in another tab, the window must be switched in the code.
I used the following:
driver.switch_to.window(driver.window_handles['Nr']), Nr being the index for the tab- if there are 2, the main one, and the second one-in my case with the form, the index will pe 1-counting starts from zero.

Selenium CSS_Selector looking for an element with specific text is turning out a 'NoSuchElementException'

I'm having an issue with looking for an element with a particular text attribute utilizing CSS_Selectors in Selenium. Here is the current line of code I have:
element = driver.find_element(By.CSS_SELECTOR, "li.adTypeItem[text='CLASS']")
I've had trouble using the attribute selector brackets in CSS_Selectors in the past, and clearing this up would really go a long way to better understanding how to use CSS_Selectors in the future.
Please note - i'm not looking for a element with a class, but rather the actual text that is displayed with that element.

Is this the only place on the page with the text "class" displayed? If so you can try:
driver.find_element(By.LINK_TEXT('CLASS'));
driver.find_element(By.PARTIAL_LINK_TEXT('CLASS'));

Python Splinter clicking button CSS

I'm having trouble selecting a button in my Splinter script using the find_by_css method. The documentation is sparse at best, and I haven't found a lot of good articles out there with examples.
br.find_by_css('div#edit-field-download-files-und-0 a.button.launcher').first.click()
...where br is my browser instance.
I've tried a few different ways of writing it. I'm really not sure how I'm supposed to do it because the documentation doesn't give any hard examples of the syntax.
Here's a screenshot of the element.
Sorry the screenshot kind of sucks.
Does anyone have any experience with this?

The css selector looks alright, just that i am not sure from where have you got find_by_css as a method?
How about this :-
br.find_element_by_css_selector("div#edit-field-download-files-und-0 a.button.launcher").click()
Selenium provides the following methods to locate elements in a page:
find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector
To find multiple elements (these methods will return a list):
find_elements_by_name
find_elements_by_xpath
find_elements_by_link_text
find_elements_by_partial_link_text
find_elements_by_tag_name
find_elements_by_class_name
find_elements_by_css_selector

I'm working on something similar where I'm trying to click stuff on a webpage. The documentation for find_by_css() is very poor and you need to type the css path to the element you want to click.
Say we want to go to the about tab on python.org
from splinter import Browser
from time import sleep
with Browser() as browser: #<--Create browser instance (firefox default driver)
browser.visit('http://www.python.org') #<--Visits url string
browser.find_by_css('#about > a').click()
# ^--Put css path here in quotes
sleep(5)
If your connection is good you might not get the chance to see the about tab getting clicked but you should end up on the about page.
The hard part is figuring out the css path to an element. However once you have it, the find_by_css() method looks pretty easy

I like the W3Schools reference for CSS selection parameters: http://www.w3schools.com/cssref/css_selectors.asp
As for your code... I recommend breaking this down into a few steps, at least during debug. The call to br.find_by_css('css_string') returns a list of elements. So you can grab that list and check the count.
elems = br.find_by_css('div#edit-field-download-files-und-0 a.button.launcher')
if len(elems) == 1:
elems.first.click()
If you don't check the length of the returned list and call '.first' on an empty list, you'll get an exception. If len > 1, you're probably getting things you don't expect.
Each id on a page is unique, and you can daisy-chain searches, so you can use a few different statements to make this happen:
id_elems = br.find_by_id('edit-field-download-files-und-0')
if id_elems:
id_elem = id_elems.first
a_elems = id_elem.find_by_tag("a")
for e in a_elems:
if e.has_class("button launcher"):
print('Found it!')
e.click()
This is, of course, just one of many ways to do this.
Lastly, Splinter is a wrapper around Selenium and other webdrivers. It's possible that, even after you find the element to click, the actual click won't do anything. If this happens, you can also try clicking on the wrapped Selenium object, available as e._element. So you could try e._element.click() if necessary.

Get element inside current element using xpath

I have multiple questions inside which there are more than 1 options.
After selecting the required question element as question_element
I am unable to get the first text box inside this element. I used
question_element.find_elements_by_xpath("//textarea")
but it gives me list of all the elements with tag textarea in the whole webpage. I tried
question_element.find_elements_by_xpath("/textarea")
question_element.find_elements_by_xpath("./textarea")
but they didn't give any results. How do I get the first element with tag name textarea inside the question_element

There are two variants that work for search within already found element (not within the whole page):
question_element.find_elements_by_xpath(".//textarea")

Try like this
question_element.find_elements_by_xpath("//textarea[position()=1]")