I've just started learning python and I'm using selenium to interact with the web pages.
I have to repeat a specific action until a certain element is present in the HTML code, so I wrote something like this:
\\\\\\\\\\\\\\\\\\\
t = driver.find_element_by_id("i3")
t.send_keys(Codice)
while driver.find_element_by_id("i3")==t:
time.sleep(5)
t.send_keys(Keys.RETURN)
\\\\\\\\\\\\\\\\\\\\\\
But it doesn't work because it doesn' stop at the end so I wanted to know what does the function driver.find_element_by_id("i3") returns when it doesn't find the element.
Also, is there a more appropriate cycle for doing this?
Thank you and sorry for the bad english
Your code is unclear, what I can say here is the while loop to validate the element is presented can be done by
while driver.find_elements_by_id("i3")
Since driver.find_elements_by_id return list of web elements. In case elements found list in not empty and true, in case no elements found the list is empty and is false.
Related
Currently I'm trying to scrape data from a website. Therefore I'm using Selenium.
Everything is working as it should. Until I realised I have to scrape a tooltiptext.
I found already different threads on stackoverflow that are providing an answer. Anyway I did not manage to solve this issue so far.
After a few hours of frustration I realised the following:
This span has nothing to do with the tooltip I guess. Because the tooltip looks like this:
There is actually a span that I can't read. I try to read it like this:
bewertung = driver.find_elements_by_xpath('//span[#class="a-icon-alt"]')
for item in bewertung:
print(item.text)
So Selenium finds this element. But unfortunatly '.text' returns nothing. Why is it always empty ?
And what for is the span from the first screenshot ? Btw. it is not displayed at the Website as well.
Since you've mentioned Selenium finds this element, I would assume you must have print the len of bewertung list
something like
print(len(bewertung))
if this list has some element in it, you could probably use innerText
bewertung = driver.find_elements_by_xpath('//span[#class="a-icon-alt"]')
for item in bewertung:
print(item.get_attribute("innerText"))
Note that, you are using find_elements which won't throw any error instead if it does not find the element it will return an empty list.
so if you use find_element instead, it would throw the exact error.
Also, I think you've xpath for the span (Which does not appear in UI, sometime they don't appear until some actions are triggered.)
You can try to use this xpath instead:
//i[#data-hook='average-stars-rating-anywhere']//span[#data-hook='acr-average-stars-rating-text']
Something like this in code:
bewertung = driver.find_elements_by_xpath("//i[#data-hook='average-stars-rating-anywhere']//span[#data-hook='acr-average-stars-rating-text']")
for item in bewertung:
print(item.text)
I have a page with a list of cards with information.
The XPATHS of each cards are:
self.automatic_payments_cards_list = (By.XPATH , '//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info')
I'm trying to get the text of a specific elements for every card in the page.
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[1]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[2]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[3]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]
I know that with this code i get all the text on each card
for i in range(len(self.driver.find_elements(*self.automatic_payments_cards_list))):
print(self.driver.find_element(*self.automatic_payments_cards_list)[i].text)
But i don't want to get all the text on the cards, only the text on this specifics XPATHS
//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info[**X**]**/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]**
Can you guys guide me in finding a solution to this?
The best way to actually achieve this is by using find element_by_xpath on the element.
all_card_els = driver.find_elements_by_xpath('//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info')
for card_el in all_card_els:
specific_el_within_card = card_el.find_element_by_xpath('.//lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]')
The . at the starting of the xpath is essential to make sure that the search is within the selected el, without the . you will always end up getting the first el which matches this xpath on the page. You can now use specific_el_within_card however way you like inside the loop, or append it to an external list.
PS: You can access the text via specific_el_within_card.text() as you mentioned you wanted to extract info for each card.
It is simple.:-) Transfer your xpath as dynamical string and pass it, like you do in a loop for e.g.
parent_locator_String=string1+ iterator + string2
"//*[#id="page-inner"]/div/div/main/lseg-gateway-billing-payment-line-info["+i+"]/lseg-card/div/lseg-card-container/ng-transclude/div/div[4]/div/div[3]"
I'm using selenium, with find_element_by_path method to do some web scraping, I have some problem to get a path which change through pages, I know how the path is written, but one of the string within the path change through my loop, I would like to know how can I use regex to solve it.
I have this code for one of the page but when I go through all pages the string "NUMBER" below changes:
browser.find_element_by_xpath(re.compile('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[NUMBER]/div').click()
I want to know if it was possible to use regex in order to say that it has to click whatever the "NUMBER" as long as the rest of the path is the same so I tried this but I'm not sure about the syntax and how to use regex here:
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div['). + re.compile("^[1-9]\d*$") + ']/div').click()
browser.find_element_by_xpath(re.compile('^//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[')).click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[1]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[9]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[4]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[10]/div').click()
browser.find_element_by_xpath('//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div[6]/div').click()
the path evolves more or less in this manner (randomly) but not gradually one by one.
How do I solve this problem?
Welcome to SO.
If you are trying to pass the NUMBER as part of xpath in your loop then you can do the below.
If NUMBER in an integer:
browser.find_element_by_xpath("//*#id='exhibDetail:exhib']/section[3]/div[2]/div/div[2]/div/div/div[%i]/div"%(NUMBER)).click()
If NUMBER is a string
browser.find_element_by_xpath("//*#id='exhibDetail:exhib']/section[3]/div[2]/div/div[2]/div/div/div[%s]/div"%(NUMBER)).click()
I want to know if it was possible to use regex in order to say that it
has to click whatever the "NUMBER" as long as the rest of the path is
the same
If you want to select those div elements disregarding their position (that is what the predicates [1], [2], etc. are testing) then just don't use the predicates at all:
//*[#id="exhibDetail:exhib"]/section[3]/div[2]/div/div[2]/div/div/div/div
the code:
I want to change the value of this textbox, but I can't do it.
Here is my code for it:
driver.find_element_by_name("inps_19_1").send_keys("value", "my value"), not working,
driver.find_element_by_css_selector("input[name='inps_19_1']").send_keys("whatever") not working.
Any advice?
I can't see the full HTML, but a problem I've run into a number of times was either elements being contained iframes or shadowroots that you'd have to switch to before the driver can find it. iframes are easy enough to switch to
driver.switch_to_frame("frameName")
Then you can find the element after.
If the element is contained within a shadowroot, this answer shows how to expand them https://stackoverflow.com/a/37253205/10007528
Hi I am trying to locate an element by its class name and the text that it contains
<div class="fc-day-number">15</div>
there are a bunch of fc-day-number on the page with different values, I need the one with for example 15.
I do
driver.find_element_by_class_name("fc-day-content")
but I also need it to be equal to 15 and I am stuck here, please help.
You can use xpath:
driver.find_element_by_xpath("//div[#class='fc-day-content' and text()='15']")
fc_day_contents = driver.find_elements_by_class_name("fc-day-content")
the_one_you_want = [x for x in fc_day_contents if "15" == x.text][0]
first line puts all elements with class name "fc-day-content" in a list ( also notice how its elementSSSSSSSSS with an S, this returns a list of all elements by_class_name, by_name, by_id or wahtever)
second line, goes through each element and looks to see if it has the text "15" as its text, and returns it as a (probably smaller) list
the [0] at the end of it, returns the first item in the list (you can remove it, if you want a list of all the ones that are "15" )
For things like this, prefer to use JavaScript:
els = driver.execute_script("""
return Array.prototype.slice.call(document.getElementsByClassName("fc-day-content"))
.filter(function (x) { return x.textContent === "15"; });
""")
assert len(els) == 1
el = els[0]
What this does is get all elements that have the class fc-day-content as the class. (By the way, your question uses fc-day-content and fc-day-number. Unclear which one you're really looking for but it does not matter in the grand scheme of things.) The call to Array.prototype.slice creates an array from the return value of getElementsByClassName because this method returns an HTMLCollection and thus filter is not available on it. Once we have the array, run a filter to narrow it to the elements that have 15 for text. This array of elements is returned by the JavaScript code. The assert is to make sure we don't unwittingly get more than one element, which would be a surprise. And then the element is extracted from the list.
(If you care about IE compatibility, textContent is not available before IE 9. If you need support for IE 8 or earlier, you might be able to get by with innerText or innerHTML or you could check that the element holds a single text node with value 15.)
I prefer not to do it like TehTris does (find_elements_by_class_name plus a Python loop to find the one with the text) because that method takes 1 round-trip between Selenium client and Selenium server to get all the elements of class fc-day-content plus 1 round-trip per element that was found. So if you have 15 elements on your page with the class fc-day-content, that's 16 round-trips. If you run a test through Browser Stack or Sauce Labs, that's going to slow things down considerably.
And I prefer to avoid an XPath expression with #class='fc-day-content' because as soon as you add a new class to your element, this expression breaks. Maybe the element you care about has just one CSS class now but applications change. You could use XPath's contains() function but then you run into other complications, and once you take care of everything, it becomes a bit unwieldy. See this answer for how to use it robustly.