I am using Python & Selenium to scrap the content of a certain webpage. Currently, I have the following problem: There are multiple div-classes with the same name, but each div-class has different content. I only need the information for one particular div-class. In the following example, I would need the information in the first "show_result"-class since there is the "Important-Element" within the link text:
<div class="show_result">
<a href="?submitaction=showMoreid=77" title="Go-here">
<span class="new">Important-Element</span></a>
Other text, links, etc within the class...
</div>
<div class="show_result">
<a href="?submitaction=showMoreid=78" title="Go-here">
<span class="new">Not-Important-Element</span></a>
Other text, links, etc within the class...
</div>
<div class="show_result">
<a href="?submitaction=showMoreid=79" title="Go-here">
<span class="new">Not-Important-Element</span></a>
Other text, links, etc within the class...
</div>
With the following code I can get the "Important-Element" and its link:
driver.find_element_by_partial_link_text('Important-Element'). However, I also need the other information within the same div-class "show-result". How can I refer to the entire div-class that contains the Important-Element in the link text? driver.find_elements_by_class_name('show_result') does not work since I do not know in which of the div-classes the Important-Element is located.
Thanks,
Finn
Edit / Update: Ups, I found the solution on my own using xpath:
driver.find_element_by_xpath("//div[contains(#class, 'show_result') and contains(., 'Important-Element')]")
I know you've found an answer but I believe it's wrong since you would also select the other nodes because Important-Element is still in Non-Important-Element.
Maybe it works for your specific case since that's not really the text you're after. But here are a few more answers:
//div[#class='show_result' and starts-with(.,'Important-Element')]
//div[span[text()='Important-Element']]
//div[contains(span/text(),'Important-Element') and not(contains(span/text(),'Non'))]
There are more ways to write this...
Ups, i found the solution on my own via xpath:
driver.find_element_by_xpath("//div[contains(#class, 'show_result') and contains(., 'Important-Element')]")
Related
I need help with finding an exact element and click it with following-sibling based on specific id number and then classes and a (href or class).
Here is simplified code, the below example occurs many times just with different id:
<div class="class_1" id="1234567">
<div class="class_2">
<div class="class_3">
<div class="class_3.1">
<div class="class_3.2">
<div class="class_3.3">
<div class="class_3.3.1">
<div class="class_3.3.1.1">
<div class="class_3.3.1.2">
<div class="class_3.3.1.3">
...
How can I locate an element with id and class for example something like this and click on it:
driver.find_element(By.XPATH, 'class=class_1 and id="2222222" and class="event-media-icon live-icon icon-white').click()
The xpath you are looking for will look like the following:
//div[#class='class_1' and(#id='1234567')]//a[#data-sport='soccer']
I guess the elements between the upper div and the goal a are not important so we can omit them.
The href value looks not unique too so I preferred using data-sport attribute that can be more unique.
To give more precise answer I need to see that web page with dev tools.
This xpath should work fine too
.//div[#class='class_1' and #id='1234567']//following-sibling::a[#data-sport='soccer']
Hi I can't figure out how to get an element by CSS and matching with text. I know it can be done with Xpath but I'd rather use CSS.
<div class="button-face">
<div class="button-face-caption"> Text I want to find 1</div>
</div>
<div class="button-face">
<div class="button-face-caption"> Text I want to find 2</div>
</div>
So in by CSS would be something like...
driver.find_element_by_css('div.button-face-caption')
But how can add the text matching to that? i tried with contains and innerText and none seem to work.
As you said it's supported in xpath:
This would be a solution with an xpath using contains and text()
driver.find_element_by_xpath('//div[#class="button-face-caption" and contains(text(),"Text I want to find")]')
The xpath being:
//div[#class="button-face-caption" and contains(text(),"Text I want to find")]
For css, look here: https://sqa.stackexchange.com/q/362/34209 which should allow us to use:
div:contains('Text I want to find')
Which would lead us to
driver.find_element_by_css("div:contains('Text I want to find')")
However this comes with a BIG caveat:
:contains() is not part of the current CSS3 specification so it will
not work on all browsers, only ones that implemented it before it was
pulled. (see w3.org/TR/css3-selectors)
As workaround you can create your own function
def find_by_css(selector, text=''):
return [element for element in driver.find_elements_by_css_selector(selector) if text in element.text][0]
Then you can call it as
find_by_css('div.button-face-caption') # search only by CSS-selector
or
find_by_css('div.button-face-caption', 'Text I want to find 2') # search by CSS + text
As per the following discussions:
CSS selector :contains doesn't work with Selenium
css pseudo-class :contains() no longer allows anchors
The :contains pseudo-class isn't in the CSS Spec and is not supported by either Firefox or Chrome (even outside WebDriver).
Solution
You need to consider the ancestor of the <div class="button-face"> element and traverse down. Let us assume that both the <div class="button-face"> are with in a parent <div class="class">:
<div class="class">
<div class="button-face">
<div class="button-face-caption"> Text I want to find 1</div>
</div>
<div class="button-face">
<div class="button-face-caption"> Text I want to find 2</>
</div>
</div>
So to identify the element with text as:
Text I want to find 1:
div.class div:first-child > div.button-face-caption
Text I want to find 2:
div.class div:nth-child(2) > div.button-face-caption
References
You can find a couple of relevant detailed discussions in:
selenium.common.exceptions.InvalidSelectorException with “span:contains('string')”
Finding link using text in CSS Selector is not working
In python selenium, how to create xpath for below code which needs only id and class:
<button type="button" id="ext-gen756" class=" x-btn-text">Save</button>
And I also need to select Global ID from below drop-down without clicking it.
<div class="x-combo-list-item">Global ID</div>
My below solution is not working-
//div[#class='x-combo-list-item']/div[contains(.,'Global ID')]
I do not want to mention droplist sequence number like-
//div[#class='x-combo-list-item']/div[1]
If you want to club id and class together in your xpath try like this-
driver.find_element_by_xpath('//button[#id="ext-gen756"][#class=" x-btn-text"]');
You can also try the same using AND -
driver.find_element_by_xpath('//button[#id="ext-gen756" and #class=" x-btn-text"]');
EDITED
Your xpath seem incorrect. Use following -
driver.find_element_by_xpath('//div[#class="x-combo-list-item"][contains(.,"Global ID")]');
Just answering my own question after a long time had a look on this. The Question was posted when I was new in xpath topics.
<button type="button" id="ext-gen756" class=" x-btn-text">Save</button>
in terms of id and class:
driver.find_element_by_xpath("//button[#id='ext-gen756'][#class=' x-btn-text']")
Also sometime if Id's are dynamic and changes for every reload of the page then you may try:
driver.find_element_by_xpath("//button[#type='Save'][contains(#id,'ext-gen')][#class=' x-btn-text']")
Here I have used #type and for the #id contains option as prefix(ext-gen) usually remains the same for the dynamic ID's
Here is the HTML I'm dealing with
<a class="_54nc" href="#" role="menuitem">
<span>
<span class="_54nh">Other...</span>
</span>
</a>
I can't seem to get my XPath structured correctly to find this element with the link. There are other elements on the page with the same attributes as <a class="_54nc"> so I thought I would start with the child and then go up to the parent.
I've tried a number of variations, but I would think something like this:
crawler.get_element_by_xpath('//span[#class="_54nh"][contains(text(), "Other")]/../..')
None of the things I've tried seem to be working. Any ideas would be much appreciated.
Or, more cleaner is //*[.='Other...']/../.. and with . you are directly pointing to the parent element
In other scenario, if you want to find a tag then use css [role='menuitem'] which is a better option if role attribute is unique
how about trying this
crawler.get_element_by_xpath('//a[#class="_54nc"][./span/span[contains(text(), "other")]]')
Try this:
crawler.get_element_by_xpath('//a[#class='_54nc']//span[.='Other...']');
This will search for the element 'a' with class as "_54nc" and containing exact text/innerHTML "Other...". Furthermore, you can just edit the text "Other..." with other texts to find the respective element(s)
I want to read the amount value (24.40) from this HTML.
<div id="order-total" class="clear-fix" style="margin-bottom:20px;">
<h3 class="col-left">Order total</h3>
<h3 class="col-right" style="display: block;">
<span class="credit-total-to-order" data-total-to-order="24.40">$ 24.40</span>
credits
</h3>
</div>
xpath - /html/body/div/header/section/form/div[5]/h3[2]/span
css - html body.ui-lang-en div#slave-edit.string-v2 header#slave-edit-header.edit
section#order-form form#frm-order-translation div#order-total.clear-fix
h3.col-right span.credit-total-to-order
I know I should use find_element_by_class_name or find_element_by_css_selector.
But not sure what should be the argument.
How can I do it?
Why not select the value from the element and parse the string to get the answer you need. For example, you can split the string and disregard the dollar to return the number you need.
someString = selenium.find_element_by_css_selector(".credit-total-to-order").text
someString.split(' ')[1]
Bear in mind - this will only work for the example you have provided.
Its not necessary to use find_element_by_class_name or find_element_by_css_selector..You can achive it with xpath like this
driver.find_element_by_xpath("//span[#class='credit-total-to-order']").text
UPDATE:
As per your updated html it looks like the style makes your element hidden.Mean while I also came to notice that the value you want to get is also stored in an attribute data-total-to-order.
So you can do somthing like this :
driver.find_element_by_xpath("//span[#class='credit-total-to-order']").get_Attribute("data-total-to-order")