Develop Reference

Selenium How to Find Xpath based on Text conditions

I have been learning selenium and Implementing It.I tried uploading files from the bot. During this I came to know the position(upload button content) keeps on changing on each reload. I managed to trace out option for .jpg file but I cannot track .pdf file uploading. Much detail on my code below.
<div _ngcontent-kkx-c10="" class="form-group document-upload" xpath="1">
<div _ngcontent-kkx-c10="" class="document-upload-info">
<span _ngcontent-kkx-c10="" class="document-type">
KYC Form <!----><span _ngcontent-kkx-c10="" style="margin-left:5px;color:red;">* </span>
</span>
</div>
<!----><!---->
<label _ngcontent-kkx-c10="" class="btn btn-outline upload-button" style="border: 1px solid transparent; border-radius: 0.25rem; border-color: #0078D7; color: #0078D7 !important;">
Upload <input _ngcontent-kkx-c10="" hidden="" multiple="" type="file" accept=".pdf">
</label>
<!----><!----><!---->
</div>
<div _ngcontent-kkx-c10="" class="form-group document-upload" xpath="1">
<div _ngcontent-kkx-c10="" class="document-upload-info">
<span _ngcontent-kkx-c10="" class="document-type">
Photo Form <!----><span _ngcontent-kkx-c10="" style="margin-left:5px;color:red;">* </span>
</span>
</div>
<!----><!---->
<label _ngcontent-kkx-c10="" class="btn btn-outline upload-button" style="border: 1px solid transparent; border-radius: 0.25rem; border-color: #0078D7; color: #0078D7 !important;">
Upload <input _ngcontent-kkx-c10="" hidden="" multiple="" type="file" accept=".jpg">
</label>
<!----><!----><!---->
</div>
<div _ngcontent-kkx-c10="" class="form-group document-upload" xpath="1">
<div _ngcontent-kkx-c10="" class="document-upload-info">
<span _ngcontent-kkx-c10="" class="document-type">
Citizenship Form <!----><span _ngcontent-kkx-c10="" style="margin-left:5px;color:red;">* </span>
</span>
</div>
<!----><!---->
<label _ngcontent-kkx-c10="" class="btn btn-outline upload-button" style="border: 1px solid transparent; border-radius: 0.25rem; border-color: #0078D7; color: #0078D7 !important;">
Upload <input _ngcontent-kkx-c10="" hidden="" multiple="" type="file" accept=".pdf">
</label>
<!----><!----><!---->
</div>
I did following way to upload image.
photo_filepath_input_box = driver.find_element_by_xpath("//input[#accept='.jpg']").send_keys(
'/home/daniel/Desktop/website.jpg')
Here during bot loading the above three Forms could appear in any position i.e Kyc form could arrive at the end (third position) or second and similar for rest. So I want to know If I could check condition like if KYC form text is presenet then i need to click the xpath right below it i.e Label Any hint on this ?
Here is second file upload issue.
(+) button click code
WebDriverWait(self.driver,5).until(EC.presence_of_element_located((By.XPATH,"//span[contains(text(),'Citizenship Certificates')]/../..//i[#class='fa fa-plus ' ]"))).click()
second upload citizen ship code.
self.driver.find_element_by_xpath("((//span[contains(text(),'Citizenship Certificates ')]/../..//input[#type='file' and(#accept='.pdf')])[2]").send_keys('/home/navaraj/Desktop/{}'.format(row[75]))

To upload files by Selenium the better approach is to send the file path directly to the input element, not by clicking elements.
Here you have 2 forms containing upload .pdf files inputs and single input for uploading .jpg files.
So, to upload the .pdf file in KYC form you can do this: .
Let's say your file is located on the disk in "C://Downloads//kyc_pdf_file.pdf" you can do this:
kyc_pdf_file_path = "C://Downloads//kyc_pdf_file.pdf"
driver.find_element_by_xpath("//div[.//span[#class='document-type' and (contains(.,'KYC'))]]//input[#type='file' and(#accept='.pdf')]").send_keys(kyc_pdf_file_path)
Uploading file to Citizenship form:
citizenship_pdf_file_path = "C://Downloads//citizenship_pdf_file.pdf"
driver.find_element_by_xpath("//div[.//span[#class='document-type' and (contains(.,'Citizenship'))]]//input[#type='file' and(#accept='.pdf')]").send_keys(citizenship_pdf_file_path)
While uploading ".jpg" will be similarly:
jpg_file_path = "C://Downloads//jpg_file.jpg"
driver.find_element_by_xpath("//input[#type='file' and(#accept='.jpg')]").send_keys(jpg_file_path)
UPD
The XPath for the Citizenship Certificates pdf upload is
//span[contains(text(),'Citizenship Certificates ')]/../..//input[#type='file' and(#accept='.pdf')]
And for the KYC Form is
//span[contains(text(),'KYC Form')]/../..//input[#type='file' and(#accept='.pdf')]
Here is second file upload issue.
(+) button click code
WebDriverWait(self.driver,5).until(EC.presence_of_element_located((By.XPATH,"//span[contains(text(),'Citizenship Certificates')]/../..//i[#class='fa fa-plus ' ]"))).click()
second upload citizen ship code.
self.driver.find_element_by_xpath("((//span[contains(text(),'Citizenship Certificates ')]/../..//input[#type='file' and(#accept='.pdf')])[2]").send_keys('/home/navaraj/Desktop/{}'.format(row[75]))

How to find selector for "new" button? Do I need to use Switch frames?

<div id="crmMasthead" tabindex="-1">
<div id="crmTopBar" class="ms-crm-TopBarContainer ms-crm-TopBarContainerGlobal newNavBarMode">
<div id="crmAppMessageBar" class="crmAppMessageBar" style="display: none; height: 0px;">
<div id="crmRibbonManager" currentribbonelement="commandContainer15" style="height: 62px; display: block; visibility: visible;">
<div id="commandContainer15" style="display: inline;">
<ul class="ms-crm-CommandBar-Menu" role="application">
<li id="ewrb_importfile|NoRelationship|HomePageGrid|Mscrm.HomepageGrid.ewrb_importfile.NewRecord" class="ms-crm-CommandBarItem ms-crm-CommandBar-Menu ms-crm-CommandBar-Button" tabindex="-1" title="New Create a new Import File record." command="ewrb_importfile|NoRelationship|HomePageGrid|Mscrm.NewRecordFromGrid" style="white-space: pre-line; display: inline-block;">
<span class="ms-crm-CommandBar-Button ms-crm-Menu-Label-Hovered" tabindex="-1" style="max-width:200px">
<a class="ms-crm-Menu-Label" tabindex="0" onclick="return false">
<img class="ms-crm-ImageStrip-New_16 ms-crm-commandbar-image16by16" tabindex="-1" src="/_imgs/imagestrips/transparent_spacer.gif" style="vertical-align:top"/>
<span class="ms-crm-CommandBar-Menu" tabindex="-1" style="max-width:150px" command="ewrb_importfile|NoRelationship|HomePageGrid|Mscrm.NewRecordFromGrid"> New </span>
<div class="ms-crm-div-NotVisible"> Create a new Import File record. </div>
</a>
</span>
</li>
The Xpath Shows me like this:
.//*[#id='ewrb_importfile|NoRelationship|HomePageGrid|Mscrm.HomepageGrid.ewrb_importfile.NewRecord']/span/a
and If i use this, Selenium doesn't click the button

First try to remove "."(dot) from your xpath and check it if it works.
Secondly, try to write the xpath yourself. For this a node, try this one:
//a[#class="ms-crm-Menu-Label"]
You should check it if the part of html that you share is inside an iframe node or not. Otherwise, you should share more. With the current part that you shared, it is not possible to say that if it is inside an iframe or not.
Also, it can be a good idea to check the visibility of the button. The last thing: do you receive any error message. If yes, share it.

python + selenium button in div overlay clickable in testing but not when automated

<div class="t-window-content t-content" style="overflow: auto; width: 400px; height: 389.4px;">
<div id="frmSchTarget">
<form action="/Search/SearchTypePreName"
method="post" onclick="Sys.Mvc.AsyncForm.handleClick(this, new Sys.UI.DomEvent(event));"
onsubmit="Sys.Mvc.AsyncForm.handleSubmit(this, new Sys.UI.DomEvent(event), { insertionMode: Sys.Mvc.InsertionMode.replace, onBegin: Function.createDelegate(this, PreNameListWindow), onSuccess: Function.createDelegate(this, Form_OnSuccess) });"><input type="hidden" id="NameList"
name="NameList">
<div style="text-align: left; padding-left: 10px">
<span style="margin-right: 20px;">
<button id="TreeViewButtonCollapse" name="TreeViewButtonCollapse" onclick="ExpandCollapseTreeView('NameListTreeView', false); return false;" title="Collapse All">
<img src="/Content/Images/toggle_up.png">
</button>
<button id="TreeViewButtonExpand" name="TreeViewButtonExpand" onclick="ExpandCollapseTreeView('NameListTreeView', true); return false;" title="Expand All">
<img src="/Content/Images/toggle_down.png">
</button>
</span>
</div>
<div id="divSelectNamesArea" style="height: 325px; overflow: auto;">
"i removed a very large list from here"
</div>
<hr class="PopupLineStyle">
<div style="text-align: left; padding-left: 10px">
<span style="margin-right: 20px;"><input type="checkbox" id="Checkbox1" name="SelectAllPrenamesToggle" onclick="SelectAllOrNoneInTreeView('NameListTreeView', this.checked);">
All / None</span>
<input type="submit" class="t-button" value="Done"
onclick="GetNameListString('NameListTreeView');">
</div>
</form>
</div>
</div>
The above code has basically been the bane of my existence for about a week now. I am trying to run a webscraper and upon submitting a request a div overlay appears on the page. The code snippet above shows how the 'Done' button used to get rid of it is implemented.
In testing I am able to use
buttons = driver.find_elements_by_xpath('//input[contains(#class,"t-button")]')
to find all buttons present and then it is button [5], however once I automate this does not work.
I have also tried an implicit wait with the following xpath:
//*[#id="frmSchTarget"]/form/div[3]/input
This also is a no-go. Please help, I have been trying just about everything an searched through stackoverflow for over a week already. I just can't figure it out.

// wait 15s before click button for debug purpose, in case click button before
// it be rendered out on page
time.sleep(15)
button_done = driver.find_element_by_css_selector("#frmSchTarget input[value='Done']")
button_done.click()
if above code work, remove the time.sleep() and run again, if failed, it means
you need wait some moment before click it, please use explicit wait replace time.sleep()

Python 3 - Selenium - scraping data from nested divs

I'm reasonably new to python and I'm trying to check which div class appears first on a page. I've done this with table rows but I can't seem to wrap my head around how to do this with divs.
What I'm trying to determine is whether the latest update is an email sent <div class="EMAIL SENT"> or a notes added <div class="Notes">. The most recent item will appear first from the top, but other actions may have taken place since then, for example, <div class="Updated">
I've not managed to write any code to do this or event get close, but in my head I imagine it to work like this.
for sub_div_classes in browser.find_element_by_class_name('cb'):
classname = ~check name of sub_div_class
if classname = "EMAIL SENT":
class_info = browser.find_element_by_class_name('plus_header_Additional_info').text
print(class_info) ¬output: EMAIL SENT :Email sent on 20-03-2016 00:22:09 by [REDACTED]
trigger_1()
if classname = "Notes":
trigger_2()
~move on to next div class in list
Below is the page code I'm trying to work with. I'd be really appreciative of any advice or assistance anyone can provide.
<div class="cb" style="margin:5px 0 0 0;">
<div class="Updated">
<div class="plus_header_Additional_info">Updated :Incident Updated on 20-03-2016 00:22:52 by User = [REDACTED]
<img src="images/minus.png" style="float:right;">
</div>
<div class="plus_content" style="display: block;" id="contentDivImg2_0">
<div>
Assigned to STRIKE1,
by User = [REDACTED].
</div>
<br>
</div>
</div>
<div class="Updated">
<div class="plus_header_Additional_info">Updated :PEND CLIENT STRIKE - 1 added on 20-03-2016 00:22:36 by [REDACTED].
<img src="images/minus.png" style="float:right;">
</div>
<div class="plus_content" style="display: block;" id="contentDivImg2_1">
<div>
</div>
<br>
</div>
</div>
<div class="EMAIL SENT">
<div class="plus_header_Additional_info">EMAIL SENT :Email sent on 20-03-2016 00:22:09 by [REDACTED]
<img src="images/minus.png" style="float:right;">
</div>
<div class="plus_content" style="display: block;" id="contentDivImg2_2">
<div>
To :- [NAME]#[DOMAIN].CO.UK Subject: Ticket - [IN-000999999] Description : Dear User,
[REDACTED]
</div>
<br>
</div>
</div>
<div class="Updated">
<div class="plus_header_Additional_info">Updated :Incident Updated on 12-03-2016 10:56:15 by User = [REDACTED]
<img src="images/minus.png" style="float:right;">
</div>
<div class="plus_content" style="display: block;" id="contentDivImg2_3">
<div>
Status:- PROGRESSING changed to PEND CLIENT,
Assigned to SOFTWARE DEPLOYED,
by User = [REDACTED].
</div>
<br>
</div>
</div>
<div class="Notes">
<div class="plus_header_Additional_info">Notes :Notes Added on 12-03-2016 10:55:53 by [REDACTED].
<img src="images/minus.png" style="float:right;">
</div>
<div class="plus_content" style="display: block;" id="contentDivImg2_4">
<div>
<textarea id="notes4" name="notes1" cols="" class="emailForm_input1" style="width: 97%; overflow: hidden; word-wrap: break-word; resize: horizontal; height: 237px;" readonly="readonly">Hello,
[REDACTED]
</textarea>
</div>
<br>
</div>
</div>
</div>

Use an or with an xpath:
.xpath("//div[#class='Notes' or #class='EMAIL SENT']")[0]
If Notes comes first you will get Notes and vice versa.
If we change a bit of your html snippet like below, adding some text to <div class="EMAIL SENT">in email and changing a later tag class to <div class="Notes">in notes:
We can see using lxml how it works:
In [13]: from lxml.etree import fromstring, HTMLParser
In [14]: xml = fromstring(html, HTMLParser())
In [15]: xml.xpath("//div[#class='Notes' or #class='EMAIL SENT']")
Out[15]: [<Element div at 0x7f96598d4ea8>, <Element div at 0x7f96598d4ef0>]
In [16]: xml.xpath("//div[#class='Notes' or #class='EMAIL SENT']")[0].text
Out[16]: 'in email\n '
In [17]: xml.xpath("//div[#class='Notes' or #class='EMAIL SENT']")[1].text
Out[17]: 'in notes\n
So with selenium you want to just find the element by xpath.

How to select undisplayed options from dropdown check list in Selenium using Python

I am trying to write a Selenium driver to test a web page that uses drop down (combo) check lists. The following code exhibits the problem.
#!/usr/bin/python
from selenium import webdriver
from selenium.webdriver.support.ui import Select
driver = webdriver.Firefox()
driver.get("http://dropdown-check-list.googlecode.com/svn/trunk/doc/ddcl-tests.html")
selector = driver.find_element_by_id("s1")
allOptions = selector.find_elements_by_tag_name("option")
for option in allOptions:
print "Value is", option.get_attribute("value")
option.click()
When I run it, I get the following output:
Value is Low
Traceback (most recent call last):
File "./ddcl-test.py", line 24, in <module>
option.click()
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 51, in click
self._execute(Command.CLICK_ELEMENT)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 225, in _execute
return self._parent.execute(command, params)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 160, in execute
self.error_handler.check_response(response)
File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 149, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: u'Element is not currently visible and so may not be interacted with' ; Stacktrace: Method fxdriver.preconditions.visible threw an error in file:///var/folders/d4/qbgb29wx7z7fpr15t___x24h0000gn/T/tmpBzUUcu/extensions/fxdriver#googlecode.com/components/command_processor.js
It can't click on the element(s) because it's not displayed.
How can I resolve this? Or is this an untestable case in Selenium?

The problem is that the <SELECT> you're trying to access is deliberately hidden by jQuery:
<select id="s1" class="s1" tabindex="8" multiple="" style="display: none;">
<option>Low</option>
<option>Medium</option>
<option>High</option>
</select>
WebDriver won't click on a hidden element. Period. It's deliberate, because an end user can't click on it either. WebDriver doesn't want to let you do things humans can't do.
Instead, you have to interact with the browser the same way a human would: by clicking on whatever element jQuery exposes to the human. For this example, the human UI is:
<span id="ddcl-s1" class="ui-dropdownchecklist ui-dropdownchecklist-selector-wrapper ui-widget" style="display: inline-block; cursor: default; overflow: hidden;">
<span class="ui-dropdownchecklist-selector ui-state-default" style="display: inline-block; overflow: hidden; white-space: nowrap; width: 85px;" tabindex="8">
<span class="ui-dropdownchecklist-text" style="display: inline-block; white-space: nowrap; overflow: hidden; width: 81px;" title=" ">
</span>
</span>
</span>
<div id="ddcl-s1-ddw" class="ui-dropdownchecklist ui-dropdownchecklist-dropcontainer-wrapper ui-widget" style="position: absolute; left: -33000px; top: -33000px; height: 74px; width: 91px;">
<div class="ui-dropdownchecklist-dropcontainer ui-widget-content" style="overflow-y: auto; height: 74px;">
<div class="ui-dropdownchecklist-item ui-state-default" style="white-space: nowrap;">
<input id="ddcl-s1-i0" class="active" type="checkbox" tabindex="8" disabled="" index="0" value="Low">
<label class="ui-dropdownchecklist-text" for="ddcl-s1-i0" style="cursor: default;">Low</label>
</div>
<div class="ui-dropdownchecklist-item ui-state-default" style="white-space: nowrap;">
<input id="ddcl-s1-i1" class="active" type="checkbox" tabindex="8" disabled="" index="1" value="Medium">
<label class="ui-dropdownchecklist-text" for="ddcl-s1-i1" style="cursor: default;">Medium</label>
</div>
<div class="ui-dropdownchecklist-item ui-state-default" style="white-space: nowrap;">
<input id="ddcl-s1-i2" class="active" type="checkbox" tabindex="8" disabled="" index="2" value="High">
<label class="ui-dropdownchecklist-text" for="ddcl-s1-i2" style="cursor: default;">High</label>
</div>
</div>
</div>
So it looks like the thing to interact with is one of the <input id="ddcl-s1-i*" ...> elements, but it's really not easy to be sure.
This is why some of us think JavaScript frameworks that reconstruct existing HTML capabilities from spans and divs are a Really Bad Idea.

I've tried endless times to click on elements that are not visible and it's just not doable with selenium, or at least I haven't found a way yet. It doesn't look like you're trying to click on the drop down button first so I would suggest to click on the drop down menu button first to expose the menu and I think that should get you past your issue.