I'm new to all of this but I've learned a few things about python not long time ago, could you help me specify the correct the XPath for selenium to click?
I've tried this way, but didn't work, obviously :(
self.selenium.click("xpath=//html/body/div/div/div/div[4]/ul/li[3]/a")
If you're wandering where did i get that ugly XPath, it's from Firebug's copy XPath option.
I think that the HTML snippet is as long as hell so i couldn't do more than this:
<html>
<body>
<div id="outer_wrapper">
<div id="container">
<div id="header">
<div id="menunav">
<ul>
<li><a title="Login page" href="[dest]">Login</a></li>
<li><a title="" href="[dest]">Sitemap</a></li>
**<li><a title="" href="[dest]">Administration</a></li>**
</ul>
</div>
</div>
</div>
</div>
</body>
</html>
Below are a few example locators you could use to click the Administration link (based on your XPath and HTML snippet). The correct Selenium command is click.
link=Administration
css=a:contains(Administration)
css=#menunav a:nth-child(3)
xpath=id('menunav')/descendant::a[3]
//a[text()='Administration']
//a[contains(text(), 'Administration')]
I hope this points you in the right direction.
Related
I'm trying to web scrape a table from an iframe. In order to switch the driver to that frame I'm using driver.find_element_by_xpath, but the problem is that the path in the html code includes some namespaces that I cannot get Python to figure out using the local-name() function.
Here is the chunk of the HTML I'm using:
<xbrl:campo-captura xbrl:solo-lectura="true" xbrl:id-hecho-plantilla="ar_pros_CorporateStructure_11933a35-3932-44c0-b394-f0ebd4f722d2"
id="8a97271e-df5c-4fbe-bedf-513ea1508bf2"><div>
<div>
<i style="cursor:pointer; float:right;margin-right:-20px;" id="d9fa20ae-c55f-4344-baf5-0112a13827b6" class="i i-arrow-down-2 botonDetalleOperacionXbrl">
</i>
<div id="abrir_nota_F2a26d5a7-2934-4ff0-86df-7a8983c05e47" style="cursor:pointer;float:right;margin-right:-20px;margin-top:20px;" data-toggle="tooltip" data-placement="right" title="Abrir nota">
<i class="fa fa-external-link"></i>
</div>
</div>
<div class="campoTextBlock">
<div id="F2a26d5a7-2934-4ff0-86df-7a8983c05e47">
<div class="celdaAnchoFijo textBlockLimit div-default divTextBlockMaximo" id="divAreaTextod9fa20ae-c55f-4344-baf5-0112a13827b6" style="overflow-y:hidden">
<iframe scrolling="no" id="frame_8a97271e-df5c-4fbe-bedf-513ea1508bf2" style="width:100%;height:100%" frameborder="0"></iframe>
</div>
</div>
</div>
<div>
</div>
</div></xbrl:campo-captura>
I want to get to the "iframe" using something like:
framLogin= driver.find_element_by_xpath('//[local-name()="campo-captura"][#*[local-name()="id-
hecho-plantilla" and .="ar_pros_CorporateStructure_11933a35-3932-44c0-b394-f0ebd4f722d2"]]
/div[2]/div/div/iframe')
The message I get is
Given xpath expression ... is invalid: SyntaxError: Document.evaluate: The expression is not a legal expression.
I've already looked for more information but all I have found is not for Python.
I'm aware I could get to the iframe by using its id, but later on I want to make a loop to scrap the same tables in other URLs with the exact same format, and the iframe's id is not constant.
Your immediate syntax error can be fixed by changing
//[local-name()="campo-captura"]
to
//*[local-name()="campo-captura"]
^
This is the html code exemple:
<div aria-label="Continue" class="my-class" data-visualcompletion="ignore"></div>
<div class="div1-class">
<div class="div1-class2">
<span class="area-span" dir="auto">
<span class="text-span">Continue</span>
</span>
</div>
</div>
<div class="div2-class" data-visualcompletion="ignore"></div>
I'm trying:
continue = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CLASS_NAME, 'my-class')) )
continue.click()
but it doesn't work in any of the ways I tried.
You should check the xpaths in the browser console.
And try doing this ?
driver.findElement(By.xpath("//div[contains(#class,'my-class')]"));
I am trying to use selenium to loop through a list of properties on a web page and return the property address and auction time. I have the following python code so far and html for the web page below.
I'm able to return the links to every property in the list, but can't seen to return the values I need from the "H4" tags. I think I'm doing something wrong with getting the elements by Xpath but I can't seem to figure it out.
Any help would be greatly appreciated!
HTML:
<div data-elem-id="asset_list_content">
<a href="/details/123-memory-lane">
<div data-elm-id="asset_2352111_address" class="styles__address-container--2l39p styles__u-mr-1--3qZyj">
<h4 data-elm-id="asset_2352111_address_content_1" class="styles__asset-font-big--vQU7K">123 memory-lane</h4>
<label data-elm-id="asset_2352111_address_content_2" class="styles__asset-font-small--2JgrX">POWDER SPRINGS, GA 30127, Cobb County</label>
</div>
<div class="styles__auction-container--45DZU styles__u-ml-1--34mF_">
<h4 data-elm-id="asset_2352111_auction_date" class="styles__asset-font-big--vQU7K">Apr 04, 10:00am</h4>
</div>
</a>
<a href="/details/456-memory-lane">
<div data-elm-id="asset_8463157_address" class="styles__address-container--2l39p styles__u-mr-1--3qZyj">
<h4 data-elm-id="asset_8463157_address_content_1" class="styles__asset-font-big--vQU7K">456 memory-lane</h4>
<label data-elm-id="asset_8463157_address_content_2" class="styles__asset-font-small--2JgrX">POWDER SPRINGS, GA 30127, Cobb County</label>
</div>
<div class="styles__auction-container--45DZU styles__u-ml-1--34mF_">
<h4 data-elm-id="asset_8463157_auction_date" class="styles__asset-font-big--vQU7K">March 10, 10:00am</h4>
</div>
</a>
</div>
Python (Selenium):
propertyList = browser.find_elements_by_xpath('//div[#data-elm-id="asset_list_content"]')
for element in propertyList:
propertyLinks = element.find_elements_by_tag_name('a')
for propertyLink in propertyLinks:
propertyAddress = propertyLink.get_element_by_xpath('//h4[1]')
propertyAuctionTime = propertyLink.get_element_by_xpath('//h4[2]')
print(propertyAddress).text
print(propertyAuctionTime).text
Output:
propertyAddress = propertyLink.get_element_by_xpath('//h4[1]')
AttributeError: 'WebElement' object has no attribute 'get_element_by_xpath'
The error seems to be you are using get_element_by_xpath(), which isn't a valid method. You used find_elements_by_xpath() in your code before that moment, and to find the elements you are looking for you just need to use the method that only finds a single element: find_element_by_xpath().
Below represents the element tag from where I got the element by its CSS selector "#xwt_widget_navigation__slidemenu__SlideOutNavigationButton_1"
<div class="xwtSlideMenuLevel0 hasIcon xwtSlideOutNavigationButton xwtSlideOutNavigationButtonFocused dijitFocused" data-dojo-attach-event="ondijitclick: _onItemClick, onmouseenter: _mouseEnter, onmouseleave: _mouseLeave" id="xwt_widget_navigation__slidemenu__SlideOutNavigationButton_1" widgetid="xwt_widget_navigation__slidemenu__SlideOutNavigationButton_1">
<div class="xwtSlideOutNavigationButtonInner" tabindex="0" role="button" data-dojo-attach-point="buttonNode,focusNode" title="Topology">
<div class="xwtSlideOutNavigationButtonIcon topologyIcon" data-dojo-attach-point="iconNode">
</div>
<div class="xwtSlideOutNavigationButtonTitle" data-dojo-attach-point="titleNode">
Topology
</div>
<div class="xwtSlideOutNavigationArrow" data-dojo-attach-point="arrowNode">
<span class="xwtSlideOutNavigationArrowIcon"></span>
</div>
</div>
</div>
Problem is when I do a
topo = driver.find_element_by_css_selector("xwt_widget_navigation__slidemenu__SlideOutNavigationButton_1")
topo.click()
This returns a None, no error or any sort.Am running this on a Linux cmd line and as such using the pyvirtualdisplay. When I do a screenshot it doesn't show that anything happened but when I do a debug using the pdb.set_trace() to step through the code it works. I have looked at the examples on StackOverflow and elsewhere but I couldn't find anything similar or helpful. Can someone tell me what am doing wrong?
I think you are missing a # as your css_selector is basically using the element id. Try the following:
topo = driver.find_element_by_css_selector("#xwt_widget_navigation__slidemenu__SlideOutNavigationButton_1")
Or
topo = driver.find_element_by_id("xwt_widget_navigation__slidemenu__SlideOutNavigationButton_1")
I am trying to access the comment textbox in a generic huffington post artical. When I right click inspect element I get the following HTML code:
<div class="UFIInputContainer">
<div class="_1cb _5yk1">
<div class="_5yk2" tabindex="-2">
<div class="_5rp7">
with the line <div class="_1cb _5yk1"> highlighted.
from selenium import webdriver
driver = webdriver.Chrome()
'''
Just pretend that I put in some code to log in to facebook
so I can actually post a comment on huffington post
'''
driver.get.('http://www.huffingtonpost.com/entry/worst-suicide-squad-reviews_us_57a1e213e4b0693164c34744?')
'''
Just a random artical about a movie
'''
comment_box = driver.find_element_by_css_selector('._1cb._5yk1')
'''
since this is a compound class I think I should use find_by_css_selector
'''
When I run this though, I get the error message: "no such element found". I have tried other methods of trying to get a hold of the comment textbox but I get the same error message and I am at a lost of how to access it. I am hoping somebody can shed some light on this problem.
edit: This is a more complete HTML code:
<html lang="en" id="facebook" class="svg ">
<head>...</head>
<body dir="ltr" class="plugin chrome webkit win x1 Locale_en_US">
<div class="_li">
<div class="pluginSkinLight pluginFontHelvetica">
<div id="u_0_0">
<div data-reactroot class="_56q9">
<div class="_2pi8">
<div class="_491z clearfix">...</div>
<div spacing="medium" class="_4uyl _1zz8 _2392 clearfix" direction="left">
<div class="_ohe lfloat">...</div>
<div class>
<div class="UFIInputContainer">
<div class="_1cb _5yk1">
<div class="_5yk2" tabindex="-2">
<div class="_5rp7">
</div>
</div>
<div class="UFICommentAttachmentButtons clearfix">...</div>
<!-- react-empty: 39 -->
<div class="_4uym">...</div>
</div>
</div>
</div>
::after
You have to switch to the iframe containing the text box. Try the following approach, it should work:
Clicking the load comments button might be required first if load comment button is displayed
load_comment = driver.find_element_by_css_selector('.comment-button.js-comment-button')
load_comment.click()
driver.switch_to_frame(driver.find_element_by_css_selector('.fb_ltr.fb_iframe_widget_lift'))
comment_box = driver.find_element_by_css_selector('._1cb._5yk1')
comment_box.send_keys('Test')