Develop Reference

Web element not found but appears when inspected manually

I am unable to access this menu in selenium the web element doesn't appear in inspector until manually done
<a id="cke_3275" class="cke_menubutton cke_menubutton__table cke_menubutton_off cke_menubutton__table" href="javascript:void('Table Properties')" title="Table Properties" tabindex="-1" _cke_focus="1" hidefocus="true" role="menuitem" aria-haspopup="false" aria-disabled="false" onmouseover="CKEDITOR.tools.callFunction(666,5);" onmouseout="CKEDITOR.tools.callFunction(667,5);" onclick="CKEDITOR.tools.callFunction(668,5); return false;">
<span class="cke_menubutton_inner">
<span class="cke_menubutton_icon">
<span class="cke_button_icon cke_button__table_icon" style="background-image:url(https://lms.testbook.com/vendor/ckeditor/plugins/icons.png?t=E6FD);background-position:0 -1896px;background-size:auto;">
</span>
</span>
<span class="cke_menubutton_label">
Table Properties
</span>
</span>
</a>
I tried accessing parent, click and actions.perform() nothing seems to work.
When i hover over the menu contents i see javascript:void('contentname'), i pasted this in the inspector and found the web element.

iframe=WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "/html/body/iframe")))
driver.switch_to.frame(iframe)
driver.find_element_by_tagname("a")
switch to iframe and then add rest of the code
driver.switch_to.default_content()
you need to switch to default content after you are done with the element and want to interact with an element outside the iframe

Why can't I select this? - Selenium

The following thing is embedded into a webpage. It is not part of the actual site:
What I've been trying to do is select the 'previous meetings' tab, using Selenium in Python.
These are the elements for both buttons (upcoming meetings and previous):
<div role="tab" aria-disabled="false" aria-selected="true" class="ant-tabs-tab-active ant-tabs-tab" tabindex="0" aria-label="Upcoming Meetings">Upcoming Meetings</div>
::before
Upcoming Meetings
</div>
<div role="tab" aria-disabled="false" aria-selected="false" class=" ant-tabs-tab" aria-label="Previous Meetings">Previous Meetings</div>
::before
Previous Meetings
</div>
I've tried driver.find_element_by_xpath('//*[#class=" ant-tabs-tab"]').click() but that didn't detect the element.
When I view the actual source code of the webpage, these Zoom elements didn't show up, so I think that's why Selenium doesn't recognise it either.
It doesn't have an ID or anything else, so I can't try them. Is there anything I can do? (e.g. Just click a location on the screen)
Or am I doing something wrong?
It seems to be an iframe, starting with
<iframe src="about:blank" name="tool_content" id="tool_content" class="tool_launch" allowfullscreen="allowfullscreen" webkitallowfullscreen="true" mozallowfullscreen="true" tabindex="0" title="Tool Content" style="height:100%;width:100%;" allow="geolocation *; microphone *; camera *; midi *; encrypted-media *; autoplay *" data-lti-launch="true"></iframe>
If that helps.
Update:
I tried using an extension for Firefox that gave you the exact xpath of an element:
driver.find_element_by_xpath('/html/body/div[1]/div/div/div[2]/div/div/div[1]/div[1]/div/div/div/div[1]/div[2]').click()
And this still didn't work - how come??

Due to having an iframe just switch to it
driver.switch_to.frame("tool_content")

Try using
driver.switch_to.frame(driver.find_element_by_class_name("tool_launch"))

Double click works in Firefox but not chrome (Python/Selenium)

I have an element that requires a double_click().perform() action. It works great in Firefox, but doesn't in Chrome. A single click displays a menu, but a double click takes you to a directory. Here is my python/selenium step:
(webdriver.ActionChains(self.browser)).double_click(self.browser.find_element_by_id('nav-link-shopall')).perform()
The error message returns that the text I'm wanting to verify exists does not exist. Which is true if the double_click().perform() doesn't do its job.
I'm using Python 2.7.8 and Selenium 2.45.0.
I'm using Amazon.com as an example because the behavior is the same as the proprietary code I am testing. So here is the HTML code I'm attempting to click:
<div class="nav-left">
<div id="nav-shop">
<a href="/gp/site-directory/ref=nav_shopall_btn" class="nav-a nav-a-2" data-nav-tabindex="15" id="nav-link-shopall" tabindex="1">
<span class="nav-line-1">Shop by</span>
<span class="nav-line-2">"Department"
<span class="nav-icon nav-arrow" style="visibility:visible;"></span>
</span>
</a>
</div>
</div>

Try this:
variable = self.browser.find_element_by_id('nav-link-shopall')
actions = ActionChains(driver)
actions.move_to_element(variable)
actions.double_click(variable)
actions.perform()
You should use move to element.

Using scrapy to get the "Next Page" data

I need to grab a commodity website's review data, but it's user data is paged .The comments per page are 10 strips , and there are about 100 pages. How can I crawl all of them out?
My intention is to use the yield and Request method to crawl the "Next Page" link, and then using the Xpath to extract data. But I can't jump to the next page to extract the data.
Here is the Html code about the "Next Page" link:
<div class="xs-pagebar clearfix">
<div class="Pagecon">
<div class="Pagenum">
<a class="pre-page pre-disable">
<a class="pre-page pre-disable">
<span class="curpage">1</span>
2
3
<span class="elli">...</span>
Next Page
Final Page
</div>
</div>
</div>
What does href="#" exactly mean?

Unfortunately you will not be able to do this with scrapy. href="#" is an anchor link that just links nowhere (to make this look like a link). What really happens is the javascript onclick handler that is executed. You will need to have a method of executing the javascript to do this for your use case. You might want to look into Splinter to do this.

python, locating and clicking a specific button with selenium

Using python and selenium I need to locate and click a specific button on a webpage. While under normal circumstances this can be done with the commands
next = driver.find_element_by_css_selector(".next")
next.click()
that won't work in this case due to the webpages coding. Where both the button leading to the previous page and the one leading to the next share the same class name. Thus this code will only go back and forth between the first and second page
As far as I can tell the only way to tell the two buttons apart, is that the button leading to the previous page lies within
<li class="previous">
#button
</li>
and the one that leads to the next page lies within
<li class="next">
#button
</li>
Is there some way of having selenium only select and click the button that lies in the "next" li class?
The complete button code:
the previous button:
<li class="previous">
<a class="next" rel="nofollow" onclick="qc.pA('nrForm', 'f76', 'QClickEvent', '1', 'f28'); return false;" href="">
Previous
</a>
</li>
the next button:
<li class="next">
<a class="next" rel="nofollow" onclick="qc.pA('nrForm', 'f76', 'QClickEvent', '3', 'f28'); return false;" href="">
Next
</a>
</li>

I think you should use .find_element_by_xpath().
For example:
next = driver.find_element_by_xpath("//li[#class='next']/a")
prev = driver.find_element_by_xpath("//li[#class='previous']/a")
or:
next = driver.find_element_by_xpath("//a[text()='Next']")
prev = driver.find_element_by_xpath("//a[text()='Previous']")

With CSS selectors:
next = driver.find_element_by_css_selector('li.next>a')
CSS Selectors are cool and useful: http://www.w3schools.com/cssref/css_selectors.asp
XPATH, not so much - it should be used only as a last resort.