Consider:
from selenium import webdriver
import os
from selenium.webdriver import chrome
driver = webdriver.Chrome()
driver.get("http://nmcunited.me.showenter.com/%D7%9C%D7%94-%D7%9C%D7%94-%D7%9C%D7%A0%D7%93.html")
driver.implicitly_wait(15)
christmasTag = driver.find_element_by_id('f_2561406_1')
christsmasTag.click()
driver.close()
I used the Python code above in order to practice on some basic Selenium operations.
I couldn't find any element, neither by name nor by id. No matter what I tried, I always got the same error, which stated that the element I'm looking for doesn't exist (the idea was to press one of the buttons, if it matters...).
This is a part of the HTML code of the website:
<!-- Main page -->
<td
valign="top">
<table width="100%" cellspacing="0" cellpadding="0" border="0">
<!-- Page Content -->
<!-- Page Content -->
<tr>
<td valign="top">
<table border="0" width="100%" cellpadding="3" cellspacing="0">
<tr>
<!-- The form -->
<td valign="top">
<form id="form2" name="form2" method="post" action="/site/form/sendForm.asp" OnSubmit="return CheckSubmit();" enctype="multipart/form-data">
<!-- Form Field List -->
<table id='sop_form_594602' class='sop_all_form' align="center" border="0" width="100%" cellpadding="3" cellspacing="0"
>
<tr>
<td style="color:;" class="changeText14">
<span style="font-weight:bold;">באיזה חג מפוטר ראין גוסלינג?</span><br>
<input type="radio" name="f_2561406" id="f_2561406_0" value="1. חג הפסחא" class="rachbox" checked>
<label for="f_2561406_0">1. חג הפסחא</label>
<br>
<input type="radio" name="f_2561406" id="f_2561406_1" value="2. חג המולד" class="rachbox">
<label for="f_2561406_1">2. חג המולד</label>
<br>
<input type="radio" name="f_2561406" id="f_2561406_2" value="3. סליבסטר" class="rachbox">
<label for="f_2561406_2">3. סליבסטר</label>
<br>
<input type="radio" name="f_2561406" id="f_2561406_3" value="4. חג ראש השנה" class="rachbox">
<label for="f_2561406_3">4. חג ראש השנה</label>
<br>
</td>
</tr>
This is the said website (broken link)
By the way, this is the first program that uses Selenium I'm trying to run. Is there a possibility that I need to change some settings or that I need to install something else?
The items you are trying to find are inside of an iframe. You need to switch the context of the webdriver to the frame first.
from selenium import webdriver
import os
from selenium.webdriver import chrome
driver = webdriver.Chrome()
driver.get("http://nmcunited.me.showenter.com/%D7%9C%D7%94-%D7%9C%D7%94-%D7%9C%D7%A0%D7%93.html")
driver.implicitly_wait(15)
frame = driver.find_element_by_css_selector('div.tool_forms iframe')
driver.switch_to.frame(frame)
christmasTag = driver.find_element_by_id('f_2561406_1')
christmasTag.click()
driver.close()
Verify whether the "id" is getting changed and generated dynamically by refreshing the page. If it is the case then use the below XPath expression to find the element:
"//input[#value='2. חג המולד']"
Related
Hi we are running this code and it is driving my crazy
we capture a data table in table this works
then grab all th and it's text in sizes this works
then we want to grab all underlying rows in TR; and after loop over columns in rows : does not work! the color_rows object is always empty .. but when testing with xpath in the browser it does! work ... why? how?
My question is: how can I grab the tbody/tr's?
Expected flow
loop over TR's
Access, TR 1 by 1, get 1st TD
Get all TD's data that have class form-control
table = response.xpath('//div[#class="content"]//table[contains(#class,"table")]')
sizes = table.xpath('./thead//th/text()').getall()[1:] #works!
color_rows = table.xpath('./tbody/tr') #does not work! object empty
for color_row in color_rows:
color = color_row.xpath('/td[1]/b/text()').get().strip()
print(color)
stocks = color_row.xpath('/td/div[input[#class="form-control"]]/div//text()').getall()
for size, stock in zip(sizes, stocks)
Our html data looks like this
<table class="table">
<thead>
<tr>
<th id="ctl00_cphCEShop_colColore" class="text-left" colspan="2">Colore</th>
<th>S</th>
<th>M</th>
<th>L</th>
</tr>
</thead>
<tbody>
<tr>
<td id="x">
<b>White</b>
<input type="hidden" name="data" value="3230/201">
</td>
<td id="avail">
Avail:
</td>
<td id="1">
<div>
<input name="cell" type="text" class="form-control">
<div class="text-center">179</div>
</div>
</td>
<td id="2">
<div>
<input name="cell" type="text" class="form-control">
<div class="text-center">360</div>
</div>
</td>
etc etc
Apparently tbody tags are often omitted in HTML but aded by the browser.
In this case there was no (real) body tag making the xpath object miss!
And hence the troubles with xpath (if you really think the tbody tag is there)
Why do browsers insert tbody element into table elements?
I am learning Selenium using python and was trying to automate form filling in some website.
However, I am unable to identify any element under a frame and it gave me NoSuchElementException error. I have tried using different find_element function e.g. CSS, XPATH (pretty sure the XPATH is correct as i copied it from Firebug) and also tried to include explicit wait using WebDriverWait. However, nothing came through.
The command I have tried before:
driver.find_element_by_id("testFormNumForLastAttempt");
driver.find_element_by_xpath("//input[#type='text' and #name='testFormNumForLastAttempt']");
wait = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.ID, "testFormNumForLastAttempt")));
I was trying to switch to another frame
driver.switch_to.frame("main")
raise exception_class(message, screen, stacktrace)
NoSuchFrameException: Message: No frame found
I have extracted the relevant HTML code below and was looking for solution to identify the testFormNumForLastAttempt input parameter.
Thanks in advance. Any help will be much appreciate.
<html>
<frameset rows="0,0,*" frameborder="no" border="0">
<frame name="applet_container" scrolling="no" marginwidth="0" marginheight="0" noresize src="">
<frame name="javascript_container" scrolling="no" marginwidth="0" marginheight="0" noresize src="/repoes/td-es-app517/RETJSContainer_WOC.do?__AUTHENTICATION_REQUEST_PARAMETER_MECHANISM__=applet">
<frame name="main" scrolling="yes" noresize src="/repoes/td-es-app517/Loading.do" marginwidth="0" marginheight="0" frameborder="0">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="Cache-Control" content="no-cache">
<meta http-equiv="Expires" content="0">
<meta http-equiv="Pragma" content="no-cache">
<div id="authByTestForm">
<table width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td>Test Form No. for Last Attempt <span class="star">*</span> </td>
<td>
<input type="text" size="10" maxlength="8" name="testFormNumForLastAttempt" id="testFormNumForLastAttempt" value="" class="uppercase" autocomplete="off">
</td>
</tr>
<tr>
<td valign="top">Date of Birth <span class="star">*</span></td>
<td>
<table class="example_table">
<tbody>
<tr>
<td colspan="7">
<input type="text" size="4" maxlength="4" id="testFormBirthYear" name="testFormBirthYear" onkeyup="moveOnMax(this,'testFormBirthMonth')" value="" autocomplete="off"> Year
<input type="text" size="2" maxlength="2" id="testFormBirthMonth" name="testFormBirthMonth" onkeyup="moveOnMax(this,'testFormBirthDay')" value="" autocomplete="off"> Month
<input type="text" size="2" maxlength="2" id="testFormBirthDay" name="testFormBirthDay" value="" autocomplete="off"> Day
</td>
</tr>
<tr>
<td>Example:</td>
<td class="example_cell">1995</td>
<td>Year</td>
<td class="example_cell">09</td>
<td>Month</td>
<td class="example_cell">08</td>
<td>Day</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
The <input> tag with id as testFormNumForLastAttempt seems to be nested within 3 layers of <frame>.
So to click() within the element you have to:
Ignore (safely) the presence of parent or child <frameset> tags.
Induce WebDriverWait for the first layer of frame_to_be_available_and_switch_to_it().
Induce WebDriverWait for the second layer of _frame_to_be_available_and_switch_to_it().
Induce WebDriverWait for the third layer of frame_to_be_available_and_switch_to_it().
Induce WebDriverWait for the desired element_to_be_clickable().
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"applet_container")))
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"frame[name='javascript_container'][src$='__AUTHENTICATION_REQUEST_PARAMETER_MECHANISM__=applet']")))
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//frame[#name='main' and contains(#src, 'Loading')]")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input#testFormNumForLastAttempt[name='testFormNumForLastAttempt']"))).click()
You can find a relevant discussion in How to locate and click on an element which is nested within multiple frame and frameset through Selenium using Webdriver and C#
Outro
How can I select a html element no matter what frame it is in in selenium?
Ways to deal with #document under iframe
I'm using Ghost for Python 2.7 and I'm trying to click in a link which is in a table. The problem is that I have no ID, name... This is the HTML code:
<table id="table_webbookmarkline_2" cellpadding="4" cellspacing="0" border="0" width="100%">
<tr valign="top">
<td>
<a href="/dana/home/launch.cgi?url=.ahuvs%3A%2F%2Fhq0l5458452ERA-w-Xz8G3LKe8JNM%2F.ISDXWXaWXUivecOc" target="_blank" onClick='javascript:openBookmark(
this.href, "yes", "yes");
return false;' ><img src="/dana-cached/imgs/icn18x18WebBookmarkPop.gif" alt="This will open in a new TAB" width="18" height="18" border="0" ></a>
</td>
<td width="100%" align="left">
<a href="/dana/home/launch.cgi?url=.ahuvs%3A%2F%2Fhq0l5458452ERA-w-Xz8G3LKe8JNM%2F.ISDXWXaWXUivecOc" target="_blank" onClick='JavaScript:openBookmark(
this.href, "yes", "yes");
return false;' ><b>**LINK WHERE I WANT TO CLICK**</b> </a><br><span class="cssSmall"></span>
</td>
</tr>
</table>
How can I click in this kind of link ?
Seems like Ghost's Session.click() takes a CSS selector. Here only the table has an ID, so a selector that takes the second td that is a descendant of that ID and finds the a element should work:
session.click('#table_webbookmarkline_2 td:nth-child(2) a')
From this Deutsche Börse web page, under the table header Issuer I want to get the string content 'db X-trackers' in the cell next to the one with Name in it.
Using my web browser, I inspect that table area and get the code, which I've pasted into this XML tree just so that I can test my xPath.
<root>
<div class="row">
<div class="col-lg-12">
<h2>Issuer</h2>
</div>
</div>
<div class="table-responsive">
<table class="table">
<tbody>
<tr>
<td>Name</td>
<td class="text-right">db X-trackers</td>
</tr>
</tbody>
</table>
</div>
</root>
According to FreeFormatter.com, my xPath below succeeds in retrieving the correct element (Text='db X-trackers'):
my_xpath = "//h2['Issuer']/ancestor::div[#class='row']/following-sibling::div//td['Name']/following-sibling::td[1]/text()"
Note: It goes to <h2>Issuer</h2> first to identify the right place to start working from.
However, when I run this on the actual web page using Selenium WebDriver, None is returned.
def get_sibling(driver, my_xpath):
try:
find_value = driver.find_element_by_xpath(my_xpath).text
except NoSuchElementException:
return None
else:
value = re.search(r"(.+)", find_value).group()
return value
I don't believe anything is wrong in the function itself, so either the xPath must be faulty or there is something in the actual web page source code that throws it off.
When studying the actual Source code in Chrome, it looks a bit messier than what I see with Inspector, which is what I used to create the little XML tree above.
<div class="box">
<div class="row">
<div class="col-lg-12">
<h2>Issuer</h2>
</div>
</div>
<div class="table-responsive">
<table class="table">
<tbody>
<tr>
<td >
Name
</td>
<td class="text-right" >
db X-trackers
</td>
</tr>
<tr>
<td >
Product Family
</td>
<td class="text-right" >
db X-trackers
</td>
</tr>
<tr>
<td >
Homepage
</td>
<td class="text-right" >
<a target="_blank" href="http://www.etf.db.com">www.etf.db.com</a>
</td>
</tr>
</tbody>
</table>
</div>
Are there some peculiarities in the source code above, or is my xPath (or function) wrong?
I would use the following and following-sibling axis:
//h2[. = "Issuer"]/following::table//td[. = "Name"]/following-sibling::td
First we locate the h2 element, then get the following table element. In the table element we look for the td element with Name text and then get the following td sibling.
I am trying to use Requests module to login into a site and get the html of the landing page. I am new to these stuff and I can't find a decent tutorial for this.
Here's the information that I have about that page
HTML of the form for login (url:http://14.139.251.99:8080/jopacv06/html/checkouts)
<FORM NAME="form" METHOD="POST" ACTION="./memberlogin" onsubmit="this.onsubmit= function(){return false;}">
<table class='loginTbl' border='1' align="center" cellspacing='3' cellpadding='3' width='60%'>
<input type="hidden" name="hdnrequesttype" value="1" />
<thead>
<tr>
<td colspan='3' align="middle" class='loginHead'>Login</td>
</tr>
</thead>
<tbody class='loginBody'>
<tr>
<td class='loginBodyTd1' nowrap="nowrap">Employee ID</td>
<td class='loginBodyTd2'><input type='text' name='txtmemberid' id='txtmemberid' value='' class='loginTextBox' size='30' maxlength='8'/></td>
<td class='loginBodyTd3' rowspan='2'><input type="submit" class="goclearbutton" value=" Go "></td>
</tr><input type='hidden' name='txtmemberpwd' id='txtmemberpwd' value='' />
</tbody>
<tfoot>
<tr>
<td colspan='3' class='loginFoot'>
<font class='loginRed'>New Visitor?</font>
Send your registration request to library !
</td>
</tr>
</tfoot>
</table>
</form>
I came to know that I may need to set cookie , so the cookie name in the landing page is JSESSIONID(in case that's reqd). And I discovered that once I successfuly log in then I would have to use beautifulSoup to get the details. Please help me how to combine these pieces together.
You will have to do something like this,
import requests
response = requests.post("http://14.139.251.99:8080/jopacv06/html/checkouts/memberlogin", data = {'txtmemberid': '1'})
if response.status_code == 200:
html_code = response.text
// Do whatever you want to do further with this HTML now.