Develop Reference

Clicking radio button Selenium/Python

I am creating a Linkedin job scraper in order of most recent, but I am finding it really difficult to target the 'Most recent' radio button as shown below.
So far, the 'Most relevant' menu is clicked on, but will not click on 'Most recent'. Help would be appreciated I can't seem to figure this one out :/
Code snippet
driver.get('https://uk.linkedin.com/jobs/bioinformatics-jobs?position=1&pageNum=0')
driver.implicitly_wait(10)
driver.find_element_by_class_name('collapsible-dropdown').click() # Clicks 'sort-by' menu.
driver.find_element_by_xpath('//*[#id="sortBy-0"]').click() # Error!
HTML
<button aria-expanded="true" aria-label="Sort By filter. Most relevant filter is currently applied. Clicking this button displays all Sort By filter options." class="filter-button filter-button--selected dropdown-to-modal__button collapsible-dropdown__button" data-tracking-control-name="public_jobs_sortBy" type="button">
Most relevant
<icon class="filter-button__icon"></icon>
</button>
<div class="collapsible-dropdown__list no-focus-ring" tabindex="-1">
<!-- -->
<fieldset class="filter-values-container">
<legend class="filter-values-container__legend sr-only">Sort By filter options</legend>
<div class="filter-values-container__filter-values">
<div class="filter-values-container__filter-value">
<input checked="" form="jserp-filters" id="sortBy-0" name="sortBy" type="radio" value="R"/>
<label for="sortBy-0">
Most relevant
</label>
</div>
<div class="filter-values-container__filter-value">
<input form="jserp-filters" id="sortBy-1" name="sortBy" type="radio" value="DD"/>
<label for="sortBy-1">
Most recent
</label>
</div>
</div>
</fieldset>
<button aria-label="Apply filters" class="filter__submit-button" data-tracking-control-name="public_jobs_sortBy" form="jserp-filters" type="submit">
Done
</button>
</div>
<div class="filter-values-container__filter-value">
<input id="sortBy-1" form="jserp-filters" name="sortBy" value="DD" type="radio">
<label for="sortBy-1">
Most recent
</label>
</div>
</div>
</fieldset>
<button class="filter__submit-button" aria-label="Apply filters" form="jserp-filters" data-tracking-control-name="public_jobs_sortBy" type="submit">
Done
</button>
</div>

try this
driver.find_element_by_xpath('//*[#id="jserp-filters"]/ul/li[1]/div/div/div/fieldset/div/div[2]').click()
your xpath was incomplete

Things to be noted down :
You need Explicit waits.
JS intervention.
Reliable locators.
Prefer CSS over Xpath.
Code :-
driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.get("https://uk.linkedin.com/jobs/bioinformatics-jobs?position=1&pageNum=0")
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.collapsible-dropdown>button[data-tracking-control-name='public_jobs_sortBy']"))).click()
driver.execute_script("arguments[0].click();", driver.find_element_by_css_selector("input[value='DD']"))
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

how to get the second value of the same class in selenium?

I am trying to simulate in selenium when two input types are left blank and when the input field gets unfocused it will return an invalid message. What I want to do is get both of this invalid message that has the same span classname.
Here is my python code:
fn_input = driver.find_element(By.XPATH, "//input[#name='fn']")
fn_input.send_keys('')
ln_input = driver.find_element(By.XPATH, "//input[#name='ln']")
ln_input.send_keys('')
driver.find_element_by_id('submit_search').click()
time.sleep(5)
fn_error_field = driver.find_element_by_class_name('formfield__message')
Above code works fine for finding the invalid message for fn_input. My question is how do I get the second invalid message for ln_input?
HTML:
<div class="formfield formfield--notification formfield--column-view">
<div class="formfield__label">First name</div>
<div class="formfield__input-wrapper">
<input
name="fn"
id="fn"
type="text"
class="input formfield__input"
value=""
/><span
class="formfield__icon-secondary icon-close icon-color-notification"
></span
><span class="formfield__message">Invalid field</span>
</div>
</div>
<div
class="formfield formfield--notification formfield--column-view capitalized"
>
<div class="formfield__label">Last Name</div>
<div class="formfield__input-wrapper">
<input
name="ln"
id="ln"
type="text"
class="input formfield__input"
value=""
/><span
class="formfield__icon-secondary icon-close icon-color-notification"
></span
><span class="formfield__message">Invalid field</span>
</div>
</div>
<button id="submit_search" type="button" class="button button--primary">
</i>submit
</button>

There are several ways to achieve this.
Collect your element use list with .find_elements_*:
elements = driver.find_elements_by_class_name('formfield__message')
#for second element
elements[1].text
Use following xpath with following-sibling:
element = driver.find_element_by_xpath('//input[#name="ln"]//following-sibling::span[#class="formfield__message"]')
XPath Axes

If you have locators with same name you need to use findelements in selenium
List<WebElement> Selects=driver.findElements(By.Xpath("//input[#name='ln']"));
// if you want select 2 nd element use below line
Selects.get(1).click();

How to use python selenium find_element_by_xpath find out above key word

I am trying to find a string. But it doesn't seem to work.
HTML：
<form name="form1" method="post" action="?cz=del&wbid=7683290543&zjt=aaa&lx=CNAME&xl=%C4%AC%C8%CF&fs=" onSubmit="return b_ifsf('delete？');" id="form1">
<td style="width:120px">
<input type="hidden" name="ip" value="aaa.xxx.com.a.bdydns.com." >
<input type="submit" name="rpt$btnDelete" value="delete" />
</td>
</form>
<form name="form1" method="post" action="?cz=del&wbid=2324242122&zjt=bbb&lx=CNAME&xl=%C4%AC%C8%CF&fs=" onSubmit="return b_ifsf('delete？');" id="form1">
<td style="width:120px">
<input type="hidden" name="ip" value="bbb.xxx.com.a.bdydns.com." >
<input type="submit" name="rpt$btnDelete" value="delete" />
</td>
</form>
<form name="form1" method="post" action="?cz=del&wbid=2324242553&zjt=ccc&lx=CNAME&xl=%C4%AC%C8%CF&fs=" onSubmit="return b_ifsf('delete？');" id="form1">
<td style="width:120px">
<input type="hidden" name="ip" value="ccc.xxx.com.a.bdydns.com." >
<input type="submit" name="rpt$btnDelete" value="delete" />
</td>
</form>
How to find out the key word bbb.xxx.com.a.bdydns.com. and then hit submit to delete it?

#EVNRaja's solution was in the right direction.
To locate the text bbb.xxx.com.a.bdydns.com. then click the associated element with value attribute as delete you can use either of the following solutions:
Using xpath and click():
driver.find_element_by_xpath("//form[#id='form1' and #name='form1']//input[#name='ip' and #value='bbb.xxx.com.a.bdydns.com.']//following::input[1]").click()
Using xpath and submit():
driver.find_element_by_xpath("//form[#id='form1' and #name='form1']//input[#name='ip' and #value='bbb.xxx.com.a.bdydns.com.']//following::input[1]").submit()

The URL you are trying to identify is quoted as hidden element.
The html code you have provided:
<input type="hidden" name="ip" value="bbb.xxx.com.a.bdydns.com." >
All the hidden elements in a browser may have a purpose.
Example:
Consider there is text-field and it doesn't numeric values as input, if end-user enters any numeric values there will be an error code displays next to the text-field.
Here, until we enter a numeric text the error message (text inside html tag element) will be hidden.
In the html code you have shared, the value which you want to inspect was quoted inside input tag and has a type="hidden" name="ip" value="bbb.xxx.com.a.bdydns.com.", we can write a compound xpath as follows:
A example with multiple compound statements:
//input[#type = 'hidden' and #name = 'ip' and contains(#value, 'bbb.xxx.com.a.bdydns.com.')]/following-sibling::input
or
A Simple example:
//input[contains(#value, 'bbb.xxx.com.a.bdydns.com.')]/following-sibling::input
With this xpath code we can directly identify the submit and in the next step you can click the button.

You should be able to use a css selector combination of:
[value='bbb.xxx.com.a.bdydns.com.'] + input
Code:
driver.find_element_by_css_selector("[value='bbb.xxx.com.a.bdydns.com.'] + input").click() #.submit()
The first part is an attribute = value css selector then the "+" is an adjacent sibling combinator, followed by an element selector; saying, find input tag element that is an adjacent sibling element to element with attribute value having value of bbb.xxx.com.a.bdydns.com.

How to get buttons of all forms on webpage using python selenium?

I searched for answer before ask here but didn't get lucky enough. So here it goes, I am doing web scraping using python selenium. before choosing selenium I checked for mechanize, scrapy but I failed to execute some button clicks with them then I checked selenium it seems okay with some cons. I am saying all this because if I chose the wrong tool then please correct me before it is too late.
My question about selenium is how do I get all elements on webpage using xpath. eg: On webpage each webpage I got 10 forms and each of them has a button. So I want to get all the forms on web page to loop on them and click button one by one.
eg:
<form id="#F0">
<input type="button" name="itens" value="Items" class="texField2" onClick="somefunction()"/>
</form>
<form id="#F0">
<input type="button" name="itens" value="Items" class="texField2" onClick="somefunction()"/>
</form>
<form id="#F1">
<input type="button" name="itens" value="Items" class="texField2" onClick="somefunction()"/>
</form>
<form id="#F2">
<input type="button" name="itens" value="Items" class="texField2" onClick="somefunction()"/>
</form>
<form id="#F3">
<input type="button" name="itens" value="Items" class="texField2" onClick="somefunction()"/>
</form>
<form id="#F4">
<input type="button" name="itens" value="Items" class="texField2" onClick="somefunction()"/>
</form>
Another thing, What I am doing is I click on button on 1st form (it takes me to another page, I go back to history and click on 2nd button then go back to history and so on) it seems scraping would be slow. Is there any better way to do the same?
Thank you !!
Edit:
from selenium import webdriver
mydriver = webdriver.Firefox()
baseurl = """http://www.comprasnet.gov.br/consultalicitacoes/ConsLicitacao_Filtro.asp?numprp=&dt_publ_ini=24/02/2016&dt_publ_fim=10/03/2016&chkModalidade=1,2,3,20,5,99&chk_concor=31,32,41,42&chk_pregao=1,2,3,4&chk_rdc=1,2,3,4&optTpPesqMat=M&optTpPesqServ=S&chkTodos=-1&chk_concorTodos=-1&chk_pregaoTodos=-1&txtlstUf=&txtlstMunicipio=&txtlstUasg=&txtlstGrpMaterial=&txtlstClasMaterial=&txtlstMaterial=&txtlstGrpServico=&txtlstServico=&txtObjeto="""
mydriver.get(baseurl)
mydriver.find_element_by_id('ok').click()
buttons = mydriver.find_element_by_xpath("//form//input[#type='button']")
for button in buttons:
button.click()

You can get forms buttons by xpath with code below:
buttons = driver.find_elements_by_xpath(".//form//input[#type='button']")
And iterate them via simple for loop:
for button in buttons:
button.click()
Alternativelly you can use find_elements_by_css_selector function to get elements:
buttons = driver.find_elements_by_css_selector("input[type='button']")

How to execute javascript after filling in a form using selenium?

Just getting started with selenium, this is the HTML of the page:
<div id="signInForm">
<form action="/cgi-bin/VmLoginCgi" method="POST" name="signIn1" id="signIn">
<h2 id="loginTitle">Sign in to view and change your settings</h2>
<div class="field formField noHint username clearfix">
<label for="username"></label>
</div>
<div style="position:relative;" class="field formField noHint password clearfix"><label for="password">Settings Password</label><input type="password" autocomplete="off" value="" maxlength="15" class="name required onefiftyPX inactive" name="jgwhnZLOXn" id="password" onkeypress="handleKeyPress(event)"></div>
<div>
Sign In
</div>
</form>
<p>
<span style="font-weight:bold;">Don't know your password?</span><br>You'll find your default password on the bottom of your Super Hub.
</p>
</div>
I'm filling in the text box by doing:
ele = browser.find_element_by_id("password")
ele.send_keys("supersecretpassword")
But can't submit the actual form by executing the javascript. I've tried:
browser.execute_script("SignIn()")
But get a big long error, any pointers?

Submit the form with submit():
ele.submit()
It would find the element's parent form and submit it.
Or, if that's not gonna work in your case, find the Sign In button and click it:
driver.find_element_by_link_text("Sign In").click()