I am trying to write python code which does the following:
1) Logs into OKCupid
2) Goes to a users questions page
3) Answers an unanswered question.
I am using RoboBrowser to do this. I can do steps 1), 2) and I can get the form of the question I want to submit, but once I submit (using RoboBrowser's submit) it doesn't seem to go through to OKCupid (in that the question isn't registered as answered on my profile).
Here is the code I have. Note the My_RoboBrowser just allows me to pass False to RoboBrowser.open for the verify argument.
class My_RoboBrowser(RoboBrowser):
def __init__(self, auth=None, parser=None, headers=None, user_agent=None, history=True):
RoboBrowser.__init__(self, parser=None, user_agent=None, history=True)
def Open(self, vURL, vVerify=True):
response = self.session.get(vURL, verify=vVerify)
self._update_state(response)
browser = My_RoboBrowser()
urlL = 'https://okcupid.com/login'
browser.open(url)
form = browser.get_form(id='loginbox_form')
form['username'] = 'Username'
form['password'] = 'Password'
browser.submit_form(form)
urlQ = 'https://www.okcupid.com/profile/USER/questions?low=1'
browser.open(urlQ)
Question_Tag = browser.find_all(class_="not_answered")[0]
ID = Question_Tag.get('data-qid')
#Get the form to fill out
Form = browser.get_form(id='answer_'+str(ID))
Form['my_answer'].value = '1'
Form['their_answer'].value = ['1']
Form['importance'].value = '1'
browser.submit_form(Form)
Also if it is helpful when I look at the form object Form in an IPython notebook it says:
<RoboForm my_answer=, their_answer=[], importance=>
before the submit and
<RoboForm my_answer=1, their_answer=['1'], importance=1>
after.
Finally, if it is helpful, the following is the code for one of the forms I am trying to answer (which was gotten by inspecting the element)
<form id="answer_179268" name="answer_179268" class="answer_area okform initialized">
<div class="container my_answer">
<input id="my_answer_1_179268" name="my_answer" value="1" false="" type="radio">
<label class="radio" for="my_answer_1_179268">
<span class="icon"></span>
Yes
</label>
<input id="my_answer_2_179268" name="my_answer" value="2" false="" type="radio"></input>
<label class="radio" for="my_answer_2_179268">
<span class="icon"></span>
No
</label>
</div>
<div class="container acceptable_answers">
<div class="title">
<p>Answer(s) you’ll accept</p>
</div>
<label class="checkbox acceptable_answer" for="their_answer_1_179268">
<input id="their_answer_1_179268" class="acceptable_answer" name="their_answer" value="1" false="" type="checkbox"></input>
<span class="icon"></span>
Yes
</label>
<label class="checkbox acceptable_answer" for="their_answer_2_179268">
<input id="their_answer_2_179268" class="acceptable_answer" name="their_answer" value="2" false="" type="checkbox"></input?
<span class="icon"></span>
No
</label>
<label class="checkbox irrelevant" for="their_answer_any_179268">
<input id="their_answer_any_179268" class="irrelevant" name="their_answer" value="irrelevant" type="checkbox"></input>
<span class="icon"></span>
Any of the above
</label>
</div>
<div class="container importance">
<div class="title">
<p>Importance</p>
</div>
<div class="importance_radios">
<input id="importance_179268_5" name="importance" value="5" false="" type="radio"></input>
<label class="importance_5 radio" for="importance_179268_5" data-count="5">
<span class="icon"></span>
<div class="bar"></div>
<span class="label"></span>
</label>
<input id="importance_179268_4" name="importance" value="4" false="" type="radio"></input>
<label class="importance_4 radio" for="importance_179268_4" data-count="4">
<span class="icon"></span>
<div class="bar"></div>
<span class="label">A little</span>
</label>
<input id="importance_179268_3" name="importance" value="3" false="" type="radio"></input>
<label class="importance_3 radio" for="importance_179268_3" data-count="3">
<span class="icon"></span>
<div class="bar"></div>
<span class="label">Somewhat</span>
</label>
<input id="importance_179268_2" name="importance" value="2" false="" type="radio"> </input>
<label class="importance_2 radio" for="importance_179268_2" data-count="2">
<span class="icon"></span>
<div class="bar"></div>
<span class="label"></span>
</label>
<input id="importance_179268_1" name="importance" value="1" false="" type="radio"> </input>
<label class="importance_1 radio" for="importance_179268_1" data-count="1">
<span class="icon"></span>
<div class="bar"></div>
<span class="label">Very</span>
</label>
</div>
<div class="irrelevant_message">
<span class="irrelevant_text">Irrelevant</span>
<span class="message_text">(Because you’ll accept any answer, this question is marked irrelevant)</span>
</div>
</div>
<div id="explanation_container_179268" class="container explanation">
<div id="answer_179268_explanationContainer" class="inputcontainer textarea noresize empty">
<textarea id="answer_179268_explanation" class="noresize" placeholder="Explain your answer (optional)" false=""></textarea>
<span class="message empty" style="height: 0"></span>
<div class="icon"></div>
</div>
</div>
<button id="submit_btn_179268" class="submit_btn flatbutton disabled small">Answer</button>
<button id="cancel_btn_179268" class="cancel_btn flatbutton silver small">Cancel</button>
<a class="skip_btn inner" href="javascript:void(0)" draggable="false">Skip question</a>
<div id="public_container_179268" class="answer_privately">
<label class="checkbox" for="private_179268">
<input id="private_179268" false="" type="checkbox"></input>
<span class="icon"></span>
<span class="text">Answer privately</span>
</label>
</div>
</form>
I've seen several cases where someone couldn't submit b/c the site used Javascript to submit forms. In other words, they could use Robobrowser or Mechanize to log in and fill out a form, but the form itself could not be submitted -- as submission relied on JS. That may be your problem here. If that's the case, you should try using Selenium.
You can likely verify whether JS is being used to submit in your browser's inspector. <ctrl + shift + i / network / *clear* inspector's network panel before clicking submit / click submit / check type for your Post>
I think that's how I verified, but others here would know better than me. Good luck!
You should take a look at https://github.com/IvanMalison/okcupyd. It will let you do this without using a browser.
Related
So, I have multiple jobs in view with "Report" modal. in that modal there is a form with reasons to report. The problem is that radio button highlights only in first form. if I open report for fifth job it will change first job report modal and there will be no button highlight, but if you open modal of first job, its highlight. I made sure to change name= attribute of every group, so I don't understand why it's happening
<!-- Modal report posting -->
<div class="modal" id="{{post.id}}-1" tabindex="-1" role="dialog" aria-labelledby="validate-modal-label" aria-hidden="true">
<div class="modal-dialog modal-dialog-centered" role="document">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title" id="exampleModalLongTitle">Report</h5>
<button type="button" class="close" data-dismiss="modal" aria-label="Close">
<span aria-hidden="true">×</span>
</button>
</div>
{{post.id}}
<div class="modal-body">
<form action="" class="report_job_form" method="post" novalidate>
<input type="hidden" name="crf-protect" id="crf-protect" value="{{csrf_token()}}">
<div class="report-element">
<div class="report-category hoverable">
<p>
<label for="fkj">
<input type="radio" id="fkj" name="report{{post.Position}}_value" value="Fake Job">
<span>Fake Job</span>
</label>
</p>
</div>
<div class="report-category hoverable">
<p>
<label for="disc-3">
<input type="radio" id="disc-3" name="report{{post.Position}}_value" value="Discrimination">
<span>Discrimination</span>
</label>
</p>
</div>
<div class="report-category hoverable">
<p>
<label for="advert">
<input type="radio" id="advert" name="report{{post.Position}}_value" value="Advertisement">
<span>Advertisement</span>
</label>
</p>
</div>
<div class="report-category hoverable">
<p>
<label for="other-2">
<input type="radio" id="other-2" name="report{{post.Position}}_value" value="Other">
<span>Other</span>
</label>
</p>
</div>
</div>
<div class="input-field col s12">
<textarea id="textarea7" class="materialize-textarea" data-length="120" name="details"></textarea>
<label for="textarea2">More Details</label>
</div>
<input type="hidden" name="user-ident" id="user-ident" value="{{current_user.id}}" readonly>
<input type="hidden" name="post-ident" id="post-ident" value="{{post.id}}" readonly>
<input class="btn report-submit" type="submit">
</form>
</div>
Original HTML Code
<form id="commentUpdateForm" method="post" data-commentid="">
<fieldset>
<input type="hidden" name="point" value="">
<div class="grade_star">
<span class="ico_star star_rate">
<span class="ico_star inner_star" style="width: 0%;"></span>
</span>
<em class="num_rate">0<span class="txt_g">/5</span></em>
<span class="txt_word">Please rate it</span>
</div>
<div class="write_review">
<label for="tfReview" class="lab_review">What did you like about this place?</label>
<textarea name="contents" id="tfReview2" class="tf_review"></textarea>
</div>
</fieldset>
</form>
If I click 3 out of 5 points, the HTML code will be changed like this.
<form id="commentUpdateForm" method="post" data-commentid="">
<fieldset>
*<input type="hidden" name="point" value="3">*
<div class="grade_star">
<span class="ico_star star_rate">
*<span class="ico_star inner_star" style="width: 60%;"></span>*
</span>
<em class="num_rate">0<span class="txt_g">/5</span></em>
<span class="txt_word">Normal</span>
</div>
<div class="write_review">
<label for="tfReview" class="lab_review">What did you like about this place?</label>
<textarea name="contents" id="tfReview2" class="tf_review"></textarea>
</div>
</fieldset>
</form>
So what I want to do is to make it possible to modify it with Python Selenium code for points, but I don't know.
You can locate the element with selenium webdriver and modify its value attribute like that:
element = driver.find_element_by_xpath('//form[#id="commentUpdateForm"]//input[name="point"]')
element.setAttribute("value", "3")
I'm brand new to coding and am following the Udemy Zero to Mastery course. I'm currently having trouble getting my HTML5 to process a function I've made in Python. I want to return my function, submit_form, when I click on 'send message'. Here's the HTML5:
<section id="three">
<h2>Get In Touch</h2>
<p>This is the contact section.</p>
<div class="row">
<div class="col-8 col-12-small">
<form method="post" action="index.html">
<div class="row gtr-uniform gtr-50">
<div class="col-6 col-12-xsmall"><input type="text" name="name" id="name"
placeholder="Name" /></div>
<div class="col-6 col-12-xsmall"><input type="email" name="email" id="email"
placeholder="Email" /></div>
<div class="col-12"><textarea name="message" id="message" placeholder="Message"
rows="4"></textarea></div>
</div>
</form>
<ul class="actions">
<!-- <button type="submit" class="btn btn-default btn-lg">Send Message</button> -->
<li><input type="button" value="Send Message" /></li>
</form>
</ul>
</div>
And here is my python function:
#app.route('/submit_form', methods=['POST', 'GET'])
def submit_form():
return 'form submitted!'
I'm having a lot of trouble understanding how to make the send message button return my function. Any help is much appreciated, thank you!
I've been trying to write a program which gets my school timetable which is locked behind a login screen, however when I run my python script I am greeted by:
Sorry, we are unable to complete this request
An error occured while submitting the form:
Invalid Request Origin
This is the code I've written so far:
import requests
login_url = 'https://WEBSITE/login?page=%2Fhomepage'
timetable_url = 'https://WEBSITE/timetable'
payload = {
'username': 'USERNAME',
'password': 'PASSWORD'
}
with requests.Session() as session:
post = session.post(login_url, data=payload, verify = True)
r = session.get(timetable_url)
print (post.text)
And this is the html from the login page:
<div id="content">
<form id="login" method="post" action="/login?page=%2Flogin%2F">
<input type="hidden" name="page" value="/login/" />
<div class="row collapse">
<div class="small-12 column">
<div class="logo">
<img src="/images/logo.php?logo=skin_logo_login&size=normal" alt="Schoolbox" />
</div>
</div>
<div class="small-12 column">
<p id="error-msg" class="alert-box alert"></p>
</div>
<div class="small-12 column">
<input type="text" name="username" id="username" placeholder="Username" />
<input type="password" name="password" id="password" placeholder="Password" />
</div>
</div>
<div class="row collapse">
<div class="small-12 column">
<p>Forgotten your password?</p>
</div>
<div class="small-12 column">
<input id="rememberme" type="checkbox" name="rememberme" value="1" />
<label for="rememberme">Remember me</label>
</div>
<div class="small-12 column login-links">
<input type="submit" name="Submit" value="Login" alt="Login" />
</div>
</div>
</form>
</div>
How exactly would I be able to solve this error, or would there be any other more viable solutions? Thank you.
This error Invalid Request Origin means that your origin does not match what is restricted by the server. You could try adding the Origin: header specifying your timetable_url, but it is likely that the page can only be called by another page on the server.
Can't figure out why Google won't let me log into my account. I put my email inside a variable called my_mail then search for it in the html but the results came back false.
I am using the requests & beautifulsoup modules to do so.
What I have:
from bs4 import BeautifulSoup
import requests
form_data={'Email': 'ExampleEmail#gmail.com', 'Passwd': 'Password'}
post = "https://accounts.google.com/signin/challenge/sl/password"
my_mail = 'ExampleEmail#gmail.com'
with requests.Session() as s:
soup = BeautifulSoup(s.get("https://accounts.google.com/ServiceLogin?elo=1").text, "html.parser")
for inp in soup.select("#gaia_loginform input[name]"):
if inp["name"] not in form_data:
form_data[inp["name"]] = inp["value"]
s.post(post, form_data)
print(my_mail in "https://mail.google.com/mail/u/0/#inbox")
Form info:
<form novalidate="" method="post" action="https://accounts.google.com/signin/challenge/sl/password" id="gaia_loginform">
<input name="Page" type="hidden" value="PasswordSeparationSignIn">
<input type="hidden" name="GALX" value="c90FYb-yd6E">
<input type="hidden" name="gxf" value="AFoagUVMILWKQvqlGV0i8sJG83jiQlrWzg:1475424777331">
<input id="profile-information" name="ProfileInformation" type="hidden" value="">
<input type="hidden" id="_utf8" name="_utf8" value="☃">
<input type="hidden" name="bgresponse" id="bgresponse" value="js_disabled">
<input type="hidden" id="pstMsg" name="pstMsg" value="1">
<input type="hidden" id="dnConn" name="dnConn" value="">
<input type="hidden" id="checkConnection" name="checkConnection" value="youtube:230:1">
<input type="hidden" id="checkedDomains" name="checkedDomains" value="youtube">
<div class="form-panel first valid" id="gaia_firstform">
<div class="slide-out ">
<div class="input-wrapper focused">
<div id="identifier-shown">
<div>
<label class="hidden-label" for="Email">
Enter your email</label>
<input id="Email" type="email" value="" spellcheck="false" name="Email" placeholder="Enter your email" autofocus="">
<input id="Passwd-hidden" type="password" spellcheck="false" class="hidden">
</div>
</div>
<span role="alert" class="error-msg" id="errormsg_0_Email"></span>
</div>
<div style="display: none" id="identifier-captcha">
<input type="hidden" name="identifiertoken" id="identifier-token" value="">
<input type="hidden" name="identifiertoken_audio" id="identifier-token-audio">
<div class="audio-box">
<div id="playIdentifierAudio"></div>
</div>
<div id="captcha-box" class="captcha-box">
<div id="captcha-img" class="captcha-img" data-alt-text="Visual verification">
</div>
<span class="captcha-msg">
Letters are not case-sensitive
</span>
</div>
<label for="identifier-captcha-input" class="hidden-label"></label>
<input type="text" id="identifier-captcha-input" name="identifier-captcha-input" class="captcha" placeholder="Enter the letters above" title="Type the characters you see or numbers you hear">
</div>
<input id="next" name="signIn" class="rc-button rc-button-submit" type="submit" value="Next">
<a class="need-help" href="https://accounts.google.com/signin/usernamerecovery?hl=en">
Find my account
</a>
</div>
</div>
<a href="https://accounts.google.com/ServiceLogin" tabindex="-1">
<img id="back-arrow" class="back-arrow " aria-label="Back" tabindex="0" alt="Back" src="https://www.gstatic.com/images/icons/material/system/1x/arrow_back_grey600_24dp.png">
</a>
<div class="form-panel second">
<div class="slide-in hide-form">
<div>
<p id="profile-name"></p>
<span id="email-display"></span>
</div>
<div>
<div id="password-shown"></div>
</div>
<input id="signIn" name="signIn" class="rc-button rc-button-submit" type="submit" value="Sign in">
<label class="remember">
<input id="PersistentCookie" name="PersistentCookie" type="checkbox" value="yes" checked="checked">
<span>
Stay signed in
</span>
<div class="bubble-wrap" role="tooltip">
<div class="bubble-pointer"></div>
<div class="bubble">
For your convenience, keep this checked. On shared devices, additional precautions are recommended.
Learn more
</div>
</div>
</label>
<input type="hidden" name="rmShown" value="1">
<a id="link-forgot-passwd" class="need-help" href="https://accounts.google.com/signin/recovery?hl=en">
Forgot password?
</a>
</div>
</div>
<span id="inge" style="display: none" role="alert" class="error-msg">
Sorry, Google doesn't recognize that email. Create an account using that address?
</span>
<span id="timeoutError" style="display: none" role="alert" class="error-msg">
Something went wrong. Check your connection and try again.
</span>
</form>
I don't plan on using Google's API. I don't want to be restricted.
Why is the login failing? What could I do to fix it?
I need to login to scrap some information that requires a login.
Pyhton 3.4.2