After POST request, there is confirmation(jquery) box appears in browser.
So How do I select confirmation box using request api ?
Here HTML page :
<div class="sent_sms_bg">
<form name="smsAction" id="smsAction" action="/sms_all_action" method="post">
<p><input name="" id="select_all" type="checkbox" value="" /> Select All</p>
<p><input name="Delete_SMS" type="button" class="delete_selected" value="Delete Selected" />
</p>
</div>
and What I'm trying :
self.session = requests.session()
..
...
url = 'http://www.indyarocks.com/sms_all_action'
form = {
'select_all' : 'on',
'Delete_SMS': 'Delete Selected',
}
status = self.session.post(url , data = form)
POSTing directly to the url should work, as presumably the website has a form with an ajax popup to make sure you really want to submit the form, THEN POSTs the data to the URL. It wouldn't work to POST the data first and then fire the popup (as the form is already submitted so it's too late to do anything about it)
Related
edit I believe the issue is that when opening the website for the first time, you must click a "Our privacy policy has been updated..." Accept button. I'm now looking into how I might go about "clicking" this button using python requests, however the button calls a javascript function so I'm not sure if it's possible to do with just the requests library.
I am attempting to write a script for this website: https://rocket-league.com/
I need to be logged in to perform my tasks, but I'm having trouble logging in.
I feel as though I've accounted for all the correct params. I fetch the csrf_token dynamically using regex. Maybe I need to do something with the cookies, but I'm not sure what?
This is my first attempt at writing a script that interacts with websites so I'm sorry if I'm naive.
I'd be grateful for any insights that aren't just a disguised way of telling me I'm dumb and/or lazy.
Here is my code so far:
import requests;
import re;
payload = {
'csrf_token': '',
'email': 'myemail',
'password': 'mypassword',
'submit': 'Login'
}
url = 'https://rocket-league.com/login'
with requests.Session() as s:
r = s.get(url)
m = re.search("<input type='hidden' name='csrf_token' value='(.+)'", r.text)
if m: payload['csrf_token'] = m.group(1)
else: print("couldnt find csrf_token")
p = s.post('https://rocket-league.com/functions/login.php', data=payload)
print(p.text)
Running this code prints out HTML that still has the login form in it which means I am not being logged in.
When I login from my browser with the developer tools > network open, I get this information for my post request:
Headers:
Request URL: https://rocket-league.com/functions/login.php
Request method: POST
Status code: 302 Found
Version HTTP/2.0
Cookies:
__cfduid: d2c83b2c9ad728195366656a56592f6d71549577451
acceptedPrivacyPolicy: 2.0
euconsent: BObnsHAObnsHAABABAENCF-AAAAkF7_______9______9uz_Ov_v_f__33e8__9v_l_7_-___u_-33d4-_1vf99yfm1-7ftr3tp_87ues2_Xur__59__3z3_tphPhA
fantasy_rlcs_id: hA8fga9ghIgaFGA9
PHPSESSID: lrse3tgov95eg574sqga6la9jc
Params:
csrf_token: 3f2588113e8921f52dc3eb78e51246a1
email: myemail
password: mypassword
submit: Login
Here is the actual HTML for the login form:
<form class="rlg-form" method="post" action="/functions/login.php">
<input type="hidden" name="csrf_token" value="3accad82ad0957cab634f805a7e28beb">
<input class="rlg-input" type="email" name="email" placeholder="Email" required="">
<input class="rlg-input" type="password" name="password" placeholder="Password" autocomplete="off" required="">
<fieldset class="rlg-checkbox">
<input type="checkbox" name="rememberme" id="rememberme-login">
<label for="rememberme-login">Remember me?</label>
</fieldset>
<input class="rlg-btn-primary" type="submit" name="submit" value="Login">
</form>
In html, I have a form consisting of three radio button
<form action="{{ url_for('handle_data') }}" method="post">
<input type="radio" name="option" value="banana"> Banana<br>
<input type="radio" name="option" value="apple">apple<br>
<input type="radio" name="option" value="peach"> peach<br>
<input type="submit" value="Submit Text">
</form>
In my app.py, I have the method defined for this:
#app.route('/result/<fruit>', methods=['POST'])
def handle_data(fruit):
fruit = request.form['option']
return render_template("result.html",fruit = fruit)
What I wanna do was redirecting from my current html file to a new URL, and the new URL is corresponding to which radio button I click. For example, if I click "apple" and hit the "submit text" button, I will be redirected to "/result/apple".
I did that in the way above, it gives me a server error. What did I do wrong in this case?
Either do dynamic URLs OR parse form data, you don't need to do both. In this case, I would only use the result URL without the dynamic URL with because your template is expecting your dynamic URL in url_for to be completed. You can still use dynamic URLs, but I believe you would need to add some JavaScript to populate the URL as you click the submit button. Since the template is being rendered before you select a fruit, it doesn't know to pass a fruit at the end of your URL, thus your request would only go to /result/ instead of /result/apple if you submit apple.
#app.route('/result',methods=['POST'])
def handle_data():
fruit = request.form.get["option"]
return render_template("result.html",fruit=fruit)
I just start learning Python and want to make a script to submit Form.
I found Form use CGI, Here the Form:
<div class="box" id="url_upload">
<div class="tabcontent">
<div class="progress_div"></div>
<div class="reurlupload">
<div class="progress_div"></div>
<form method="post" id="uploadurl" action="https://af03.ayefiles.com/cgi-bin/upload.cgi?upload_type=url">
<input type="hidden" name="sess_id" value="xv71zsrmtr38oh3z">
<input type="hidden" name="utype" value="reg">
<input type="hidden" name="file_public" value="1">
<div class="leftintab">
<p style="margin:0px;">
You can enter up to <b>20</b> URLs, one URL per row</br>
Max file size is <b>10240 Mb</b>
</p>
<textarea name="url_mass" style="width:100%; margin-top: 10px;" placeholder="e.g. http://example.com/xxxxxxxxxx.xyz"></textarea>
</div>
I make python script using request as below:
#I have session with my login & password as cookie
#Go to form page
login = s.get('https://ayefiles.com/?op=upload_form')
login_html = html.fromstring(login.content)
hidden_inputs = login_html.xpath('//input[#type="hidden"]')
# Input query data
form = {x.attrib["name"]: x.attrib["value"] for x in hidden_inputs}
form ['sess_id']= 'xv71zsrmtr38oh3z'
form['utype']= 'reg'
form ['file_public']= '1'
form['url_mass'] = longurl
# POST
login = s.post('https://af03.ayefiles.com/cgi-bin/upload.cgi?upload_type=url', data=form)
print (login.url)
My expected result for login.url ==> ayefiles.com/?op=upload_result&st=OK&fn=xxxxx
But my result fail, ==> ayefiles.com/?op=upload_result&st=Torrent%20engine%20is%20not%20running&
fn=undef
how to solve my problem? What's wrong with my code?
Please kindly help me with correct code.
My mistake at part multiform data.
Correct code :
form ={'sess_id':(None,'xv71zsrmtr38oh3z'),'utype':(None,'reg'),'file_public':(None,'1'),'url_mass':(None,longurl)}
login = s.post('https://af03.ayefiles.com/cgi-bin/upload.cgi?upload_type=url', data=form)
I did exactly with https://docs.djangoproject.com/en/1.7/topics/forms/ , but after the html page rendered, the form is hidden. And the source code from html is as following:
<form action = "/blogcore/register/" method = "post">
<input type='hidden' name='csrfmiddlewaretoken' value='0aFX5lRZm4qdP5CfHoZbqSohGEipkWN4' />
<input type = "submit" value = "Next" />
How can I do to make the form display?
The form is not hidden. Only the csrf token is hidden, which is correct.
I'm trying to write some Python (3.3.2) code to log in to a website using the Requests module. Here is the form section of the login page:
<form method="post" action="https://www.ibvpn.com/billing/dologin.php" name="frmlogin">
<input type="hidden" name="token" value="236647d2da7c8408ceb78178ba03876ea1f2b687" />
<div class="logincontainer">
<fieldset>
<div class="clearfix">
<label for="username">Email Address:</label>
<div class="input">
<input class="xlarge" name="username" id="username" type="text" />
</div>
</div>
<div class="clearfix">
<label for="password">Password:</label>
<div class="input">
<input class="xlarge" name="password" id="password" type="password"/>
</div>
</div>
<div align="center">
<p>
<input type="checkbox" name="rememberme" /> Remember Me
</p>
<p>Request a Password Reset</p>
</div>
</fieldset>
</div>
<div class="actions">
<input type="submit" class="btn primary" value="Login" />
</div>
</form>
Here is my code, trying to deal with hidden input:
import requests
from bs4 import BeautifulSoup
url = 'https://www.ibvpn.com/billing/clientarea.php'
body = {'username':'my email address','password':'my password'}
s = requests.Session()
loginPage = s.get(url)
soup = BeautifulSoup(loginPage.text)
hiddenInputs = soup.findAll(name = 'input', type = 'hidden')
for hidden in hiddenInputs:
name = hidden['name']
value = hidden['value']
body[name] = value
r = s.post(url, data = body)
This just returns the login page. If I post my login data to the URL in the 'action' field, I get a 404 error.
I've seen other posts on StackExchange where automatic cookie handling doesn't seem to work, so I've also tried dealing with the cookies manually using:
cookies = dict(loginPage.cookies)
r = s.post(url, data = body, cookies = cookies)
But this also just returns the login page.
I don't know if this is related to the problem, but after I've run either variant of the code above, entering r.cookies returns <<class 'requests.cookies.RequestsCookieJar'>[]>
If anyone has any suggestions, I'd love to hear them.
You are loading the wrong URL. The form has an action attribute:
<form method="post" action="https://www.ibvpn.com/billing/dologin.php" name="frmlogin">
so you must post your login information to:
https://www.ibvpn.com/billing/dologin.php
instead of posting back to the login page. POST to soup.form['action'] instead:
r = s.post(soup.form['action'], data=body)
Your code is handling cookies just fine; I can see that s.cookies holds a cookie after requesting the login form, for example.
If this still doesn't work (a 404 is returned), then the server is using additional techniques to detect scripts vs. real browsers. Usually this is done by parsing the request headers. Look at your browser headers and replicate those. It may just be the User-Agent header that they parse, but Accept-* headers and Referrer can also play a role.