I am filling a form of a web page with the help of mechanize module but I am getting error when I run my code.
I just want to fill the form and submit it successfully.
My attempt :
code snippet from this stack answer
import re
from mechanize import Browser
username="Bob"
password="admin"
br = Browser()
# Ignore robots.txt
br.set_handle_robots( False )
# Google demands a user-agent that isn't a robot
br.addheaders = [('User-agent', 'Firefox')]
br.open("https://fb.vivoliker.com/app/fb/token")
br.select_form(name="order")
br["u"] = [username]
br["p"]=[password]
response = br.submit()
Output :
Error (FormNotFoundError)
but what should I enter the name in br.select_form() because when I see source code of web page their is no name attribute set to that form.
Html source code of form from web page
<div class="container">
<form ls-form="fb-init">
<input type="hidden" name="machine_id">
<div class="form-group row">
<input id="u" type="text" class="form-control" placeholder="Facebook Username / Id / Email / Mobile Number" required="required">
</div>
<div class="form-group row">
<input id="p" type="password" class="form-control" placeholder="Facebook Password" required="required">
</div>
<div class="form-group row mt-3">
<button type="button" id='generating' class="btn btn-primary btn-block" onclick="if (!window.__cfRLUnblockHandlers) return false; get()" data-cf-modified-4e9e40fa9e78b45594c87eaa-="">Get Access Token</button>
</div>
<div ls-form="event"></div>
</form>
Expected output :
My form should be submit with the values that I given.
see javascript of this webpage given below .
I want to fill and submit form of this web page :
Web page source
I believe the form you want to select is ls-form=fb-init
However, since mechanize module requires replacing hyphens with underscores to convert HTML attrs to keyword arguments, you would want to write it like this:
br.select_form(ls_form='fb-init')
To clarify, the correct form to select is not named 'order', the form is named 'fb-init' and it is a ls-form (written as 'ls_form' with underscore). So with the change, it should be like this:
import re
from mechanize import Browser
username="Bob"
password="admin"
br = Browser()
# Ignore robots.txt
br.set_handle_robots( False )
# Google demands a user-agent that isn't a robot
br.addheaders = [('User-agent', 'Firefox')]
br.open("https://fb.vivoliker.com/app/fb/token")
br.select_form(ls_form='fb-init')
And then continue from there.
Related
Here is my HTML code:
<html>
<div class="echo-chat-body" dir="ltr">
<div class="new-chat-form">
<div>
<h1 class="echo-title desktop-only">
Topic?
</h1>
<hr class="desktop-only full-width">
<input id="topic" name="topic" autocomplete="false" placeholder="Topic?" class="js-chat-topic answer input input-text desktop-input js-chat-required-false" type="text">
</div>
<div>
<h1 class="echo-title desktop-only">
What's your name?
</h1>
<hr class="desktop-only full-width">
<input id="name" name="name" autocomplete="false" placeholder="What's your name?" class="js-chat-name answer input input-text desktop-input js-chat-required-true" type="text">
</div>
<div class="form-action">
<label></label>
<button id="startchat" class="echo-button button-secondary js-submit-new-chat center" disabled="">
Start Chat
</button>
</div>
</div>
</div>
</html>
Here is my Python code:
from urllib.parse import urlencode
from urllib.request import Request, urlopen
url = 'file:///C:/Users/SuperDesktop/HTML/Desktop/Chat.html'
post_fields = {'topic': 'Hello', 'name' : 'My Name', 'startchat': 'startchat'}
"""
Topic is 'Hello'.
Name is 'My name'.
startchat is the button which should be clicked.
" 'startchat': 'startchat' " is not the proper code to click the button.
"""
request = Request(url, urlencode(post_fields).encode())
json = urlopen(request).read().decode()
print(json)
I tried that code but it always shows different errors.
How to create a program in python that will auto-fill those 2 boxes and click on submit?
I want to fill this form via python code, I don't want to use Selenium, Automated Browser or chromedriver
I don't want python code opening a browser and filling those things and clicking start button
I also tried to use Selenium, Automated Browser, opening browser and chrome driver but that was not the thing that I wanted.
I wanted a python code that will fill those boxes and send the form within Terminator.
I just start learning Python and want to make a script to submit Form.
I found Form use CGI, Here the Form:
<div class="box" id="url_upload">
<div class="tabcontent">
<div class="progress_div"></div>
<div class="reurlupload">
<div class="progress_div"></div>
<form method="post" id="uploadurl" action="https://af03.ayefiles.com/cgi-bin/upload.cgi?upload_type=url">
<input type="hidden" name="sess_id" value="xv71zsrmtr38oh3z">
<input type="hidden" name="utype" value="reg">
<input type="hidden" name="file_public" value="1">
<div class="leftintab">
<p style="margin:0px;">
You can enter up to <b>20</b> URLs, one URL per row</br>
Max file size is <b>10240 Mb</b>
</p>
<textarea name="url_mass" style="width:100%; margin-top: 10px;" placeholder="e.g. http://example.com/xxxxxxxxxx.xyz"></textarea>
</div>
I make python script using request as below:
#I have session with my login & password as cookie
#Go to form page
login = s.get('https://ayefiles.com/?op=upload_form')
login_html = html.fromstring(login.content)
hidden_inputs = login_html.xpath('//input[#type="hidden"]')
# Input query data
form = {x.attrib["name"]: x.attrib["value"] for x in hidden_inputs}
form ['sess_id']= 'xv71zsrmtr38oh3z'
form['utype']= 'reg'
form ['file_public']= '1'
form['url_mass'] = longurl
# POST
login = s.post('https://af03.ayefiles.com/cgi-bin/upload.cgi?upload_type=url', data=form)
print (login.url)
My expected result for login.url ==> ayefiles.com/?op=upload_result&st=OK&fn=xxxxx
But my result fail, ==> ayefiles.com/?op=upload_result&st=Torrent%20engine%20is%20not%20running&
fn=undef
how to solve my problem? What's wrong with my code?
Please kindly help me with correct code.
My mistake at part multiform data.
Correct code :
form ={'sess_id':(None,'xv71zsrmtr38oh3z'),'utype':(None,'reg'),'file_public':(None,'1'),'url_mass':(None,longurl)}
login = s.post('https://af03.ayefiles.com/cgi-bin/upload.cgi?upload_type=url', data=form)
I am trying to loggin to my account using the following python code without success. The login-process is in two steps on two pages. First enter login, second enter password. I am using Python3:
from bs4 import BeautifulSoup
import requests, lxml.html
with requests.Session() as s:
#First login page
login = s.get('https://accounts.ft.com/login')
login_html = lxml.html.fromstring(login.text)
#getting the form inputs
hidden_inputs = login_html.xpath(r'//form//input')
form = {x.name: x.value for x in hidden_inputs}
#filling inputs with email
form['email'] = 'me#mail.com'
response = s.post('https://accounts.ft.com/login', data=form)
# Receive reponse 200
#Second login page
login_html = lxml.html.fromstring(response.text)
#getting inputs
hidden_inputs = login_html.xpath(r'//form//input')
form = {x.name: x.value for x in hidden_inputs}
#filling inputs with email and password
form['email'] = 'me#mail.com'
form['password'] = 'p****word'
response = s.post('https://accounts.ft.com/login', data=form)
#Receive reponse 200
#Trying to read an article being loggedIn
page = s.get('https://www.ft.com/content/173695cc-1a98-11e7-a266-12672483791a')
soup = BeautifulSoup(page.content, 'html.parser')
print(soup.prettify())
# data-next-is-logged-in="false" => Please Register to read this page...
Here is what the Form looks like:
<div class="js-container" data-component="two-step-login-form" id="content">
<div class="lgn-box">
<form action="/login/submitEmail" class="js-email-lookup-form" data-test-id="enter-email-form" method="POST" name="enter-email-form" novalidate="">
<input name="location" type="hidden" value="" />
<input name="continueUrl" type="hidden" value="" />
<input name="readerId" type="hidden" value="" />
<input name="loginUrl" type="hidden" value="/login" />
<div class="lgn-box__title">
<h1 class="lgn-heading--alpha">
Sign in
</h1>
</div>
<div class="o-forms-group">
<label class="o-forms-label" for="email">
Email address
</label>
<input autocomplete="off" autofocus="" class="o-forms-text js-email" id="email" maxlength="64" name="email" required="" type="email">
<input id="password" name="password" style="display:none" type="password">
<label for="password">
</label>
</input>
</input>
</div>
<div class="o-forms-group">
<button class="o-buttons o-buttons--standout o-buttons--big" name="Next" type="submit">
Next
</button>
</div>
</form>
</div>
Here is what my data passed to POST looks like:
form
{'password': 'p****word', 'continueUrl': '', 'loginUrl': '/login', 'email': 'me#mail.com', 'readerId': '', 'location': ''}
The POST request returns for both 1st and 2nd loggin page a 200 response. But it seems that I am still not logged in.
I have tried using http://accounts.ft.com/sso/redirects?email=me#mail.com as a URL for POST request, returning a 405 Bad Request error
I am not sure that I am actually not logged in, bud I have no idea how to monitor that.
Is it possible that the website prevents me from logging-in if not in a web-browser?
Try using selenium to simulate the web browser as it appears that FT blocks automated access.
Alternatively you can see if a site has been archived with something like archive.is (which will pull most sites into a more machine friendly setup).
Finally, there is both a datamining API and a headline API that the FT offers at their developer page
<div class="group group-form group-form-requiredinformation">
<h2 class="sr">Required Information</h2>
<ol class="list-input">
<li class="field required text" id="field-email">
<label for="email">E-mail</label>
<input class="" id="email" type="email" name="email" value="" placeholder="example: username#domain.com" required aria-required="true" aria-described-by="email-tip" />
<span class="tip tip-input" id="email-tip">This is the e-mail address you used to register with edX</span>
</li>
<li class="field required password" id="field-password">
<label for="password">Password</label>
<input id="password" type="password" name="password" value="" required aria-required="true" />
<span class="tip tip-input">
I am trying to write some code that will parse a webpage and check for updated content periodically, my problem is that I need to be logged in but I am unsure how to pass in my login details, the above is some of the source of the login page, how is it possible to achieve this?
I have tried something like the code below but to no avail.
browser = mechanize.Browser()
browser.open("https:xxxx")
browser.select_form(nr = 0)
browser.form['username'] = "email"
browser.form['password'] = "xxxxx"
browser.submit()
At least, the field is called email, not username:
browser.form['email'] = "email"
browser.form['password'] = "xxxxx"
browser.submit()
store the last updated date time in a variable and have a javascript timer function to invoke a function to check the last updated date time and if the content was updated more than the predefined time, make a call to backend and update the info on the page.
FYI.. there is no browser object. there is document object which has properties and methods to manipulate the page
I'm trying to write some Python (3.3.2) code to log in to a website using the Requests module. Here is the form section of the login page:
<form method="post" action="https://www.ibvpn.com/billing/dologin.php" name="frmlogin">
<input type="hidden" name="token" value="236647d2da7c8408ceb78178ba03876ea1f2b687" />
<div class="logincontainer">
<fieldset>
<div class="clearfix">
<label for="username">Email Address:</label>
<div class="input">
<input class="xlarge" name="username" id="username" type="text" />
</div>
</div>
<div class="clearfix">
<label for="password">Password:</label>
<div class="input">
<input class="xlarge" name="password" id="password" type="password"/>
</div>
</div>
<div align="center">
<p>
<input type="checkbox" name="rememberme" /> Remember Me
</p>
<p>Request a Password Reset</p>
</div>
</fieldset>
</div>
<div class="actions">
<input type="submit" class="btn primary" value="Login" />
</div>
</form>
Here is my code, trying to deal with hidden input:
import requests
from bs4 import BeautifulSoup
url = 'https://www.ibvpn.com/billing/clientarea.php'
body = {'username':'my email address','password':'my password'}
s = requests.Session()
loginPage = s.get(url)
soup = BeautifulSoup(loginPage.text)
hiddenInputs = soup.findAll(name = 'input', type = 'hidden')
for hidden in hiddenInputs:
name = hidden['name']
value = hidden['value']
body[name] = value
r = s.post(url, data = body)
This just returns the login page. If I post my login data to the URL in the 'action' field, I get a 404 error.
I've seen other posts on StackExchange where automatic cookie handling doesn't seem to work, so I've also tried dealing with the cookies manually using:
cookies = dict(loginPage.cookies)
r = s.post(url, data = body, cookies = cookies)
But this also just returns the login page.
I don't know if this is related to the problem, but after I've run either variant of the code above, entering r.cookies returns <<class 'requests.cookies.RequestsCookieJar'>[]>
If anyone has any suggestions, I'd love to hear them.
You are loading the wrong URL. The form has an action attribute:
<form method="post" action="https://www.ibvpn.com/billing/dologin.php" name="frmlogin">
so you must post your login information to:
https://www.ibvpn.com/billing/dologin.php
instead of posting back to the login page. POST to soup.form['action'] instead:
r = s.post(soup.form['action'], data=body)
Your code is handling cookies just fine; I can see that s.cookies holds a cookie after requesting the login form, for example.
If this still doesn't work (a 404 is returned), then the server is using additional techniques to detect scripts vs. real browsers. Usually this is done by parsing the request headers. Look at your browser headers and replicate those. It may just be the User-Agent header that they parse, but Accept-* headers and Referrer can also play a role.