Develop Reference

How do I fill in a loginForm using BeautifulSoup

I want to fill in a loginForm using BeautifulSoup (I know it's not the easiest way) but I don't know how I can do that. Here under the loginForm.
<form id="loginForm" class="loginContainer" accept-charset="utf-8" action="/wipb2bweb/j_spring_security_check;jsessionid=C647FBABEB8C4ADC3DCCA3C6C875BECC.prod1" method="post"><fieldset>
<legend>Login</legend>
<img src="/wipb2bweb/_ui/themes/carhartt/images/loginTitle.png"
alt="Login" class="title" />
<form>
<div class="formRow">
<label for="User"><img
src="/wipb2bweb/_ui/themes/carhartt/images/loginLabelUser_de.png"
alt="Benutzername" />
</label>
<input id="User" name="j_username" type="text" value=""/></div>
<div class="formRow">
<label for="Password"><img
src="/wipb2bweb/_ui/themes/carhartt/images/loginLabelPass_de.png"
alt="Passwort" />
</label>
<input id="Password" name="j_password" type="password" value=""/></div>
<input type="submit" class="hiddenSubmit"/>
<a href="#" title="Login" class="btnGreyTextured">
<span>Login</span>
</a>
</form>

Invalid Request Origin after Python post request

I've been trying to write a program which gets my school timetable which is locked behind a login screen, however when I run my python script I am greeted by:
Sorry, we are unable to complete this request
An error occured while submitting the form:
Invalid Request Origin
This is the code I've written so far:
import requests
login_url = 'https://WEBSITE/login?page=%2Fhomepage'
timetable_url = 'https://WEBSITE/timetable'
payload = {
'username': 'USERNAME',
'password': 'PASSWORD'
}
with requests.Session() as session:
post = session.post(login_url, data=payload, verify = True)
r = session.get(timetable_url)
print (post.text)
And this is the html from the login page:
<div id="content">
<form id="login" method="post" action="/login?page=%2Flogin%2F">
<input type="hidden" name="page" value="/login/" />
<div class="row collapse">
<div class="small-12 column">
<div class="logo">
<img src="/images/logo.php?logo=skin_logo_login&size=normal" alt="Schoolbox" />
</div>
</div>
<div class="small-12 column">
<p id="error-msg" class="alert-box alert"></p>
</div>
<div class="small-12 column">
<input type="text" name="username" id="username" placeholder="Username" />
<input type="password" name="password" id="password" placeholder="Password" />
</div>
</div>
<div class="row collapse">
<div class="small-12 column">
<p>Forgotten your password?</p>
</div>
<div class="small-12 column">
<input id="rememberme" type="checkbox" name="rememberme" value="1" />
<label for="rememberme">Remember me</label>
</div>
<div class="small-12 column login-links">
<input type="submit" name="Submit" value="Login" alt="Login" />
</div>
</div>
</form>
</div>
How exactly would I be able to solve this error, or would there be any other more viable solutions? Thank you.

This error Invalid Request Origin means that your origin does not match what is restricted by the server. You could try adding the Origin: header specifying your timetable_url, but it is likely that the page can only be called by another page on the server.

Login automatically using python

So I've seen that there are a couple of similar questions but none of them worked for me. This is the login page that I'm trying to log in to:
https://cas.lu.se/cas/login?service=http%3A%2F%2Fwww.ceq.lth.se%2Frapporter%2F%3Flasar_lp%3Dalla%26program%3Dsamtliga%26kurskod%3DEDAA45%26sort%3Dkurskod
The form has the following html:
<div class="box fl-panel" id="login">
<form action="/cas/login;jsessionid=C32FA3678B0093A4DB474BFD56858B6C?service=http%3A%2F%2Fwww.ceq.lth.se%2Frapporter%2F%3Flasar_lp%3Dalla%26program%3Dsamtliga%26kurskod%3DEDAA45%26sort%3Dkurskod" class="fm-v clearfix" id="fm1" method="post">
Please enter your userid without "#lu.se" at the end.<br/><br/>
<div class="row fl-controls-left">
<label class="fl-label" for="username"><span class="accesskey">U</span>sername:</label>
<input accesskey="u" autocomplete="false" class="required" id="username" name="username" onblur="makeLowercase();" size="25" tabindex="1" type="text" value=""/>
</div>
<div class="row fl-controls-left">
<label class="fl-label" for="password"><span class="accesskey">P</span>assword:</label>
<input accesskey="p" autocomplete="off" class="required" id="password" name="password" size="25" tabindex="2" type="password" value=""/>
</div>
<div class="row btn-row">
<input name="lt" type="hidden" value="LT-61720-irxrtm4JCzVNIJ2pkpidljNGOw5KYX"/>
<input name="execution" type="hidden" value="e1s1"/>
<input name="_eventId" type="hidden" value="submit"/>
<input accesskey="l" class="btn-submit" name="submit" tabindex="4" type="submit" value="LOGIN"/>
<input accesskey="c" class="btn-reset" name="reset" tabindex="5" type="reset" value="CLEAR"/>
</div>
</form>
</div>
So far I've tried using urllib in the following way:
from urllib import request, parse
auth = parse.urlencode({"username": "??",
"password": "??",
"lt": "LT-61720-irxrtm4JCzVNIJ2pkpidljNGOw5KYX",
"execution": "e1s1",
"_eventId": "submit",
"submit": "LOGIN", })
auth = auth.encode("ascii")
req = request.Request(url,
with request.urlopen(url) as f:
res = f.read()
The problem is that the only thing I get back is the html for the original page itself, and not the page that it is supposed to load after authentication. I'm also not sure what the action attribute contains in the form tag...

Failed to log into Google using Python

Can't figure out why Google won't let me log into my account. I put my email inside a variable called my_mail then search for it in the html but the results came back false.
I am using the requests & beautifulsoup modules to do so.
What I have:
from bs4 import BeautifulSoup
import requests
form_data={'Email': 'ExampleEmail#gmail.com', 'Passwd': 'Password'}
post = "https://accounts.google.com/signin/challenge/sl/password"
my_mail = 'ExampleEmail#gmail.com'
with requests.Session() as s:
soup = BeautifulSoup(s.get("https://accounts.google.com/ServiceLogin?elo=1").text, "html.parser")
for inp in soup.select("#gaia_loginform input[name]"):
if inp["name"] not in form_data:
form_data[inp["name"]] = inp["value"]
s.post(post, form_data)
print(my_mail in "https://mail.google.com/mail/u/0/#inbox")
Form info:
<form novalidate="" method="post" action="https://accounts.google.com/signin/challenge/sl/password" id="gaia_loginform">
<input name="Page" type="hidden" value="PasswordSeparationSignIn">
<input type="hidden" name="GALX" value="c90FYb-yd6E">
<input type="hidden" name="gxf" value="AFoagUVMILWKQvqlGV0i8sJG83jiQlrWzg:1475424777331">
<input id="profile-information" name="ProfileInformation" type="hidden" value="">
<input type="hidden" id="_utf8" name="_utf8" value="☃">
<input type="hidden" name="bgresponse" id="bgresponse" value="js_disabled">
<input type="hidden" id="pstMsg" name="pstMsg" value="1">
<input type="hidden" id="dnConn" name="dnConn" value="">
<input type="hidden" id="checkConnection" name="checkConnection" value="youtube:230:1">
<input type="hidden" id="checkedDomains" name="checkedDomains" value="youtube">
<div class="form-panel first valid" id="gaia_firstform">
<div class="slide-out ">
<div class="input-wrapper focused">
<div id="identifier-shown">
<div>
<label class="hidden-label" for="Email">
Enter your email</label>
<input id="Email" type="email" value="" spellcheck="false" name="Email" placeholder="Enter your email" autofocus="">
<input id="Passwd-hidden" type="password" spellcheck="false" class="hidden">
</div>
</div>
<span role="alert" class="error-msg" id="errormsg_0_Email"></span>
</div>
<div style="display: none" id="identifier-captcha">
<input type="hidden" name="identifiertoken" id="identifier-token" value="">
<input type="hidden" name="identifiertoken_audio" id="identifier-token-audio">
<div class="audio-box">
<div id="playIdentifierAudio"></div>
</div>
<div id="captcha-box" class="captcha-box">
<div id="captcha-img" class="captcha-img" data-alt-text="Visual verification">
</div>
<span class="captcha-msg">
Letters are not case-sensitive
</span>
</div>
<label for="identifier-captcha-input" class="hidden-label"></label>
<input type="text" id="identifier-captcha-input" name="identifier-captcha-input" class="captcha" placeholder="Enter the letters above" title="Type the characters you see or numbers you hear">
</div>
<input id="next" name="signIn" class="rc-button rc-button-submit" type="submit" value="Next">
<a class="need-help" href="https://accounts.google.com/signin/usernamerecovery?hl=en">
Find my account
</a>
</div>
</div>
<a href="https://accounts.google.com/ServiceLogin" tabindex="-1">
<img id="back-arrow" class="back-arrow " aria-label="Back" tabindex="0" alt="Back" src="https://www.gstatic.com/images/icons/material/system/1x/arrow_back_grey600_24dp.png">
</a>
<div class="form-panel second">
<div class="slide-in hide-form">
<div>
<p id="profile-name"></p>
<span id="email-display"></span>
</div>
<div>
<div id="password-shown"></div>
</div>
<input id="signIn" name="signIn" class="rc-button rc-button-submit" type="submit" value="Sign in">
<label class="remember">
<input id="PersistentCookie" name="PersistentCookie" type="checkbox" value="yes" checked="checked">
<span>
Stay signed in
</span>
<div class="bubble-wrap" role="tooltip">
<div class="bubble-pointer"></div>
<div class="bubble">
For your convenience, keep this checked. On shared devices, additional precautions are recommended.
Learn more
</div>
</div>
</label>
<input type="hidden" name="rmShown" value="1">
<a id="link-forgot-passwd" class="need-help" href="https://accounts.google.com/signin/recovery?hl=en">
Forgot password?
</a>
</div>
</div>
<span id="inge" style="display: none" role="alert" class="error-msg">
Sorry, Google doesn't recognize that email. Create an account using that address?
</span>
<span id="timeoutError" style="display: none" role="alert" class="error-msg">
Something went wrong. Check your connection and try again.
</span>
</form>
I don't plan on using Google's API. I don't want to be restricted.
Why is the login failing? What could I do to fix it?
I need to login to scrap some information that requires a login.
Pyhton 3.4.2

How to send login request in python?

i am trying this python script:
import httplib
import urllib
conn = httplib.HTTPConnection("moodle.tau.ac.il")
conn.request("POST" , "/hu11/?lang=en_utf8&username=bugs&password=bunny" )
response = conn.getresponse()
print response.status, response.reason
print response.read()
conn.close()
but it doesn't log in the site as i want to. i want to build some script to help a friend to download all the files from his courses.
what do i do wrong in setting a post request to the server?
i have this form tag in site i would like to login to with python script:
<form action="" method="post" id="login">
<div class="loginform">
<div class="form-label"><label for="username">Username</label></div>
<div class="form-input">
<input type="text" name="username" id="username" size="8" maxlength=8 value="" />
</div>
<div class="clearer"><!-- --></div>
<div class="form-label"><label for="password">Password</label></div>
<div class="form-input">
<input type="password" name="password" id="password" size="5" value="" />
<br /><br />
<input type="submit" value="Login" />
<input type="hidden" name="testcookies" value="0" />
</div>
<div class="clearer"><!-- --></div>
</div>
</form>

Try BeautifulSoup or Mechanize.