I'm trying to use Mechanize to get emails from my Outlook web client, but I'm having troubles logging in. It gives me the errors listed below. I've verified that the user name and password are correct. Any ideas?
Here is my code:
import mechanize
b = mechanize.Browser()
cj = cookielib.LWPCookieJar()
b.set_cookiejar(cj)
b.open('https://mail.example.com/owa/')
br.select_form("logonForm")
b['username'] = 'myname'
b['password'] = 'password'
b.submit()
I can see that form components are being accessed correctly, but after submitting, the login page displays again with two errors:
The user name or password you entered isn't correct. Try entering it again.
Please enable cookies for this Web site.
I thought the b.set_cookiejar(cj) would take care of cookies. Could this be the root of my problem?
import mechanize
import cookielib
br = mechanize.Browser()
br.set_handle_robots( False )
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
br.open('https://webmail.server.com')
br.select_form(nr = 0)
br.form['username'] = 'username'
br.form['password'] = 'password'
br.submit()
Use this it works for me
Related
This website http://wwwhousemag.com.br/vote use facebook api to get some data, i'm trying to authenticate with facebook using mechanize but i'm getting this response:
Warning: Cannot modify header information - headers already
sent by (output started at
/home/house938/public_html/www/includes/class.php:130) in
/home/house938/public_html/www/top50djs/ver_topc4.php on line
5
code
import mechanize
from bs4 import BeautifulSoup
import Cookie
import cookielib
cookiejar =cookielib.LWPCookieJar()
#Facebook login
url = 'https://m.facebook.com'
username = 'mail'
password = 'pass'
browser = mechanize.Browser()
browser.set_cookiejar(cookiejar)
browser.set_handle_robots(False)
browser.set_handle_redirect(True)
browser.addheaders = [('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6)')]
browser.open(url)
browser.select_form(nr=0)
browser.form["email"] = username
browser.form["pass"] = password
browser.submit()
soup = BeautifulSoup(browser.response().read(),'lxml')
name = soup.find('img', class_="by img")
print name['alt'] #returning usename [ok]
url2 = 'http://www.housemag.com.br/www/top50djs/index.php?pg=ver_topc4'
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6)')]
response = browser.open(url2)
print response.get_data() #print erro response like user not logged in
This page http://www.housemag.com.br/www/top50djs/index.php?pg=ver_topc4 use facebook name to fill forms.
When i try to login with this link https://www.facebook.com/login.php?skip_api_login=1&api_key=1380514122278764&signed_next=1&next=https%3A%2F%2Fwww.facebook.com%2Fv2.5%2Fdialog%2Foauth%3Fredirect_uri%3Dhttp%253A%252F%252Fwww.housemag.com.br%252Fwww%252Ftop50djs%252Findex.php%253Fpg%253Dver_topc3%26state%3Ddedd3b4ff47a4fdf9d8055813e45e06f%26scope%3Demail%26response_type%3Dcode%26client_id%3D1380514122278764%26ret%3Dlogin%26sdk%3Dphp-sdk-5.0.0%26logger_id%3D5716f77d-48a6-a3dd-8ad6-791376d2176e&cancel_url=http%3A%2F%2Fwww.housemag.com.br%2Fwww%2Ftop50djs%2Findex.php%3Fpg%3Dver_topc3%26error%3Daccess_denied%26error_code%3D200%26error_description%3DPermissions%2Berror%26error_reason%3Duser_denied%26state%3Ddedd3b4ff47a4fdf9d8055813e45e06f%23_%3D_&display=page&locale=pt_BR&logger_id=5716f77d-48a6-a3dd-8ad6-791376d2176e
I got a security message on facebook.
How to open this page http://www.housemag.com.br/www/top50djs/index.php?pg=ver_topc4 with facebook user authenticated?
i'm building a python app that is using a login interface to automatically sign in to facebook by passing the users login data directly. Main Goal is to get the access_token from facebook.
the facebook login skript looks like this:
import mechanize
br = mechanize.Browser()
br.set_handle_equiv( True )
br.set_handle_gzip( True )
br.set_handle_redirect( True )
br.set_handle_referer( True )
br.set_handle_robots( False )
br.set_handle_refresh( mechanize._http.HTTPRefreshProcessor(), max_time = 1)
br.addheaders = [ ( 'User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1' ) ]
user = 'me#mymail.com'
password = 'myPass'
# note that client_id is pseudo-nr
url = 'https://m.facebook.com/dialog/oauth?client_id=123456789012345&redirect_uri=https://www.facebook.com/connect/login_success.html&response_type=token'
#Open URL and submit
br.open(url)
br.select_form(nr=0)
br.form['email'] = user
br.form['pass'] = password
br.submit()
result = br.geturl()
print result
Actually the script is working pretty fine, but there seems to be a redirecting problem after
br.submit()
When logging in to the given URL via Firefox, after manually adding username/password and pressing the submit button, the following URL is called:
https://www.facebook.com/login.php?login_attempt=1&next=https%3A%2F%2Fwww.facebook.com%2Fv2.0%2Fdialog%2Foauth%3Fredirect_uri%3Dhttps%253A%252F%252Fwww.facebook.com%252Fconnect%252Flogin_success.html%26response_type%3Dtoken%26client_id%3D123456789012345%26ret%3Dlogin&lwv=100
this is the same URL, that is given by br.submit() command.
unfortunately Firefox is performing another redirect, first to:
https://www.facebook.com/v2.0/dialog/oauth?redirect_uri=https%3A%2F%2Fwww.facebook.com%2Fconnect%2Flogin_success.html&response_type=token&client_id=123456789012345&ret=login&ext=143846220&hash=KefDdKCLQeJo_hJR
followed by the final destination:
https://www.facebook.com/connect/login_success.html#access_token=CAAGXPLMhY0cYMlFbAPXJnPKtEPPQCP4NNZCyJkiRLE1B0iHeK6KonmgsyrzQu4uJbfe1TDV1YrmtyFb3SuYJAVhZBazHDr49oYh3i5WThZAfgTDmtHFsmpvn3ZBpeJ3NR718Dn9rm0PX4ZCpsBA6wAItUdHDcAiOZBIZACliuVEefqseFXtqCNFN2ZC9UGslohsZD&expires_in=3780
I don't know why mechanize is not performing this redirect. Any ideas how to solve this?
I've got a script set to log into a website. The challenge is that I'm running the script on EC2 and the website is asking for me to do additional verification by sending me a custom code.
I receive the email immediately but need to be able to update that field on the fly.
This is the script
import urllib2
import urllib2
import cookielib
import urllib
import requests
import mechanize
from bs4 import BeautifulSoup
# Browser
br = mechanize.Browser()
# Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
# Browser options
br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_refresh(False)
br.set_handle_referer(True)
br.set_handle_robots(False)
# Follows refresh 0 but not hangs on refresh > 0
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
# User-Agent (this is cheating, ok?)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
# The site we will navigate into, handling it's session
br.open('https://www.website.com/login')
#select the first form
br.select_form(nr=0)
#user credentials
br['user_key'] = 'username#gmail.com'
br['user_password'] = 'somePassw0rd'
# Login
br.submit()
#enter verification code
input_var = raw_input("Enter something: ")
#put verification code in form
br['Verication'] = str(input_var)
#submit form
br.submit()
The challenge for me is that I keep getting an error saying:
AttributeError: mechanize._mechanize.Browser instance has no attribute __setitem__ (perhaps you forgot to .select_form()?)
What can I do to make this run as intended?
after you br.submit() you go straight into
br['Verication'] = str(input_var)
this is incorrect since using br.submit() will make your browser not have a form selected anymore.
after submitting i would try:
for form in br.forms():
print form
to see if there is another form to be selected
read up on the html code on the login site and check to see exactly what happens when you click login. You may have to reselect a form on that same page then assign the verification code to one of the controls
I'm learning Mechanize/BeautifulSoup by following the instruction on this site: http://stockrt.github.io/p/handling-html-forms-with-python-mechanize-and-BeautifulSoup/
Instead of writing a login and check mail for Gmail, I want to do the same for hotmail. However, I cannot find the login form by using Mechanize. The result when im trying to do the following is empty:
for f in self.br.forms():
print 'form:', f
Those codes print out nothing. By inspecting the hotmail website (login.live.com) I can see the form name is 'F1' and 2 fields are 'username' and 'passwd' but Mechanize cannot be able to catch those.
Here is the complete code:
import os
import mechanize
import cookielib
from bs4 import BeautifulSoup
import html2text
import agent
class Hotmail:
def __init__(self):
self.br = mechanize.Browser()
cj = cookielib.LWPCookieJar() # Cookie Jar
self.br.set_cookiejar(cj)
# Browser options
self.br.set_handle_equiv(True)
#self.br.set_handle_gzip(True)
self.br.set_handle_redirect(True)
self.br.set_handle_referer(True)
self.br.set_handle_robots(False)
self.br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1) # Follows refresh 0 but not hangs on refresh > 0
self.br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')] # User-Agent
def login(self, user, pswd):
self.br.open('http://login.live.com')
for f in self.br.forms():
print 'form:', f
if __name__ == '__main__':
test = Hotmail()
test.login('asd','asd')
I also tried to manually set the form 'F1' to br.form and fill out the values for it, but it doesn't work.
Any ideas why Mechanize cannot recognize any forms on Hotmail or how to get those forms?
Thank you so much!
I've been successfully scraping a website using mechanize, but I've had some problems with the page.open getting stuck (and not giving a timeout error) so I'd like to try and perform the same scrape with Requests. However, I can't figure out how to select a form to enter my login credentials. Here is the working code in mechanize:
# Browser
br = mechanize.Browser()
# Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
# Browser options
br.set_handle_equiv(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
br.set_proxies({"https": "198.102.28.100:7808", "http": "54.235.92.109:8080"})
# Open Login Page
br.open("https://example.com/login/Signin?")
br.select_form(name="signinForm_0")
br["loginEmail"] = "username"
br["loginPassword"] = 'password'
br.method = "POST"
br.submit()
#Open Page
URL = 'https://example.com'
br.open(URL, timeout=5.0)
I'm unsure how to replicate the br.select_form functionality using Python Requets. Does anyone have any ideas or experience doing this?
If I not wrong, Selenium is similar to Mechanize, but not Requests. Requests is used mostly to HTTP. Requests is similar to urllib or urllib2 but it is better. You can send request (GET or POST) and read html file from server but you need other modules to get some element on page - BeautifulSoup, lxml, pyQuery