I'm trying to fill textboxes in this site but mechanize can't find any form.Therefore I can't do anything on thoose textboxes cause they're in non-exists form.
I tried so different solutions but i failed.Here is my code :
import mechanize
def kota():
br = mechanize.Browser()
br.open("http://www.kentkartim.com/bakiyesorgula.php")
a = br.select_form(nr=0)
print(a)
With this I get this result :
mechanize._mechanize.FormNotFoundError: no form matching nr 0
I'm thinking that the reason of the result is the webpage using javascript. I need to fill the textboxes and submit them.
Related
I've had success using mechanicalsoup with single pages / single forms, but am having difficulty with a multistep problem. The pages I am attempting to navigate start here: https://webapps2.ncua.gov/CustomQuery/CUSelect.aspx
I get through the first page/form, but then I am not sure how to deal with the second page/form. The third page includes the result that I wish to scrape.
import requests
import urllib.parse
import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://webapps2.ncua.gov/CustomQuery/CUSelect.aspx")
form=browser.select_form()
browser["operand0"] = "State"
browser["operator0"] = "Not Equal"
browser["value0"] = "XX"
response = browser.submit_selected()
form2 = browser.get_current_form()
submit = browser.get_current_page().find('input', id='BtnAllAcct')
form2.choose_submit(submit)
browser.submit_selected()
submit = browser.get_current_page().find('input', id='Btndata1')
form2.choose_submit(submit)
browser.submit_selected()
Any ideas? This is my second attempt after first trying to interact with the API, but two separate forms is stumping me on that as well.
I was solving a similar issue, following the advice in MechanicalSoup follow a link without inside a button I switched to selenium
I'm trying to write a Python code to submit a simple form.
http://stulish.com/soumalya01
[Edit : http://travelangkawi.com/soumalya01/
When you use this link it returns a different page on form submit. Is good for debugging]
Any code would do
Tried both mechanize and mechanical soup. Both are unable to handle the text fields. It does not have a name only ID. But we are unable to get the element by ID
Any Code would do as long as it works. (Fill ABC in the text box and hit submit)
I just followed the documentations of mechanize. See sample code below:
from mechanize import Browser
br = Browser()
br.open('http://stulish.com/soumalya01')
br.select_form(nr=0)
form.set_all_readonly(False) #add this
br.form.set_value('ABC', nr=1)
print(br.form.controls[1])
br.submit()
I am trying to scrape data from this website. To access the tables, I need to click the "Search" button. I was able to successfully do this using mechanize:
br = mechanize.Browser()
br.open(url + 'Wildnew_Online_Status_New.aspx')
br.select_form(name='aspnetForm')
page = br.submit(id='ctl00_ContentPlaceHolder1_Button1')
"page" gives me the resulting webpage with the table, as needed. However, I'd like to iterate through the links to subsequent pages at the bottom, and this triggers javascript. I've heard mechanize does not support this, so I need a new strategy.
I believe I can get to subsequent pages using a post request from the requests library. However, I am not able to click "search" on the main page to get to the initial table. In other words, I want to replicate the above code using requests. I tried
s = requests.Session()
form_data = {'name': 'aspnetForm', 'id': 'ctl00_ContentPlaceHolder1_Button1'}
r = s.post('http://forestsclearance.nic.in/Wildnew_Online_Status_New.aspx', data=form_data)
Not sure why, but this returns the main page again (without clicking Search). Any help appreciated.
I think you should look into scrapy
you forgot some parameters in ths post request:
https://www.pastiebin.com/5bc6562304e3c
check the Post request with google dev tools
I am working on a project and I need to validate a piece of data using a third party site. I wrote a python script using the lxml package that successfully checks if a specific piece of data is valid.
Unfortunately, the site does not have a convenient url scheme for their data and therefor I can not predict the specific url that will contain the data for each unique request. Instead the third party site has a query page with a standard html text input that redirects to the proper url.
My question is this: is there a way to input a value into the html input and submit it all from my python script?
Yes there is.
Mechanize
Forms
List the forms
import mechanize
br = mechanize.Browser()
br.open(url)
for form in br.forms():
print "Form name:", form.name
print form
select form
br.select_form("form1")
br.form = list(br.forms())[0]
login form example
br.select_form("login")
br['login:loginUsernameField'] = user
br['login:password'] = password
br.method = "POST"
response = br.submit()
Selenium
Sending input
Given an element defined as:
<input type="text" name="passwd" id="passwd-id" />
you could find it using any of:
element = driver.find_element_by_id("passwd-id")
element = driver.find_element_by_name("passwd")
element = driver.find_element_by_xpath("//input[#id='passwd-id']")
You may want to enter some text into a text field:
element.send_keys("some text")
You can simulate pressing the arrow keys by using the “Keys” class:
element.send_keys("and some", Keys.ARROW_DOWN)
These are the two packages I'm aware of that can do what you've asked.
I am trying to submit a form, and get the results of the page that it heads to after submitting the form. I'm using mechanize.
1) When I'm using the code to click on the first-button, it is getting a response. But when I read the response, it is showing the source of the same page (the page where the form is located). Not of the page that the browser is redirected to after the submission of the form.
from mechanize import Browser
br = Browser()
br.open("http://link.net/form_page.php")
br.select_form(nr=0)
br.form['number'] = '0123456789'
response = br.submit(nr=0)
print response.read()
Now, when I do this, the source of the same page (i.e. form_page.php) is showing up. But, it should have shown the source of "results.php" (that is where the browser leads to when I do it manually)
2) There are multiple submit buttons in the page. I am clicking only the first one. But when I'm trying to click other submit buttons using nr=1 or nr=2, it is showing this error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/mechanize /_mechanize.py", line 524, in select_form
raise FormNotFoundError("no form matching "+description)
mechanize._mechanize.FormNotFoundError: no form matching nr 1
Can you please help me?
make sure you are selecting right form or make sure there is a form that you are selecting on the web page. you can check it by like this code :
for form in br.forms():
print form
and see what result returned to you.
This looks similar to this issue, where submit was calling some Javascript to validate the inputs before redirecting. It may be worth having a look at the HTML of the page and checking what it does on submit.
Try the following:
import mechanize
br = mechanize.Browser()
br.open("http://link.net/form_page.php")
br.select_form(nr=0)
br['number'] = '0123456789' ### try instead of 'br.form[]'
response = br.submit() ### no need to specify form again
text = response.read()
Don't forget about 'br.set_handle_robots(False)', 'br.set_all_readonly(False)', etc...