So I have an authenticated site that I want to access via the mechanize module. I'm able to log in, and then go to the page I want. However, because the page recognizes that mechanize doesn't have javascript enabled, it wants me to click a submit button to get redirected to a non javascript part of the site. How can I simply click the button and then read the contents of the page that follows that?
Or, is there a way to trick it into thinking that my javascript is enables?
Thanks!
if that submit button is really a submit input element of the form, and the redirection works as usual form submit action, and provided that it's the only form in the page, your mechanize browser instance is br, following should work
br.select_form(nr=0) # select the first form
br.submit()
afaik, there's no simple or moderately possible way, how to emulate javascript in mechanize, possible workarounds depend on what is javascript exactly doing
Related
Currently I am testing my flask app using selenium. So I have an html with simple FlaskForm in it which contains submit button. Of course I can click on it and submit a form using
btn = driver.find_element_by_id("submit-btn").click()
But the problem is that click method doesn't return anything.
So my question is how can I get at least the response code of submitted form so I can be sure that my tests run ok?
No, click() does not return anything, and you cannot get response code by selenium as well.
You need to verify test based on what the web displays, like success alert, text or something.
In flask programming, people usually use 'url_for' such as {{url_for = 'some url'}}
This way have to make URL(#app.route), template(HTML) and map each other.
But, I just want to send email when I click submit button in HTML modal.
For this, There is no page reloading. To do this, I think I have to connect between button and python function without URL, return(response)
I wonder that how to make it, help me please, I'm beginner in flask programming.
You can't trigger anything on the server without making a request to a URL.
If you don't want the page to reload, you can either redirect back to the original page after your action is finished, or you can use Ajax to make the request without changing the page; but the request itself is always to a URL.
I need to login to a website. Navigate to a report page. After entering the required information and clicking on the "Go" button(This is a multipart/form-data that I am submitting), there's a pop up window asking me to save the file. I want to do it automatically by python.
I search the internet for a couple of days, but can't find a way in python. By using urllib2, I can process up to submitting multipart form, but how can I get the name and location of the file and download it?
Please Note: There is no Href associated with the "Go" button. After submitting the form, a file-save dialog popup asking me where to save the file.
thanks in advance
There are python bindings to Selenium, that are helping in scripting simulated browser behavior allowing to do really complex stuff with it - take a look at it, should be enough for what you need.
I need to create a script that will log into an authenticated page and download a pdf.
However, the pdf that I need to download is not at a URL, but it is generated upon clicking on a specific input button on the page. When I check the HTML source, it only gives me the url of the button graphic and some obscure name of the button input, and action=".".
In addition, both the url where the button is and the form name is obscured, for example:
url = /WebObjects/MyStore.woa/wo/5.2.0.5.7.3
input name = 0.0.5.7.1.1.11.19.1.13.13.1.1
How would I log into the page, 'click' that button, and download the pdf file within a script?
Maybe Mechanize module can help.
I think that url on clicking the button maybe generated using javascript.So, to run javascript code from python script take a look at Spidermonkey.
Try mechanize or twill. HttpFox or firebug can help you to build your queries. Remember you can also pickle cookies from browser and use it later with py libs. If the code is generated by javascript it could be possible to 'reverse engineer' it. If nof you can run some javascript interpret or use selenium or windmill to script a real browser.
You could observe what requests are made when you click the button (using Firebug in Firefox or Developer Tools in Chrome). You may then be able to request the PDF directly.
It's difficult to help without seeing the page in question.
As Acorn said, you should try monitoring the actual requests and see if you can spot a pattern.
If not, then your best bet is actually to automate a fully-featured browser, that will be able to run Javascript, so you'll exactly mimic what a regular user would do. Have a look at this page on the Python Wiki for ideas, check the section Python Wrappers around Web "Libraries" and Browser Technology.
I'm trying to fill forms of an ajax box (just my term for those several forms) using the mechanize module, but it seems not to work. I'm not a web programmer but afaik the ajax box updates itself 'onchange' with an event which is handled by the browser.
Mechanize seems not to handle that, in the links list (from the iterator Browser.links) I can find a url 'javascript:AjaxRetry();' with an error msg as text which tells me that something has gone wrong.
Here is my code:
import mechanize as m
br = m.Browser()
br.open(url)
br.select_form(nr=0)
# fill in one form (in a real browser, the other form refresh and are not disabled anymore)
br.set_value(code, br.form.controls[10].name)
# how to make it refresh now?
#br.submit() doesn't work (also br.click() does not work (no clickable around at all))
Is mechanize the right module to fill forms of that ajax box?
I can't paste the link to the page where that ajax box is, because you have to be logged-in in order to see that box.
Mechanize doesn't handle javascript, see this answer for more details and an alternative solution How to properly use mechanize to scrape AJAX sites