I want to automatically save a file from a website. I don't know how to bypass the Download File prompt in python and save it directly to my c: drive.
Any help is appreciated,
Elliott
Modules like urllib2 and urlgrabber don't have a "Download File" prompt.
One idea is to use a module like mechanize to automate the query and the download.
Here you can find some documentation.
Usually when you post a question here at stackoverflow, it is a good idea to post an example at how you have tried to solve the problem, and then we will help you to improve it, rather than asking people to write the solution for you,
when you use mechanize, configure the UserAgent properly and respect the policy of the webmasters about scraping.
Related
I am writing a script to log into web pages (I know it's a bad idea, and I don't care), but I need to run it with a bookmark from Firefox. Does anyone know how to help me?
There is no pre-made solution and as it stands, you left part of your question ambiguous.
But here is one possible outline on how you could accomplish what you want from completely inside firefox.
First off you need to get python in the browser, but it is normally compiled into machine code. The solution is the Transcrypt project, which is Python is transcompiled into javascript and that script is then loaded into your browser. This allows you to run python code.
Extensions to python are made so it can access the DOM in a similiar way you can from javascript.
The transcrypt website has some examples, and some more can be found at a draft page at Mozilla.
Your bookmarklet would be to load a script to load transcrypt and feed it your python code, something like
javascript:(function(){var el=document.createElement('script');el.src='file:///something/something/transcript_loader_and_my_python_code.js';document.body.appendChild(el);})();
But the exact bookmarklet would depend on how you go about things. Writing transcript_loader_and_my_python_code.js is on your own, but if you get the examples from the links above working it should be trivial.
I'm practicing with using Python 3 and CGI to display pages in the browser. I'm running into an issue that I can't seem to figure out though.
There is an error somewhere in my python code I assume but I don't know of a way to figure out what it is.
Basically when I try to run the script the browser wants to download the python file instead. When I comment out a block of code that I suspect is causing the error, it display everything fine.
I have cgitb.enable() imported and included in the file but that's not displaying anything for this error.
Is there a way to see any python errors in the browser? Or at least be able to log the error to a text file that I can go and look at?
I suggest to use Sentry - https://www.getsentry.com/welcome/
It will give you the great interface for errors. You can use it for free.
Note: I am not affiliated with them, I just use that happily for our own solutions.
I have been trying to extract and scrape information about bugs filed for the joda-time software. However, I am finding no good way to export this information to a file using an automated script or something. I do find some python scripts online but they are mostly about converting sourceforge tickets to git or outdated.
Any help in that matter would be greatly appreciated.
Thanks,
Shivani
You can write a scraping script using BeautifulSoup. You can extract the necessary information from the HTML of each bug ticket page for the joda-time software by writing some Python code with BeautifulSoup. It's not difficult and you can then do with the extracted information what you want.
I'm sorry, I can't understand your question.
If your question is how to extract a file as a HTML and save as a HTML
You can use urllib : http://docs.python.org/library/urllib.html
If you are a project administrator, you have an export to xml option.
You also may find Migrate sourceforge tickets to GitHub issues useful.
So, I've been searching the topic for quite a while, but I am unable to find out whether it is possible or not. I suspect not, but I need to be sure.
What I'm doing:
I am building a fancy web app that uses xml, xsl and javascript. However, I need to do some transformation to a text file to convert it into a proper xml file. To do this, I use a python script.
What I need:
I am working to make as easy as possible the use of the app, but this gets kind of destroyed when the user has to execute a terminal script before anything. I need a graphical IDE for python, but I just don't have time to learn how to make one, and I would love that the entire app would work within the same web environment, much more intuitive for the user. Then, a really easy way to do that would be to use Firefox as the IDE, for example choosing the file to be transformed through it, and launching from there the python script.
What I require:
I need compatibility only with Firefox. But of course any system capable to install Python and Firefox should work as well. The user would happily install both programs, as long as they have interest in the app.
What I don't know:
Is that possible, to execute an external python script? If not, for security reasons I imagine, is there any way for the user to allow it? Maybe through the installation of a plugin?
maybe http://mozex.mozdev.org/ is what I am looking for?
Thanks!
Gerard
Yes, It is possible to execute an external python script. You can use a simpleServer (example) that receives a (Ajax) request from your page and calls a subprocess or os.startfile to launch the script. In fact, you can call the function itself from within the server and return the result (e.g. as json) to the page and use your javascript to handle it.
If you're after actually using python in firefox, rather than having the server side written in python (although the webserver can of course be running locally), I would guess pythonext (used to be pyxpcomext) is what you need.
I am interested in the Python one. I wish to automate some GUI under Windows. What is the best open source library for that with no strings attached? Thanks.
Try pyWinAuto.
There's WATSUP, but I've not tried it yet myself.
I've also heard of pyWinAuto, although the link I have is to pyWinAuto on SourceForge.