Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I'm relatively new to programming but I'm beginning work on a project that will be used in my day job. My end goal is to create a program that automatically downloads PDF's from a website once a month and puts them in specific directories on my PC. This is uncharted territory for me and so I'm trying to think through the steps.
The first problem I need to overcome is the website is password protected.
So I need to access the site then automatically enter a username and password and login. I've read Selenium would probably be the module for this.
Can anybody advise please?
Sounds like Selenium is a good fit.
You can simulate button presses, key strokes, mouse swipes, so you can log in, navigate and download your files.
You can even make it run in the background if you make sure it does what it's supposed to do.
This is a fun example
Be careful though, the website may have a way of detecting it incorporated in it.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
So, I have made a python script that posts online ad on a certain website for me automatically (kind of web scraper). But, I need that ad to be posted (AKA that script ran) every day at approximately 12h, so the ad wouldn't get pushed too far so the people could see it constantly.
The only problem is that I don't have access to my PC every day at that time (I'm usually busy), so I need a website that I can post my script to that will execute it every day at a given time. Is there a website that I can use for this purpose?
You can try pythonanywhere. But most of them has very limited services allowed when you are not paying. Also troublesome if you need to expose some of your permissions.
Personally, I run services on my personal box, usually keep it on all-day and a scheduler software to run my programs from an starting script.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am quite fluent in Python, but have only really used it for data analysis.
I would like to learn how I can interact with webpages. For instance, I want to start by writing a code that will press a button on a webpage for me.
I just don't know where to start or what to google to find resources about this.
Could anyone point me out in the right directions, or suggest some key words that I could search for?
Thanks.
Have you tried to use the pyautogui module, which allows you to programmatically control the mouse and the keyboard?
An example of an automatic form filler on a webpage is available in chapter 18 of the Automate the Boring Stuff with Python book.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Suppose there is a password-protected website that I want to access to scrape some info from it and put it into a spreadsheet. For example, it could be my personal credit card account page and I would be scraping info about the latest transactions.
A variation of this would be if the site allowed to download the transaction info as a CSV file, in which case I would want to download that file.
If I want to write such scraper in Python, what packages should I use for the task? Does it depend on how a specific website is implemented, i.e. I might need one tool to scrape one site and another tool to scrape another.
Thank you
I actually did something very similar to this, but in node. Are you definitely wanting to do this in Python?
If you want to stick to Python, take a look at these modules:
BeautifulSoup
requests
Someone wrote a really awesome module combining the above two modules:
Robobrowser
If you would like to venture down the node route, take a look at this:
nightmarejs
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
My lead coder and I want to hack a quick python program together tomorrow. Is there a site that will allow us, like Google Docs, to work on a program at the same time and run the code?
I have recently started using PythonAnywhere. It lets you start a console on your account, and share it with others. The site is currently in limited-beta, but I got an invite just a few hours after my request. When you share a console, the friend doesn't need an account.
I think Cloud 9 will do what you want, but it's not free unless you're open source.
For Mac users, there is a collaborative editor that should work for hacking together some Python. http://www.codingmonkeys.de/subethaedit/
Also, google turned up http://gobby.0x539.de/trac/ but I've never tried that.
Of course, each of you editing the script will still have to run it locally. ;)
There is such functionality at rextester.
Github?
It's not quite the same (not instant), but works well.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I am using Python to do automated tests for IE. Currently I use PAMIE. It can click links and find words very well if a web page has no iframes. However, some of my tested pages include several iframes, and PAMIE seems can't handle it correctly. It can't find the text strings in the iframes.
Does anyone recommend some Python libraries instead of PAMIE? Or is there a way to access iframes in a web page with PAMIE？ It's better if there is an example.
Thanks!
Use Selenium with it's Python client binding. Selenium can let you record your own workflow or write your own script in one of its supported languages. The tests are conducted on an actual browser as opposed to a simulated one. There's no system can top it at the moment.