I have been using selenium to do some web scraping and I'm in need for changing my ip. After having done some reserach into this I have discovered that it is fairly easy to setup and use a proxy. However, I am already paying for a VPN and therefore I would like to use it for this application as well. The free proxy lists that I have found have been way to slow to be useful for me.
I did some googling and found vpnc and other libraries but I couldn't get it to work all the way. I'm fairly new to web scraping and python so therefore I would appreciate if someone could help me on my level of knowledge.
Is it possible to do this or am I trying to achieve something that is way to difficult for an amateur like me? I'm trying to set this up on MacOS as well as Windows 7.
Related
So I recently finished my python project, grabbing values from an API and put it into my website.
Now I have no clue how I actually start the website (finding a host) and making it accessible to other people, I thought turning to here might find the solution.
I have done a good amount of research, tried "pythonanywhere" and "google app engine" but seem to not really find a solution.
I was hoping to be able to use "hostinger" as a host, as they have a good price and a good host. Contacted them but they said that they couldn't, though I could upload it to a VPS (which they have). Would it work for me to upload my files to this VPS and therefor get it to a website? or should I use another host?
I just deployed my first app and used Heroku. I'm very happy with it so far and it was fairly straight forward to set up. It does require a basic git understanding but was not hard to learn
A VPS would work, but you'll need to understand basic linux server admin to get things setup properly.
Sounds like you don't have any experience with server admin, so something like App Engine would be great for you. There are a ton of tutorials on the internet for deploying flask to GAE.
I've got a python script that I want to build a locally-hosted web gui app for, so I can use the modern styling and tools available to web apps.
The scripts I'm running take a while to process, and I want to update the web app with visual updates, or at least something akin to what the console sees when using print() in python.
My initial hosting efforts have been based on this tutorial, and I tried out the methods in this answer to try and get data to update in a streamed fashion, but the pages only showed once the entire script was finished.
I'm wondering whether web.py could help me?
Any guidance, or even the right terms to google would be appreciated. Thanks.
--
Update: I've been reading up on node.js (something I've failed to do for years..) and, please correct me if I'm wrong, but it seems like it could be the answer. I'm even considering re-writing my original functions into node.js given the existence of this serial comms library
I have tried searching online like crazy with no avail. PHP is as simple as naming the file .php and writing PHP. I know people say it's that simple for Python, but I have found no useful guides in setting it up. I merely want to practice Python on my computer via WAMP or another alternative. I am on Windows Vista.
I cannot get .py files to execute correctly. The actual text:
print("Hello!")
Appears just as that rather than "Hello!". I don't know what to do to make it actually work in my browser.
Any help or pointing towards guides would be greatly appreciated.
PHP does not execute in the browser. It is executed on the server side then the output is sent by the web server to the browser.
If you want a simple way to use Python to process web requests take a look at web.py (http://webpy.org).
Your server should handle Python code. Take a look at framework Django. And as for servers I can suggest you http://webfaction.com
I have developed a few python programs that I want to make available online.
I am new to web services, and I am not sure what I need to do in order to create a service where somebody makes a request to an URL (for example), and the URL triggers a Python program that displays something in the user's browser, or a set of inputs are given to the program via browser, and then python does whatver it is supposed to do.
I was playing with the google app engine, which runs fine with the tutorial, and was planning to use it becuase it looks easy, but the problem with GAE is that it does not work well (or does not work at all) with some libraries that I plan to use.
I guess what I am trying to do is some sort of API using my WebFaction account.
Can anybody point me in the right directions? What choices do I have in WebFaction? What are the easiest tools available?
Thank you very much for your help in advance.
Cheers
Well, your question is a little bit generic, but here are a few pointers/tips:
Webfaction allows you to install pretty much anything you want (you need to compile it / or ask the admins to install some CentOS package for you).
They provide some default Apache server with mod_wsgi, so you can run web2py, Django or any other wsgi frameworks.
Most popular Python web frameworks have available installers in Webfaction (web2py, django...), so I would recommend you to go with one of them.
I would also install supervisord to keep your service running after some reboot/crash/problem.
I would be glad to help you if you have any specific question...
I m trying to automate a Web Application validation performed by my team.I have choosen Python as the language to do this, although my exp. with Python is very limited.I have done similar things in the past using Perl. Now the problem is that after posting the url of the website it directs to a logon page which is made in Javascript. From whatever little Python I know, I believe scrapping/parsing website made in Javascript is not possible. I faced the same issue while doing this with Perl as well and wasn't able to proceed.
Any pointers or help in resolving the above issue would be highly appreciated.
Thanks
Spynner may help http://code.google.com/p/spynner/
Maybe you can take a look a Selenium. It's a firefox plugin that enables automation, but it also has a webdriver system where you can write automation scripts in various languages (including python), and a server execute the code in various browsers. I never tried the webdriver part myself, but that should do what you want.