I am trying to make a "proxy" in Python that allows the user to route all of their web traffic through a host machine (this is mainly for me and a couple people I know to confuse/avoid hackers and/or spies, who would only see web pages and so on coming in through one IP). I have run into several difficulties. The first is that I would like to be able to use the final, compiled product with Firefox, which can be set to route all of its traffic through an installed proxy program. I don't know what kind of configuration my proxy needs to have to do this. Second, the way the proxy works is by using urllib.requests.urlretrieve (yes, going to die soon, but I like it) to download a webpage onto the host computer (it's inefficient and slow, but it's only going to be used for a max of 7-10 clients) and then sending the file to the client. However, this results in things like missing pictures or broken submission forms. What should I be using to get the webpages right (I want things like SSL and video streaming to work as well as pictures and whatnot).
(wince) this sounds like "security through obscurity", and a lot of unnecessary work.
Just set up an ssh tunnel and proxy your web traffic through that.
See http://www.linuxjournal.com/content/use-ssh-create-http-proxy
Related
I have an unlimited internet connection in my house and a limited internet connection in the school.
I want to make a web browser (or something like that) that navigate from my house, get the data (including the streamings), and resends it to my browser in the school.
In Python, using WebKit, a web browser can be created easily and navigate youtube and other pages, I want to recreate that navigation in the other web browser (the one connected in my school).
School browser ⟶ send request to program or another Web browser ⟶ get page data (including streaming) ⟶ tunneling ⟶ sent to school browser.
It’s something like to do a remote web browser.
It sounds like you are trying to make a home private proxy server.
There are plenty of guides on how to do this but here's one I found by quickly looking around:
https://null-byte.wonderhowto.com/how-to/sploit-make-proxy-server-python-0161232/
Depending on your school's restriction method, a proxy server may not be enough to bypass their restrictions. You may also be able to overcome this by completely encrypting communications between your home network and your school system. To do this you would need to set up a home virtual private network (VPN). There are also many guides that you can use to achieve this.
I have an Arducam OV2640
There is a lot of documentation on how the code works wth a HTML and js script which calls the camera IP address, after arducam creates a web server.
This web server listens to get requests and sends a post response of the image.
The only problem is this is all over local wifi...source:
https://github.com/ArduCAM/Arduino/tree/master/ArduCAM/examples/ESP8266/ArduCAM_ESP8266_OV2640_Capture
Has anyone done anything such as sending to Imgur via api or for instance a cloud based program like python anywhere?
I've been trying different methods all day, but a lot it comes to people saying "you need to send each byte and the other side have a handler."
I'm asking because the local wifi method works so easily I thought there must be a way to extend this to external addresses in a nicer way?
Has anyone done or heard of anything like this?
Thanks
So i am using Scrapy to crawl some websites and i want to increase my privacy on the internet and also avoid getting banned so i read that i could achieve that by using premium proxy lists like http://www.ninjasproxy.com/ or http://hidemyass.com/ or VPN or Tor.
From what i understood a paid VPN would be a good option like the one http://hidemyass.com/ offers, but i can't seem to find any code that actually shows Scrapy integrating with a VPN like hidemyass.
I only saw an example like https://github.com/aivarsk/scrapy-proxies that shows how to use proxy lists.
How do i make Scrapy work with a VPN? If i can't are proxy lists good enough to maintain anonymity?
A VPN is something working system wide, not something that proxy selected traffic. All your internet traffic, browser, torrent, chat etc etc will be routed through the VPN, so just connect to the VPN and run the script.
I'm looking for a bit of web development advice. I'm fairly new to the area but I'm sure there are some gurus out there willing to part with some wisdom.
Objective: I'm interested in controlling a Python application on my computer from my personal web hosted site. I know, this question has been asked several times before but in each case the requirements were a bit different from my own. To reduce the length of this post I'll summarize my objective in a few bullet points:
Personal site is hosted by a web hosting company
Site uses HTML, PHP, MySQL, Python and JavaScript, the majority of everything is coded by me from the ground up
An application that is coded in Python will run on a PC within my home and will communicate with an Arduino board
The app will receive commands from the internet to control actuation via the Arduino, and will transmit sensor data back to the site (such as temperature)
Looking for the communication to be bi-directional, fast and secure
Securing the connection between site and Python app would be most ideal
I'm not looking to connect to the Python application directly, the web server must serve as the 'middle man'
So far I've considered HTTP Post and HTML forms, using sockets (Python app would run as a web server), an IRC bot and reading/writing to a text file stored on the web server.
I was also hoping to have a way to communicate with the Python app without needing to refresh the webpage, perhaps using AJAX or JavaScipt? Maybe with Flash?
Is there something I'm not considering? I feel like I'm missing something. Thanks in advance for the advice!
Just thinking out loud for how I would start out with this. First, regarding the website itself, you can just use what's easiest to you, or to the environment you're in. For example, a basic PHP page will do just fine, but if you can get a site running in Python as well, I'd prefer using the same language all over.
That said, I'm not sure why you would need to use a hosted website? Given that you're already forced to have a externally accessible PC at home for the communication, why not run a webserver on that directly (Apache, Nginx, or even something like CherryPy should do)? That webserver can then communicate with the python process that is running to control your Arduino (by using e.g. Python's xmlrpclib). If you would run things via the hosting company, you would still need some process that can handle external requests securely... something a webserver is quite good at. Just running it yourself gives you all the freedom you want, and simplifies things by lessening the number of components in your solution.
The updates on your site I'd keep quite basic: commands you want to run can be handled in the request handlers of the webserver by just calling the relevant (xmlrpclib) calls. Dynamically updating the page is best done by some AJAX calls I reckon. Based on your story, these updates are easily put in a JSON object, suitable for periodically updating only the relevant segments of your page.
I wanted to know if there was a way I can get my python script located on a shared web hosting provider to read the contents of a folder on my desktop and list out the contents?
Can this be done using tempfiles?
Server-side web scripts have no access to the client other than through requests. If you can somehow break through the browser's protection settings to get JavaScript, Java, or Flash to read the contents of the client then you stand a fighting chance. But doing so will make many people angry and is generally considered a bad idea.
Unless your desktop computer has a public, accessible IP, neither your app running on a shared web hosting provider, nor any other app and host on the internet, can get information from your desktop computer. Does your desktop computer fall within the tiny minority that does have such a public, accessible IP?
If not, and if you're willing to run the obvious risks involved of course, you can try turning the (probably dynamically assigned) IP address that your ISP gives you into a resolvable domain name, by working with such DNS providers as DynDNS -- it can be done for free.
Once you're past the hurdle of public accessibility, you need to run on your computer some server that can respond to properly authenticated requests by supplying the information you desire. For example, you could run a web server such as Apache (which is powerful indeed but perhaps a bit hard for you to set up), or the like -- and a custom app on top of it to check authentication and provide the specific information you want to make available.
If you have no privacy worry (i.e., you don't mind that any hacker in the world can look at that folder's contents), you can skip the authentication, which is the really delicate and potentially fragile part (given that there's really no way for your app, running on a shared web hosting provider, to hold "secrets" very effectively).
If you can clarify each of these issues, then we can help pinpoint the best approach (what to install and how on both your desktop computer, and that shared web hosting provider).