Develop Reference

Python - How to Send a Command to a Web Server

I have been in IT Support for many years but have always been interested in coding so I've started to train with Python. I'm working on a little coding project where I ask the user for some parameters (static IP to set for the camera, if the cam has a microphone, what they want camera to be named, etc) and then need to push these settings to an IP camera but I don't know how to "send" these commands to the IP of the camera.
For example, here's the command I run from a browser which will set the resolution on camera with IP 192.168.0.9x to 800x450:
http://192.168.0.9x/axis-cgi/param.cgi?action=update&Image.I0.Appearance.Resolution=800x450
How do I get Python to send these types of commands to a web server (the IP cam is essentially a web server)?
Thanks for any help :-)
L

Python-requests is an easy to use HTTP client.
To perform your request, start with:
import requests
params = { "Image.I0.Appearance.Resolution": "800x450",
"action": "update"
}
response = requests.get("http://192.168.0.9x/axis-cgi/param.cgi", params=params)

I think urllib2 is solution for your problem :)
import urllib2
content = urllib2.urlopen(some_url)

Look at requests. This is a third party library, but it's better than the built in urllib2.
import requests
r = requests.get(some_url)
Requests was developed with a few PEP 20 idioms in mind.
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Readability counts.

You could use webbrowser library in python to directly send the command to web page and control the camera. I used this for my FOSCAM FI8918W
for ur query it could be:
import webbrowser
webbrowser.open(http://192.168.0.9x/axis-cgi/param.cgi?action=update&Image.I0.Appearance.Resolution=800x450)
U can check the cgi commands from FOSCAM cgi reference guide

Python script through the Tor network

I have written a simple python script that fetches my ip.
import urllib
import socks
import socket
#set the proxy and port
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
#initialize the socket
socket.socket = socks.socksocket
#store the URL that we want
url = 'https://check.torproject.org/'
#open the URL and store it into 'response'
response = urllib.urlopen(url)
#parse the response
html = response.read()
#print to console
print html
Nothing too complex, however the problem starts when analyzing the response from check.torbrowser. The site will always give me an address that is different from my currently running Tor browser that is on the same page. However, the html response will say that I am being routed through the Tor network but it doesnt look to be coming from the 'standard' tor browser. The latter part I understand, though I did not include it in the code above, I was playing with User-Agent strings and other headers, so I will chalk it up to that being the primary cause. What I do not understand is where in the h-e-double hockey sticks did the IP come from that was served as a response from the py script?
My next question, which builds on top of all this, is how do I connect my python script to the tor network correctly? After a little googling, I found that tor will block traffic for everything other than the socks protocol and that an alternative is to use privoxy in conjunction with tor. My initial thought is to do some kind of routing that would result in the layering of software. In my mind, it would look like:
Python -> Privoxy -> Tor -> Destination
My end goal in all of this is to grab a .onion based address and save/read it. However, I have put that to the side after all of these problems started occurring. A little info to help get better answers: I am using a Windows machine, though I have a Linux one if there is some functionality that may be present there that would help this process, and I am using Python 2.7 though, again, this can be easily changed.
I would like to ask that the steps to make all this happen be laid out - or at least some links/direction, I am by no means afraid to read a few good blogs/tutorials about the subject. However, I feel like this is really a couple of seperate questions, and would require quiet a lengthy answer so I would be more than happy to just know that I am on the right path before I rip more of my hair out :)

Your code is correct, however your assumption that Tor will always give you the same IP address is not. Thanks to circuit isolation, a privacy feature of Tor that ensures isolation between the connections you open, you're routing the request through a different exit node than the Tor Browser will.
Reliably emulating the Tor Browser behavior is hard and I would recommend against it. Your method for connecting to the Tor network looks correct.
Tor will allow you to use any protocol you want, but yes you need to connect through the SOCKS protocol. That's fine though: almost all network protocols (http included) play nicely with SOCKS.

With torpy library you can renew circuits as you wish.
>>> from torpy.http.requests import TorRequests
>>>
>>> def show_ip(resp):
... for line in resp.text.splitlines():
... if 'Your IP address appears to be' in line:
... print(line)
...
>>> with TorRequests() as tor_requests:
... print("build circuit")
... with tor_requests.get_session() as sess:
... show_ip(sess.get("https://check.torproject.org/"))
... show_ip(sess.get("https://check.torproject.org/"))
... print("renew circuit")
... with tor_requests.get_session() as sess:
... show_ip(sess.get("https://check.torproject.org/"))
... show_ip(sess.get("https://check.torproject.org/"))
...
build circuit
<p>Your IP address appears to be: <strong>178.17.171.102</strong></p>
<p>Your IP address appears to be: <strong>178.17.171.102</strong></p>
renew circuit
<p>Your IP address appears to be: <strong>49.50.66.209</strong></p>
<p>Your IP address appears to be: <strong>49.50.66.209</strong></p>

Python urllib2 anonymity through tor

I have been trying to use SocksiPy (http://socksipy.sourceforge.net/) and set my sockets with SOCKS5 and set it to go through a local tor service that I am running on my box.
I have the following:
socks.setdefausocks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "localhost", 9050, True)
socket.socket = socks.socksocket
import urllib2
And I am doing something similar to:
workItem = "http://192.168.1.1/some/stuff" #obviously not the real url
req = urllib2.Request(workItem)
req.add_header('User-agent', 'Mozilla 5.10')
res = urllib2.urlopen(req, timeout=60)
And even using this I have been identified by the website, my understanding was that I would be coming out of a random end point every time and it wouldn't be able to identify me. And I can confirm if I hit whatsmyip.org with this that my end point is different every time. Is there some other steps I have to take to keep anonymous? I am using an IP address in the url so it shouldn't be doing any DNS resolution that might give it away.

There is no such User-Agent 'Mozilla 5.10' in reality. If the server employs even the simplest fingerprinting based on the User-Agent it will identity you based on this uncommon setting.
And I don't think you understand TOR: it does not provide full anonymity. It only helps by providing anonymity by hiding you real IP address. But it does not help if you give your real name on a web site or use such easily detectable features like an uncommon user agent.
You might have a look at the Design and Implementation Notes for the TOR browser bundle to see what kind of additional steps they take to be less detectable and where they still see open problems. You might also read about Device Fingerprinting which is used to identity the seemingly anonymous peer.

Python 3 - pull down a file object from a web server over a proxy (no-auth)

I have a very simple problem and I am absolutely amazed that I haven't seen anything on this specifically. I am attempting to follow best practices for copying a file that is hosted on a webserver going through a proxy server (which does not require auth) using python3.
i have done similar things using python 2.5 but I am really coming up short here. I am trying to make this into a function that i can reuse for future scripts on this network. any assistance that can be provided would be greatly appreciated.
I have the feeling that my issue lies within attempting to use urllib.request or http.client without any clear doc on how to incorporate the use of a proxy (without auth).
I've been looking here and pulling out my hair...
http://docs.python.org/3.1/library/urllib.request.html#urllib.request.ProxyHandler
http://docs.python.org/3.1/library/http.client.html
http://diveintopython3.org/http-web-services.html
even this stackoverflow article:
Proxy with urllib2
but in python3 urllib2 is deprecated...

here is an function to retrieve a file through an http proxy:
import urllib.request
def retrieve( url, filename ):
proxy = urllib.request.ProxyHandler( {'http': '127.0.0.1'} )
opener = urllib.request.build_opener( proxy )
remote = opener.open( url )
local = open( filename, 'wb' )
data = remote.read(100)
while data:
local.write(data)
data = remote.read(100)
local.close()
remote.close()
(error handling is left as an exercise to the reader...)
you can eventually save the opener object for later use, in case you need to retrieve multiple files. the content is written as-is into the file, but it may need to be decoded if a fancy encoding has been used.

Is it possible to fetch a https page via an authenticating proxy with urllib2 in Python 2.5?

I'm trying to add authenticating proxy support to an existing script, as it is the script connects to a https url (with urllib2.Request and urllib2.urlopen), scrapes the page and performs some actions based on what it has found. Initially I had hoped this would be as easy as simply adding a urllib2.ProxyHandler({"http": MY_PROXY}) as an arg to urllib2.build_opener which in turn is passed to urllib2.install_opener. Unfortunately this doesn't seem to work when attempting to do a urllib2.Request(ANY_HTTPS_PAGE). Googling around lends me to believe that the proxy support in urllib2 in python 2.5 does not support https urls. This surprised me to say the least.
There appear to be solutions floating around the web, for example http://bugs.python.org/issue1424152 contains a patch for urllib2 and httplib which purports to solve the issue (when I tried it the issue I began to get the following error instead: urllib2.URLError: <urlopen error (1, 'error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol')>). There is a cookbook recipe here http://code.activestate.com/recipes/456195 which I am planning to try next. All in all though I'm surprised this isn't supported "out of the box", which makes me wonder if I'm simply missing out on an obvious solutions, so in short — has anyone got a simple method for fetching https pages using an authenticating proxy with urllib2 in Python 2.5? Ideally this would work:
import urllib2
#perhaps the dictionary below needs a corresponding "https" entry?
#That doesn't seem to work out of the box.
proxy_handler = urllib2.ProxyHandler({"http": "http://user:pass#myproxy:port"})
urllib2.install_opener( urllib2.build_opener( urllib2.HTTPHandler,
urllib2.HTTPSHandler,
proxy_handler ))
request = urllib2.Request(A_HTTPS_URL)
response = urllib2.urlopen( request)
print response.read()
Many Thanks

You may want to look into httplib2. One of the examples claims support for SOCKS proxies if the socks module is installed.