I have a code that I want to pass all packages through the proxy, how can I do it using:
import socket
import #Here your socket library
# The rest of the code here
Which library do you recommend me to use with Socket? Is it really possible what I'm saying?
Using Socksipy in Python 3.4 I can let the code with 3 more lines that are:
import socksipy
s=socket.socket( )
s = socks.socksocket()
s.setproxy(socks.PROXY_TYPE_SOCKS5, 'localhost', 9050)
s.connect((HOST, PORT))
Socksipy can be found at: http://socksipy.sourceforge.net/ and its very simple to use, you just need to read their wiki so than you can work with. The URL for the wiki is: https://code.google.com/p/socksipy-branch/
Related
The requirement is to scrap anonymously or change ip after certain number of calls. I use the https://github.com/kennethreitz/requests-html module to parse the HTML, but i get the below error,
socks.SOCKS5Error: 0x01: General SOCKS server failure
Code
import socks
import socket
import requests_html
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, addr='127.0.0.1', port=int('9150'))
socket.socket = socks.socksocket
session = requests_html.HTMLSession()
r = session.get('http://icanhazip.com')
r.html.render(sleep=5)
print(r.html.text)
But it works perfectly fine with requests module,
import socks
import socket
import requests
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, addr='127.0.0.1', port=int('9150'))
socket.socket = socks.socksocket
print(requests.get("http://icanhazip.com").text)
Any help to solve the issue with requests-html module would be highly appreciated.
Try:
session = requests_html.HTMLSession(browser_args=["--no-sandbox","--proxy-server=127.0.0.1:9150"])
Depends on how your proxy is set up to use tor but this worked for me!
I am trying to test socket communication on my laptop using python. However, I'm not sure why the connection is not being established? I keep getting error that the target machine is actively refusing connection. I am trying to use the same computer to run both the client and the server portion. The server is running fine but the client is the one not connecting. I think I have the hostname wrong (127.0.0.1) but not sure what Im supposed to be using? I also tried changing the server hostname to (0.0.0.0) and the IPV4 address for the hostname the client was to connect to but that didn't work either. Any help would be appreciated!
My code(server portion):
import socket
comms_socket =socket.socket()
comms_socket.bind(('127.0.0.1', 50000))
comms_socket.listen(10)
connection, address = comms_socket.accept()
while True:
print(connection.recv(4096).decode("UTF-8"))
send_data = input("Reply: ")
connection.send(bytes(send_data, "UTF-8"))
Client portion:
import socket
comms_socket = socket.socket()
comms_socket.connect(('127.0.0.1',50000))
while True:
send_data = input("Message: ")
comms_socket.send(bytes(send_data, "UTF-8"))
print(comms_socket.recv(4096).decode("UTF-8"))
Your code won't work with python 2.* , because of the differences in input(), raw_input(), bytes, etc. in python 3.* vs python 2.* . You'd have to minimally make the following changes to get it working with python 2.*. Otherwise, use python 3 to run your code:
Server program:
import socket
comms_socket =socket.socket()
comms_socket.bind(('127.0.0.1', 7000))
comms_socket.listen(10)
connection, address = comms_socket.accept()
while True:
print(connection.recv(4096).decode("UTF-8"))
send_data = raw_input("Reply: ") # Use raw_input() instead of input()
connection.send(send_data.encode("UTF-8"))
Client program:
import socket
comms_socket = socket.socket()
comms_socket.connect(('127.0.0.1',7000))
while True:
send_data = raw_input("Message: ")
comms_socket.send(send_data.encode("UTF-8"))
print(comms_socket.recv(4096).decode("UTF-8"))
If you want to use bytes as intended in your specific usecase, you should use bytesarray instead in python 2.6 or higher. Check this: the bytes type in python 2.7 and PEP-358
This is the code I am trying to run. But the program produces a socket.error. I have a network proxy with port 8080 which connects me to the Internet, what more details do I have to add here to create this socket connection?
import socket
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect(('www.pythonlearn.com', 80))
mysock.send('GET http://www.pythonlearn.com/code/intro-short.txt HTTP/1.0\n\n')
while True:
data = mysock.recv(512)
if ( len(data) < 1 ) :
break
print data;
mysock.close()
If I run your code I get the error TypeError: a bytes-like object is required, not 'str' from line 5.
Try using: mysock.send(b'GET http://www.pythonlearn.com/code/intro-short.txt HTTP/1.0\n\n') with the b before your string indicating a bytestring.
I too got a type error upon running your code, and did not have an error connecting the socket. When using the socket library, make use of the makefile method so you won't have to deal with annoying details.
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
my_sock_input = mysock.makefile('r')
my_sock_output = mysock.makefile('w')
Now my_sock_input can use methods like readline(), without any details on bytes to reserve or wotnot. Same convenience stuff for output, but with write. Remember to close all of them!
As to your problem, I tried writing similar things using my makefile variables and I wasn't recieving any message back. So there is some other issue there.
Now, the solution. A simpler way to download a url and read its contents is using the urllib.request library. If you are on Python 2.7, just import urrlib.
import urllib.request
data = urllib.request.urlopen('http://www.pythonlearn.com/code/intro-short.txt')
readable = data.read()
print(readable)
I'm trying to establish a secure socket connection in Python, and i'm having a hard time with the SSL bit of it. I've found some code examples of how to establish a connection with SSL, but they all involve key files. The server i'm trying to connect with doesn't need to receive any keys or certificates. My question is how do I essentially wrap a python socket connection with SSL. I know for a fact that the cipher i'm suppose to use is ADH-AES256-SHA, and the protocol is TLSv1. This is what i've been trying:
import socket
import ssl
# SET VARIABLES
packet, reply = "<packet>SOME_DATA</packet>", ""
HOST, PORT = 'XX.XX.XX.XX', 4434
# CREATE SOCKET
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(10)
# WRAP SOCKET ???
ssl.wrap_socket(sock, ssl_version="TLSv1", ciphers="ADH-AES256-SHA")
# CONNECT AND PRINT REPLY
sock.connect((HOST, PORT))
sock.send(packet)
print sock.recv(1280)
# CLOSE SOCKET CONNECTION
sock.close()
When I run this code, I don't get any errors, but I get a blank response. When trying to debug this code in the command line, by typing in python in the terminal and pasting in code line by line, I get what i'm assuming is a status code when running sock.send(packet). The integer response I get is 26. If anyone knows what this means, or can help in anyway it would be greatly appreciated.
Ok, I figured out what was wrong. It was kind of foolish of me. I had two problems with my code. My first mistake was when specifying the ssl_version I put in TLSv1 when it should have been ssl.PROTOCOL_TLSv1. The second mistake was that I wasn't referencing the wrapped socket, instead I was calling the original socket that I have created. The below code seemed to work for me.
import socket
import ssl
# SET VARIABLES
packet, reply = "<packet>SOME_DATA</packet>", ""
HOST, PORT = 'XX.XX.XX.XX', 4434
# CREATE SOCKET
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(10)
# WRAP SOCKET
wrappedSocket = ssl.wrap_socket(sock, ssl_version=ssl.PROTOCOL_TLSv1, ciphers="ADH-AES256-SHA")
# CONNECT AND PRINT REPLY
wrappedSocket.connect((HOST, PORT))
wrappedSocket.send(packet)
print wrappedSocket.recv(1280)
# CLOSE SOCKET CONNECTION
wrappedSocket.close()
Hope this can help somebody!
You shouldn't be setting PROTOCOL_TLSv1 (or TLSv1). This restricts the connection to TLS v1.0 only. Instead you want PROTOCOL_TLS (or the deprecated PROTOCOL_SSLv23) that supports all versions supported by the library.
You're using an anonymous cipher, because for some reason you think you don't need a certificate or key. This means that there is no authentication of the server and that you're vulnerable to a man in the middle attack. Unless you really know what you're doing, I suggest you don't use anonymous ciphers (like ADH-AES256-SHA).
I was looking for a good working ssl socket that starts the connection with a https package. This helped me a lot but is a little outdated, so here is the code for python3:
import socket
import ssl
package = "GET /ws/LiveWebcastUpdate/22000557 HTTP/1.1\r\nHost:
www.website_name.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64;
rv:80.0) Gecko/20100101 Firefox/80.0\r\nAccept: */*\r\nAccept-Language: nl,en-
US;q=0.7,en;q=0.3\r\nSec-WebSocket-Version: 13\r\nOrigin:
https://www.website_name.com\r\nSec-WebSocket-Key:
NU/EsJMICjSociJ751l0Xw==\r\nConnection: keep-alive, Upgrade\r\nPragma: no-
cache\r\nCache-Control: no-cache\r\nUpgrade: websocket\r\n\r\n"
hostname = 'www.website_name.com'
port = 443
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as ssock:
print(ssock.version())
ssock.send(package.encode())
while True:
data = ssock.recv(2048)
if ( len(data) < 1 ) :
break
print(data)
This is as simple as possible, for more information visit
https://docs.python.org/3/library/ssl.html
There is a lot of fun to be had solving these problems but for me, I found that the underlying infrastructure for python ssl is openssl. Try validating your certificates with openssl and do this before you try to get python to use that same stack.
I needed to import a root certificate into openssl before I could validate the leaf certificate.
This was helpful.
http://gagravarr.org/writing/openssl-certs/others.shtml#ca-openssl
Another interesting thing was that two different build of the same version of python on different hosts had different methods. One had ssl.get_default_verify_paths() and the other didn't had any at all. The lesson here is that python ssl is built on openssl. Different underlying libraries give you a different python.
Python SSL is built on openssl so solve certificate issues in openssl first.
This question already has answers here:
How to route urllib requests through the TOR network? [duplicate]
(3 answers)
Closed 7 years ago.
Sample code:
#!/usr/bin/python
import socks
import socket
import urllib2
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4, "127.0.0.1", 9050, True)
socket.socket = socks.socksocket
print urllib2.urlopen("http://almien.co.uk/m/tools/net/ip/").read()
TOR is running a SOCKS proxy on port 9050 (its default). The request goes through TOR, surfacing at an IP address other than my own. However, TOR console gives the warning:
"Feb 28 22:44:26.233 [warn] Your
application (using socks4 to port 80)
is giving Tor only an IP address.
Applications that do DNS resolves
themselves may leak information.
Consider using Socks4A (e.g. via
privoxy or socat) instead. For more
information, please see
https://wiki.torproject.org/TheOnionRouter/TorFAQ#SOCKSAndDNS."
i.e. DNS lookups aren't going through the proxy. But that's what the 4th parameter to setdefaultproxy is supposed to do, right?
From http://socksipy.sourceforge.net/readme.txt:
setproxy(proxytype, addr[, port[, rdns[, username[, password]]]])
rdns - This is a boolean flag than
modifies the behavior regarding DNS
resolving. If it is set to True, DNS
resolving will be preformed remotely,
on the server.
Same effect with both PROXY_TYPE_SOCKS4 and PROXY_TYPE_SOCKS5 selected.
It can't be a local DNS cache (if urllib2 even supports that) because it happens when I change the URL to a domain that this computer has never visited before.
The problem is that httplib.HTTPConnection uses the socket module's create_connection helper function which does the DNS request via the usual getaddrinfo method before connecting the socket.
The solution is to make your own create_connection function and monkey-patch it into the socket module before importing urllib2, just like we do with the socket class.
import socks
import socket
def create_connection(address, timeout=None, source_address=None):
sock = socks.socksocket()
sock.connect(address)
return sock
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
# patch the socket module
socket.socket = socks.socksocket
socket.create_connection = create_connection
import urllib2
# Now you can go ahead and scrape those shady darknet .onion sites
The problem is that you are importing urllib2 before you set up the socks connection.
Try this instead:
import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4, '127.0.0.1', 9050, True)
socket.socket = socks.socksocket
import urllib2
print urllib2.urlopen("http://almien.co.uk/m/tools/net/ip/").read()
Manual request example:
import socks
import urlparse
SOCKS_HOST = 'localhost'
SOCKS_PORT = 9050
SOCKS_TYPE = socks.PROXY_TYPE_SOCKS5
url = 'http://www.whatismyip.com/automation/n09230945.asp'
parsed = urlparse.urlparse(url)
socket = socks.socksocket()
socket.setproxy(SOCKS_TYPE, SOCKS_HOST, SOCKS_PORT)
socket.connect((parsed.netloc, 80))
socket.send('''GET %(uri)s HTTP/1.1
host: %(host)s
connection: close
''' % dict(
uri=parsed.path,
host=parsed.netloc,
))
print socket.recv(1024)
socket.close()
I've published an article with complete source code showing how to use urllib2 + SOCKS + Tor on http://blog.databigbang.com/distributed-scraping-with-multiple-tor-circuits/
Hope it solves your issues.