Im writing HTTP Proxy Server that will open socket with the browser and the request from the browser will go to my HTTP Proxy Server and my server will open socket with the server that the browser ask for and send him the request.
It will go like this:
Browser --request--> HTTP Proxy Server --request--> Web Server
Browser <--response-- HTTP Proxy Server <--response-- Web Server
Now I need those sockets will be clearly that I can use scapy to see the layers of each packet and manipulate it like I want to. (for security reasons e.g Block Phishing or something like that)
In this code I write simple socket with the browser just for testing and learning about browser behavior with HTTP Requests.
from scapy.all import *
import socket
socket_with_browser = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
print "Start"
socket_with_browser.bind(('127.0.0.1',8080))
socket_with_browser.listen(1)
conn , addr = socket_with_browser.accept()
stream_sock_browser = StreamSocket(conn)
r = stream_sock_browser.recv(4096)
r[TCP].show()
socket_with_browser.close()
I get the following Error:
Traceback (most recent call last):
File "<string>", line 254, in run_nodebug
File "C:\Python26\ProxyServer\module1.py", line 22, in <module>
r[TCP].show()
File "C:\Python26\Lib\site-packages\scapy\packet.py", line 817, in __getitem__
raise IndexError("Layer [%s] not found" % lname)
IndexError: Layer [TCP] not found
There is a way that I can get packet from socket and use it (get packet layers or something) with scapy? Maybe I declare the socket badly? By the way Im using Windows 7 and python 2.6
Six years late but your question might appear to others with similar issues.
The problem with your approach is that not every packet received has a TCP layer, so you should use the haslayer() method.
if r.haslayer(TCP): # yes, without quotation marks
r.show()
Related
Im having some problem when joining Electron and Python using web sockets (mostly as a learning experience):
Build a desktop app using Electron Ok
Build a Python program that monitors some things Ok
Connect Python with Electron using socket.io Not working
First I tested socket.io in python with the 'latency' example in https://github.com/miguelgrinberg/python-socketio/tree/master/examples/server/aiohttp. It worked great when I took a look at http://localhost:8080/ in Chrome.
The next step was to ask Electron to load the same url in the desktop app with mainWindow.loadURL('http://localhost:8080') (I used the template from the official electron-quick-start example). I just had to add two lines into the html as shown in here: Electron: jQuery is not defined for jquery to load correctly in Electron, but it again worked nicely.
Now I tried to create an index.html with the same content as latency.html and load it with mainWindow.loadFile('gui/index.html'). I changed the line var socket = io.connect(); for var socket = io.connect('ws://localhost:8080'); (I did also tried ws://127.0.0.1:8080 and ws://192.168.<x>.<x>:8080). The html file loads correctly but the socket begins to connect and disconnect repeatedly with a Bad Request error (I added two console.log lines):
(console.log) connected
(error) POST http://localhost:8080/socket.io/?EIO=3&transport=polling&t=MosNuF3&sid=c62ce5a6090c4b72bf3f7c6916da6ce7 400 (Bad Request) **polling-xhr.js:264**
(console.log) disconnected
(warning) websocket.js:235 WebSocket connection to 'ws://localhost:8080/socket.io/?EIO=3&transport=websocket&sid=c62ce5a6090c4b72bf3f7c6916da6ce7' failed: WebSocket is closed before the connection is established.
(error) POST http://localhost:8080/socket.io/?EIO=3&transport=polling&t=MosNuFS&sid=c62ce5a6090c4b72bf3f7c6916da6ce7 400 (Bad Request) **polling-xhr.js:264**
(console.log) connected
etc...
(console.log) disconnected
etc...
On the server side there is also an error when I stop the Electron app:
Unhandled exception
Traceback (most recent call last):
File "D:\Programming\Tools\Anaconda3\lib\site-packages\aiohttp\web_protocol.py", line 447, in start
await resp.prepare(request)
File "D:\Programming\Tools\Anaconda3\lib\site-packages\aiohttp\web_response.py", line 353, in prepare
return await self._start(request)
File "D:\Programming\Tools\Anaconda3\lib\site-packages\aiohttp\web_response.py", line 667, in _start
return await super()._start(request)
File "D:\Programming\Tools\Anaconda3\lib\site-packages\aiohttp\web_response.py", line 410, in _start
await writer.write_headers(status_line, headers)
File "D:\Programming\Tools\Anaconda3\lib\site-packages\aiohttp\http_writer.py", line 112, in write_headers
self._write(buf)
File "D:\Programming\Tools\Anaconda3\lib\site-packages\aiohttp\http_writer.py", line 67, in _write
raise ConnectionResetError('Cannot write to closing transport')
ConnectionResetError: Cannot write to closing transport
What does this Bad Request error mean in this context ? How would the connection work correctly ?
Thanks.
In case someone hits the same problem. The answer is pretty simple, from the documentation for socket.io here:
For security reasons, this server enforces a same-origin policy by
default. In practical terms, this means the following:
If an incoming HTTP or WebSocket request includes the Origin header, this header must match the scheme and host of the connection
URL. In case of a mismatch, a 400 status code response is returned and
the connection is rejected.
No restrictions are imposed on incoming requests that do not include the Origin header.
If necessary, the cors_allowed_origins option can be used to allow
other origins. This argument can be set to a string to set a single
allowed origin, or to a list to allow multiple origins. A special
value of '*' can be used to instruct the server to allow all origins,
but this should be done with care, as this could make the server
vulnerable to Cross-Site Request Forgery (CSRF) attacks.
When using electron to load the gui, the server is not the same as the python server that's doing the monitoring, thus the Bad Request (not same origin). The solution is to just modify as follows the server properties adding cors_allowed_origins='*':
socketio.AsyncServer(async_mode='aiohttp', cors_allowed_origins='*')
I'm trying to connect to control port (9051) of tor from a remote machine using stem python library.
dum.py
from stem import Signal
from stem.control import Controller
def set_new_ip():
"""Change IP using TOR"""
with Controller.from_port(address = '10.130.8.169', port=9051) as controller:
controller.authenticate(password='password')
controller.signal(Signal.NEWNYM)
set_new_ip()
I'm getting the following error
Traceback (most recent call last):
File "/home/jkl/anaconda3/lib/python3.5/site-packages/stem/socket.py", line 398, in _make_socket
control_socket.connect((self._control_addr, self._control_port))
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "dum.py", line 28, in <module>
set_new_ip();
File "dum.py", line 7, in set_new_ip
with Controller.from_port(address = '10.130.4.162', port=9051) as controller:
File "/home/jkl/anaconda3/lib/python3.5/site-packages/stem/control.py", line 998, in from_port
control_port = stem.socket.ControlPort(address, port)
File "/home/jkl/anaconda3/lib/python3.5/site-packages/stem/socket.py", line 372, in __init__
self.connect()
File "/home/jkl/anaconda3/lib/python3.5/site-packages/stem/socket.py", line 243, in connect
self._socket = self._make_socket()
File "/home/jkl/anaconda3/lib/python3.5/site-packages/stem/socket.py", line 401, in _make_socket
raise stem.SocketError(exc)
stem.SocketError: [Errno 111] Connection refused
Then I went through /etc/tor/torrc config file.
It says
The port on which Tor will listen for local connections from Tor
controller applications, as documented in control-spec.txt.
ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:E5364A963AF943CB607CFDAE3A49767F2F8031328D220CDDD1AE30A471
SocksListenAddress 0.0.0.0:9050
CookieAuthentication 1
My question is ,
How do I connect to control port of Tor from a remote host?
Is there is any work around or config parameter that I need to set?
a possible duplicate of Stem is giving the "Unable to connect to port 9051" error which has no answers
Tested with Tor 0.3.3.7.
ControlListenAddress config is OBSOLETE and Tor will ignore it and log the following message
[warn] Skipping obsolete configuration option 'ControlListenAddress'
You can still set ControlPort to 0.0.0.0:9051 in your torrc file. Though, Tor is not very happy about it (and rightly so) and will warn you
You have a ControlPort set to accept connections from a non-local
address. This means that programs not running on your computer can
reconfigure your Tor. That's pretty bad, since the controller protocol
isn't encrypted! Maybe you should just listen on 127.0.0.1 and use a
tool like stunnel or ssh to encrypt remote connections to your control
port.
Also, you have to set either CookieAuthentication or HashedControlPassword otherwise ControlPort will be closed
You have a ControlPort set to accept unauthenticated connections from
a non-local address. This means that programs not running on your
computer can reconfigure your Tor, without even having to guess a
password. That's so bad that I'm closing your ControlPort for you. If
you need to control your Tor remotely, try enabling authentication and
using a tool like stunnel or ssh to encrypt remote access.
All the risks mentioned in #drew010's answer still stand.
You'd need to set ControlListenAddress in addition to the ControlPort. You could set that to to 0.0.0.0 (binds to all addresses) or a specific IP your server listens on.
If you choose to do this it would be extremely advisable to configure your firewall to only allow control connections from specific IP's and block them from all others.
Also note, the control port traffic will not be encrypted, so it'd also be advisable to use cookie authentication so your password isn't sent over the net.
You could also run a hidden service to expose the control port over Tor and then connect to the hidden service using Stem and Tor.
But the general answer is ControlListenAddress needs to be set to bind to an IP other than 127.0.0.1 (localhost).
I am trying to make a server in python using sockets that I can connect to on any web browser. I am using the host as "localhost" and the port as 8888.
When I attempt to connect to it, the stuff I want to be shown shows up for a split-second, and then it goes away with the browser saying "The connection was reset".
I've made it do something very simple to test if it still does it, and it does.
Is there a way to stop this?
import time
import socket
HOST = "localhost"
PORT = 8888
def function(sck):
sck.send(bytes("test"),"UTF-8"))
sck.close()
ssck=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ssck.bind((HOST,PORT))
ssck.listen(1)
while True:
sck,addr=ssck.accept()
function(sck)
Probably the same problem as Perl: Connection reset with simple HTTP server, Ultra simple HTTP socket server, written in PHP, behaving unexpectedly, HTTP Server Not Sending Complete File To WGET, Firefox. Connection reset by peer?. That is you don't read the HTTP header from the browser but simply send your response and close the connection.
tl;dr
your function should be
def function(sck):
sck.send(bytes("HTTP/1.1 200 OK\n\n<header><title>test page</title></header><body><h1>test page!</h1></body>"),"UTF-8"))
sck.close()
With a server as simple as that, you're only creating a TCP socket.
HTTP protocols suggest that the client should ask for a page, something like:
HTTP/1.1 GET /somepath/somepage.html
Host: somehost.com
OtherHeader: look at the http spec
The response should then be:
HTTP/1.1 200 OK
some: headers
<header></header><body></body>
I'm playing a bit with Twisted and created a simple 'server'.
I'd like to let the server listening on multiple ports (1025-65535) instead of a single port.
How can i do this ?
My code:
from twisted.internet.protocol import Protocol,ServerFactory
from twisted.internet import reactor
class QuickDisconnectProtocol(Protocol):
def connectionMade(self):
print "Connection from : ", self.transport.getPeer()
self.transport.loseConnection() # terminate connection
f = ServerFactory()
f.protocol = QuickDisconnectProtocol
reactor.listenTCP(6666,f)
reactor.run()
Already tried this:
for i in range (0, 64510):
reactor.listenTCP(1025+i,f)
reactor.run()
But received an error:
Traceback (most recent call last):
File "Server.py", line 14, in <module>
File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 436, in listenTCP
File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 641, in startListening
twisted.internet.error.CannotListenError: Couldn't listen on any:2044: [Errno 24] Too many open files.
Each listening port requires a file descriptor ("open file"), and each file descriptor takes up one element of your maximum file descriptors quota.
This stack overflow question has an answer explaining how to raise this limit on Linux, and this blog post has resources as to how to do it on OS X.
That said, the other respondents who have told you that this is not a particularly sane thing to do are right. In particular, your operating system may stop working if you actually go all the way up to 65535, this overrules the entire ephemeral port range, which means you may not be able to make TCP client connections from this machine any more. So it would be good to explain in your question why you are trying to do this.
The usual solution is to have one listening port ( chosen by the server! ). If you want each client on its own port, then the server chooses the port, starts listening on it, and replies to the client with the port it will use for further requests.
It is not a real good use of port resources! If the server needs to keep state information for each client then it should issue a unique ID to each client when the client first connects and the client should use this ID for every request to the server.
However, with a little care, you can often design the system so that the server does not need to keep separate state information for each client.
When I try to connect to the Gmail server, python throws an error:
>>> from smtplib import SMTP
>>> m = SMTP('smtp.gmail.com', 587)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\smtplib.py", line 249, in __init__
(code, msg) = self.connect(host, port)
File "C:\Python27\lib\smtplib.py", line 309, in connect
self.sock = self._get_socket(host, port, self.timeout)
File "C:\Python27\lib\smtplib.py", line 284, in _get_socket
return socket.create_connection((port, host), timeout)
File "C:\Python27\lib\socket.py", line 571, in create_connection
raise err
socket.error: [Errno 10060]
The rest of the output is in a diferent language but it basically says that the host (gmail) didn't respond.
I can see my email on a browser here at my work, probably there's a network configuration that doesn't allow me to automate the email delivering.
Is there a work around to let python act as a regular browser?
First, you can only access gmail's SMTP servers as a client with some form of authentication; the recommended way is with oauth. See this page and this one for details. So, your code won't work, even when you get past this problem.
However, that doesn't explain why it's rejecting your connection before you even get a chance to log in.
The most likely possibility is that gmail's routers are maintaining a dynamic whitelist of IPs. When you use a properly-logged-in connection of some other kind, you get added to the whitelist for N seconds, meaning you're allowed to connect to port 587; otherwise, you're rejected. This would be similar to the traditional SMTP-after-IMAP auth scheme, but not restricted to IMAP, and handled at the router instead of the SMTP service (presumably to lower the cost or make DoS attacks more difficult).
There's a good way to test this: Configure Outlook, Mail.app, or some other mail client that knows how to connect to gmail, and uses SMTP to send mail via gmail. Run your script a few seconds after fetching mail in the mail client. If it works, that's the problem. And in that case, the fix is to do the same kind of connection and login (IMAP? web service?) that the mail client does.
Or, of course, you can use the sample code Google provides at the above links instead of working it out from first principles.
(Of course gmail also has to accept server-to-server SMTP connections, but they could easily have a different auth scheme for that. I'm assuming you're trying to do a client-to-server connection, rather than server-to-server.)
The other possibility is that you're on some kind of blacklist—e.g., gmail thinks your IP belongs to a spammer. This could also be dynamic—maybe anyone who makes a connection to port 587 but doesn't oauth properly gets blocked for the next N seconds. At any rate, this is also easy to test: Configure Outlook, Mail.app, etc. If this is the problem, they won't be able to send mail either.
There's a third possibility, that no one is allowed to connect to port 587, and Google wants you to use port 565 or 25 instead.
For easier debugging, you might want to consider writing an even simpler script that just creates a socket and connects, instead of using smtplib:
import socket
s = socket.socket()
s.connect(('smtp.gmail.com', 587))
Or, even more simply, just netcat from the command line:
nc smtp.gmail.com 587
To answer your side question:
Is there a work around to let python act as a regular browser?
That's not the issue. A regular browser doesn't make SMTP connections; it makes web service connections using Javascript code downloaded from gmail.com.
Of course Python can also make web services connections.
And it can act as much or as little like a "regular browser" (e.g., User-Agent, Referer, etc. headers) as you desire, but that probably isn't relevant—either the gmail web service API is public and has clear, published rules for how to authenticate yourself (in which case you just do what the rules say), or it's private and you shouldn't be trying to fool whatever protection they're using unless you want to get into an arms race.
At any rate, in this case, we know it's public, so we don't have to guess.