Develop Reference

Is it possible to start a TCP socket connection without a handshake?

I am making a SSL server, and I don't use python's library as I want to make some unorthodox changes to the process. Because of that, I cannot simply start a TCP connection since I need to transfer the encryption details with the handshake, which I can't do over sockets. So I am using scapy to make the handshake itself, but after that I would like to continue working with a TCP socket without going through the process of the handshake again. Is that possible?

If I understand your question correctly, you exchanged a few segments using scapy and now want to manufacture a normal full-blown socket out of them.
This is not easily possible: for all practical purposes your TCP is oblivious to whatever you sent in your packets and it doesn't keep any state for this TCP connection: all the state is in your application.
That said, there is a thing called TCP_REPAIR in Linux that lets you put a socket in a given state.
When this option is used, a socket is switched into a special mode, in
which any action performed on it does not result in anything defined
by an appropriate protocol actions, but rather directly puts the
socket into a state, in which the socket is expected to be at the end
of the successfully finished operation.
If you set sequence numbers correctly, the socket should "just work".,
One also needs to restore the TCP sequence numbers. To do so, the
TCP_REPAIR_QUEUE and TCP_QUEUE_SEQ options were introduced.
Of course all this is specific to a modern Linux; other operating systems may or may not have similar mechanisms.

Listening on port being used

Is there any way to listen to traffic on a specific port that another program is currently using, through the python socket module? For example:
|--> my program
external request -> Host ->|--> intended program
I am not looking to send back a response to the request, I simply want the traffic. I was looking at using the socket.SO_REUSEADDR method, but I think that's simply to avoid having to wait for the socket's timeout, not to allow another listener.

You can do that with scapy
see http://www.secdev.org/projects/scapy/doc/usage.html#sniffing
to use it in your own python program, you'd have to import the sniff command
from scapy.all import sniff
a=sniff("tcp and port 1337")
sniff has various options (callback functions, timeout, number of packets etc)

The socket.SO_REUSEADDR option helps both applications to be bound to the same port and to receive packets destined for that port. So, I am not sure what you mean by your timeout note in the question. The only catch is that for some of the platforms, including Linux, both the sockets must set SO_REUSEADDR, else, the second socket not be able to bind. So, if you cannot modify the intended program (let us say, it is a standard third-party program), then using SO_REUSEADDR might be a problem.

How to abruptly disconnect a socket without closing it appropriately

I have a Python test program for testing features of another software component, let's call the latter the component under test (COT).
The Python test program is connected to the COT via a persistent TCP connection.
The Python program is using the Python socket API for this.
Now in order to simulate a failure of the physical link, I'd like to have the Python program shut the socket down, but without disconnecting appropriately.
I.e. I don't want anything to be sent on the TCP channel any more, including any TCP SYN/ACK/FIN. I just want the socket to go silent. It must not respond to the remote packets any more.
This is not as easy as it seems, since calling close on a socket will send TCP FIN packets to the remote end. (graceful disconnection).
So how can I kill the socket without sending any packets out?
I cannot shut down the Python program itself, because it needs to maintain other connections to other components.
For information, the socket runs in a separate thread. So I thought of abruptly killing the thread, but this is also not so easy. (Is there any way to kill a Thread?)
Any ideas?

You can't do that from a userland process since in-kernel network stack still holds resources and state related to given TCP connection. Event if you kill your whole process the kernel is going to send a FIN to the other side since it knows what file descriptors your process had and will try to clean them up properly.
One way to get around this is to engage firewall software (on local or intermediate machine). Call a script that tells the firewall to drop all packets from/to given IP and port (that of course would need appropriate administrative privileges).

Contrary to Nikolai's answer, there is indeed a way to reset the connection from userland such that an RST is sent and pending data discarded, rather than a FIN after all the pending data. However as it is more abused than used, I won't publish it here. And I don't know whether it can be done from Python. Setting one of the three possible SO_LINGER configurations and closing will do it. I won't say more than that, and I will say that this technique should only be used for the purpose outlined in the question.

Weird network behaviors with UDP sockets and python

I made some python script and ran it both on my computer and some distant shell (some website that provides shell access).
I'm using threads, pipes and UDP sockets to transfer data in a P2P fashion, so each script can both receive and send through the same socket. To test if this works, I open one terminal on my computer and one other terminal with ssh, connected to my shell. I make sure the script is the same on both machines, and feed it with an ip address.
Here is the script: http://codepad.org/V9Q1KcDT
(I don't know if I should paste it directly here or not)
My problem is this: strings I send seems to land something 20% of the time, sometimes often, sometimes not, and it seems to be random...
What am I doing wrong ?
Are UDP so unreliable ?
Are python thread+pipe+socket too slow ?
Could it be some kind of network problem with my shell provider ?
Is my program flawed ?
Are pipes a good solution to communicate with threads ?
I have no problem not using a shell and I have not tried, but it's useful for testing purposes.
BTW if i'm behing a router, how does the router knows where to send the packet if I'm not the only computer connected ? (I tried when I was the only one, it behaved identically).

User Datagram Protocol(UDP) could easily stand for Unreliable Datagram Protocol. UDP provides no guarantees regarding delivery so if you need reliability you have to implement it yourself by resending messages.
That said, threading and multiprocessing and remote machines and networks are a lot of variables to diagnose at once. I'd advise you to step back and try to boil the problem down to something simpler.
For instance:
first try TCP instead of UDP
instead of threads, run 2 processes (one doing nit_send, one doing nit_recv)
run them both on your local machine (use localhost: 127.0.0.1)
Once you have this basic test working then add features back in. For instance, switch to UDP. Once you've got your UDP issues sorted out locally try introducing remote machines or threads, etc.
Regarding the router question all machines connected to your router are most likely NAT'd so they all appear to have the same IP address (the IP address of your router) to machines on the internet. Look into NAT and port-forwarding if you want to route traffic to a specific machine on your local subnet.

How to close a socket left open by a killed program?

I have a Python application which opens a simple TCP socket to communicate with another Python application on a separate host. Sometimes the program will either error or I will directly kill it, and in either case the socket may be left open for some unknown time.
The next time I go to run the program I get this error:
socket.error: [Errno 98] Address already in use
Now the program always tries to use the same port, so it appears as though it is still open. I checked and am quite sure the program isn't running in the background and yet my address is still in use.
SO, how can I manually (or otherwise) close a socket/address so that my program can immediately re-use it?
Update
Based on Mike's answer I checked out the socket(7) page and looked at SO_REUSEADDR:
SO_REUSEADDR
Indicates that the rules used in validating addresses supplied in a bind(2) call should
allow reuse of local addresses. For AF_INET sockets this means that a socket may bind,
except when there is an active listening socket bound to the address. When the listen‐
ing socket is bound to INADDR_ANY with a specific port then it is not possible to bind
to this port for any local address. Argument is an integer boolean flag.

Assume your socket is named s... you need to set socket.SO_REUSEADDR on the server's socket before binding to an interface... this will allow you to immediately restart a TCP server...
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((ADDR, PORT))

You might want to try using Twisted for your networking. Mike gave the correct low-level answer, SO_REUSEADDR, but he didn't mention that this isn't a very good option to set on Windows. This is the sort of thing that Twisted takes care of for you automatically. There are many, many other examples of this kind of boring low-level detail that you have to pay attention to when using the socket module directly but which you can forget about if you use a higher level library like Twisted.

You are confusing sockets, connections, and ports. Sockets are endpoints of connections, which in turn are 5-tuples {protocol, local-ip, local-port, remote-ip, remote-port}. The killed program's socket has been closed by the OS, and ditto the connection. The only relic of the connection is the peer's socket and the corresponding port at the peer host. So what you should really be asking about is how to reuse the local port. To which the answer is SO_REUSEADDR as per the other answers.