How can I check if a FTP server allows for TLS when I connect using ftplib? I Found easy documentation on how to use TLS, but no solution on how to check for it. My script is supposed to connect either way, but use TLS if possible.
So do I just connect using TLS and if it fails I do it without? I am sure there must be a better way.
Thank you for your help.
Note that trying an encryption only, is as bad as not using any encryption at all. And maybe even worse, as it gives an impression of security, while there's none.
When you are under an MITM attack, an attacker will divert a TCP traffic to him/her and simulate encryption rejection, making you send your credentials plain-text straight to the attacker.
Anyway:
Call FTP_TLS.auth, check if it throws or not.
Call FTP_TLS.login with secure=False.
If the server supports TLS, the FTP_TLS.auth will enable encryption and the FTP_TLS.login (even with secure=False) will continue using it. If the server does not support encryption, the FTP_TLS.login (with secure=False) continues unencrypted.
Note that FTP_TLS.login (with its secure parameter) is an undocumented method - If you do not want to use it, you can fallback to creating a plain FTP, when FTP_TLS.auth throws.
To check explicitly, you can also use FTP.voidcmd('FEAT') and look for AUTH TLS (or AUTH SSL) in the response.
But there's nothing wrong about trying AUTH TLS/AUTH SSL (what the FTP_TLS.auth does) straight away.
I'm looking to create a Twisted Conch Server/Client SSH application similar to the below:
Client <---Key1---> Server/Client <---Key2---> Server
I guess it's like an SSH MITM or command proxier.
I have read the answers to similar Twisted related questions, such as:
Twisted server-client data sharing
and I understand the theory behind passing client/server data.
The part I am unsure of is the user authentication. I want to pass the login details from the first client, through my twisted conch application to the endpoint server.
Is there a way to do this?
Also, would it be possible to do separate SSH key negotiations but then pass all data, including credentials and any commands typed, between server and client?
Unfortunately, this is a little more complicated than it seems. SSH goes through some pains to try and avoid this kind of MITM. If you don't use the same key negotiation, the publickey authentication won't succeed because string you're signing is different on both sides of the connection (see the RFC section 7 for more).
Password authentication should still work okay, though. The easiest way to do that would be to create a twisted.conch.ssh.session.SSHSession subclass which listens for packets on the packetReceived method and forwards them to the corresponding transport.sendPacket.
When using telnetlib I noticed some interesting variables:
AUTHENTICATION = chr(37) # Authenticate
XAUTH = chr(41) # XAUTH
PRAGMA_LOGON = chr(138) # TELOPT PRAGMA LOGON
SSPI_LOGON = chr(139) # TELOPT SSPI LOGON
Also, this page lists some RFCs which describe AUTHENTICATION option in detail.
If I understand correctly, if a telnet server supports authentication I should be receiving IAC DO AUTHENTICATION when connecting to it. However, when trying to connect to telnetd on a Linux machine I do not receive this option and so I can't send IAC WILL AUTHENTICATION.
Is my understandng correct? Is there a way I can ask the server to perform authentication? Is it even possible to authenticate telnet session automatically? (without listening to "Login:", "Username:", "Password:" and similar prompts)
Yes, you are understanding RFC 1416 correctly. The server should send DO first, if it supports authentication. You can try sending WILL anyway but that'll be against the spec and probably won't work anyway.
By Linux telnetd, you probably mean the netkit telnetd. And as far as I can see on its manpage:
-a authmode This option may be used for specifying what mode should be used for
authentication. Note that this option is only useful if telnetd has
been compiled with support for authentication, which is not available
in the current version. [...]
So I'd simply say it can't do that. I'm using the OpenBSD telnetd (telnet-bsd package in Gentoo); it seems to have some auth bits but I wasn't able to get it to work at all. I suspect it only supports Kerberos-based auth.
Looking at the authentication types list in RFC 2941, it seems that most authentication types were never even specified. Looking at those which are, DSS/DSA is mostly interesting (it is used in SSH as well) but I guess it will be really hard to find a compliant server and client. Sadly, it seems that there were no proposals for simple (plaintext) authentications there.
So, to sum up: yes, it would be possible if someone implemented it. But since telnet is practically dead, I doubt you'll find server implementations for that. And even if you did, you'd have to write the client-side yourself which won't be that easy with the supported schemes. So I doubt it's worth the effort.
I am looking for a way to connect a frontend server (running Django) with a backend server.
I want to avoid inventing my own protocol on top of a socket, so my plan was to use SimpleHTTPServer + JSON or XML.
However, we also require some security (authentication + encryption) for the connection, which isn't quite as simple to implement.
Any ideas for alternatives? What mechanisms would you use? I definitely want to avoid CORBA (we have used it before, and it's way too complex for what we need).
Use a client side certificate for the connection. This is a good monetization technique to get more income for your client side app.
I'm looking for a good server/client protocol supported in Python for making data requests/file transfers between one server and many clients. Security is also an issue - so secure login would be a plus. I've been looking into XML-RPC, but it looks to be a pretty old (and possibly unused these days?) protocol.
If you are looking to do file transfers, XMLRPC is likely a bad choice. It will require that you encode all of your data as XML (and load it into memory).
"Data requests" and "file transfers" sounds a lot like plain old HTTP to me, but your statement of the problem doesn't make your requirements clear. What kind of information needs to be encoded in the request? Would a URL like "http://yourserver.example.com/service/request?color=yellow&flavor=banana" be good enough?
There are lots of HTTP clients and servers in Python, none of which are especially great, but all of which I'm sure will get the job done for basic file transfers. You can do security the "normal" web way, which is to use HTTPS and passwords, which will probably be sufficient.
If you want two-way communication then HTTP falls down, and a protocol like Twisted's perspective broker (PB) or asynchronous messaging protocol (AMP) might suit you better. These protocols are certainly well-supported by Twisted.
ProtocolBuffers was released by Google as a way of serializing data in a very compact efficient way. They have support for C++, Java and Python. I haven't used it yet, but looking at the source, there seem to be RPC clients and servers for each language.
I personally have used XML-RPC on several projects, and it always did exactly what I was hoping for. I was usually going between C++, Java and Python. I use libxmlrpc in Python often because it's easy to memorize and type interactively, but it is actually much slower than the alternative pyxmlrpc.
PyAMF is mostly for RPC with Flash clients, but it's a compact RPC format worth looking at too.
When you have Python on both ends, I don't believe anything beats Pyro (Python Remote Objects.) Pyro even has a "name server" that lets services announce their availability to a network. Clients use the name server to find the services it needs no matter where they're active at a particular moment. This gives you free redundancy, and the ability to move services from one machine to another without any downtime.
For security, I'd tunnel over SSH, or use TLS or SSL at the connection level. Of course, all these options are essentially the same, they just have various difficulties of setup.
Pyro (Python Remote Objects) is fairly clever if all your server/clients are going to be in Python. I use XMPP alot though since I'm communicating with hosts that are not always Python. XMPP lends itself to being extended fairly easily too.
There is an excellent XMPP library for python called PyXMPP which is reasonably up to date and has no dependancy on Twisted.
I suggest you look at 1. XMLRPC 2. JSONRPC 3. SOAP 4. REST/ATOM
XMLRPC is a valid choice. Don't worry it is too old. That is not a problem. It is so simple that little needed changing since original specification. The pro is that in every programming langauge I know there is a library for a client to be written in. Certainly for python. I made it work with mod_python and had no problem at all.
The big problem with it is its verbosity. For simple values there is a lot of XML overhead. You can gzip it of cause, but then you loose some debugging ability with the tools like Fiddler.
My personal preference is JSONRPC. It has all of the XMLRPC advantages and it is very compact. Further, Javascript clients can "eval" it so no parsing is necessary. Most of them are built for version 1.0 of the standard. I have seen diverse attempts to improve on it, called 1.1 1.2 and 2.0 but they are not built one on top of another and, to my knowledge, are not widely supported yet. 2.0 looks the best, but I would still stick with 1.0 for now (October 2008)
Third candidate would be REST/ATOM. REST is a principle, and ATOM is how you convey bulk of data when it needs to for POST, PUT requests and GET responses.
For a very nice implementation of it, look at GData, Google's API. Real real nice.
SOAP is old, and lots lots of libraries / langauges support it. IT is heeavy and complicated, but if your primary clients are .NET or Java, it might be worth the bother.
Visual Studio would import your WSDL file and create a wrapper and to C# programmer it would look like local assembly indeed.
The nice thing about all this, is that if you architect your solution right, existing libraries for Python would allow you support more then one with almost no overhead. XMLRPC and JSONRPC are especially good match.
Regarding authentication. XMLRPC and JSONRPC don't bother defining one. It is independent thing from the serialization. So you can implement Basic Authentication, Digest Authentication or your own with any of those. I have seen couple of examples of client side Digest Authentication for python, but am yet to see the server based one. If you use Apache, you might not need one, using mod_auth_digest Apache module instead. This depens on the nature of your application
Transport security. It is obvously SSL (HTTPS). I can't currently remember how XMLRPC deals with, but with JSONRPC implementation that I have it is trivial - you merely change http to https in your URLs to JSONRPC and it shall be going over SSL enabled transport.
HTTP seems to suit your requirements and is very well supported in Python.
Twisted is good for serious asynchronous network programming in Python, but it has a steep learning curve, so it might be worth using something simpler unless you know your system will need to handle a lot of concurrency.
To start, I would suggest using urllib for the client and a WSGI service behind Apache for the server. Apache can be set up to deal with HTTPS fairly simply.
SSH can be a good choice for file transfer and remote control, especially if you are concerned with secure login. Most Linux and Solaris servers will already run an SSH service for administration, so if your Python program use ssh then you don't need to open up any additional ports or services on remote machines.
OpenSSH is the standard and portable SSH client and server, and can be used via subprocesses from Python. If you want more flexibility Twisted includes Twisted Conch which is a SSH client and server implementation which provides flexible programmable control of an SSH stack, on both Linux and Windows. I use both in production.
I'd use http and start with understanding what the Python library offers.
Then I'd move onto the more industrial strength Twisted library.
There is no need to use HTTP (indeed, HTTP is not good for RPC in general in some respects), and no need to use a standards-based protocol if you're talking about a python client talking to a python server.
Use a Python-specific RPC library such as Pyro, or what Twisted provides (Twisted.spread).
XMLRPC is very simple to get started with, and at my previous job, we used it extensively for intra-node communication in a distributed system. As long as you keep track of the fact that the None value can't be easily transferred, it's dead easy to work with, and included in Python's standard library.
Run it over https and add a username/password parameter to all calls, and you'll have simple security in place. Not sure about how easy it is to verify server certificate in Python, though.
However, if you are transferring large amounts of data, the coding into XML might become a bottleneck, so using a REST-inspired architecture over https may be as good as xmlrpclib.
Facebook's thrift project may be a good answer. It uses a light-weight protocol to pass object around and allows you to use any language you wish. It may fall-down on security though as I believe there is none.
In the RPC field, Json-RPC will bring a big performance improvement over xml-rpc:
http://json-rpc.org/wiki/python-json-rpc