I'm currently drafting an upcoming Python project, where I'm required to securely send a Python dictionary form a client to a server over the internet.
As of now, I'm considering a simple TCP communication and use AES to encrypt the dictionary before sending it.
My question is, whether this is possible. When I receive the information on the server, then decrypt it with a shared key, is the outcome still a dictionary?
There are some libraries that support the encryption of arbitrary objects, because they internally have a serializer/unserializer, but PyCrypto is not such a library. You will need to provide a binary string in Python 2 or a bytes object in Python 3.
In order to get those, you need to serialize a complex data structure such as a dictionary. After that, you can encrypt it and send through the socket. At the other end, you will need to decrypt it and unserialize. There are lots of ways for serialization. The answers to this question contain a few of them.
Unserializing foreign data can be quite dangerous, because some algorithms/formats are not designed to be resistant against fault attacks, such as Python's pickle. So it is very important to authenticate ciphertexts with a message authentication code, which must be verified before attempting decryption or even unserialization. This question contains an example.
Related
I'm implementing an RSA encryption scheme where the MySQL database data is encrypted using a public key, and only the private key can be used to decrypt it.
I've managed to do encryption and decryption using the PHP openssl library, and even added a passphrase to protect the private key.
// $sRawText is the text string to encrypt.
// $sPublicKey is the public key stored on the server.
openssl_public_encrypt($sRawText, $sResult, $sPublicKey, OPENSSL_PKCS1_OAEP_PADDING);
// $sResult is the encrypted result which can then be stored in the database.
It works great for all the purposes I need to be publicly available, which is to add new entries if a user signs up or, in the future, verify details if they log in. This is sensitive data like contact information, which I only ever use very occasionally. Data that's verify only like passwords can be hashed still, and anything that needs to be read doesn't get encrypted.
Here's how I'm presently decrypting with PHP:
// $sPrivateKey is the matching private key.
// $sPassPhrase is the pass phrase (required to decrypt the result).
// $sRawBytes is the encrypted data from the database to decrypt.
$kRsaKey = openssl_pkey_get_private($sPrivateKey, $sPassPhrase);
openssl_private_decrypt($sRawBytes, $sResult, $kRsaKey, OPENSSL_PKCS1_OAEP_PADDING);
// $sResult will be the decrypted data.
The problem with this decryption approach is that it's all taking place live on the server. Obviously, storing the private key on that server would defeat most of the security benefits, as anyone with enough access to the server could pretty easily take it. The best I can come up with using PHP on the server is to pass the private key and/or pass phrase at each time of decryption. However, that still has the key being live there, and also opens up a new attack avenue for the key and pass phrase to be intercepted in those requests.
What I'm trying to do is perform all the decryption offline. So an offline software will be provided the encrypted data in a text file, which it translates into a list of instructions based on the data and intended operation. The private key never leaves that isolated environment and the pass phrase is never stored between sessions.
I'm running into all kinds of hurdles setting up an offline PHP environment, so I thought it might be easier to try to do the decryption in Python. Everything should be standard, or so I thought.
I'm currently trying to use the Python RSA library here:
https://stuvel.eu/python-rsa-doc/reference.html#exceptions
The thing I can't figure out is how to decrypt using the passphrase-protected private key. Would anyone be able to post a really simple example, or is it fundamentally not compatible without major modifications to the Python RSA library? What's the easiest approach to be able to do this decryption offline?
Okay I'll post the answer now that I figured out.
(1) We need to get the cryptography library in Python.
On Windows, enter into the command prompt (not Python interpreter):
py -m pip install cryptography
If the above doesn't work, try replacing 'py' with 'python'.
(2) The private key needs to be in a text file "privatekey.txt".
(3) Use this code for decryption:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding
#privatekey.txt must contain the private key.
#passphrase_bytes must be the passphrase, and it needs to be in bytes (ie b'My Pass Phrase')
with open("privatekey.txt", "rb") as key_file:
private_key = serialization.load_pem_private_key(
key_file.read(),
password=passphrase_bytes,
)
#rawtext_bytes must have the bytes of the ciphertext to decrypt
plaintext = private_key.decrypt(
rawtext_bytes,
padding.OAEP(
mgf=padding.MGF1(algorithm=hashes.SHA1()),
algorithm=hashes.SHA1(),
label=None
)
)
#plaintext will now contain the result
I'm working on a program in Python that is highly valuable to certain people. I've been trying to add a user authentication system to keep people from copying and sharing the files easily. To explain further this is the flow:
1: User opens the EXE (cx_freezed python file)
2: Program asks user for their login (from a MySQL database)
3: Program checks if someone is already logged in using that info
4: If someone is logged in, don't let them log in. If someone isn't, log them in.
There are two ways I've thought about doing this but I have absolutely no idea where to start because this isn't in my typical range. I could either intigrate this into Python, or use a tool to take my exe file and add an auth (is that a thing?).
Any advice about where to remotely start would be awesome. This is out of my typical range so I'm a bit daunted by it.
iv'e done something like this in my latest project, however i'm not an expert whatsoever so take what i say with a grain of salt, there might be other, better ways to do this but its just how i implemented it.
these are the models i've worked with:
socket, json, sqlite3 , wxpython (for gui) + pycrypto (if you want to encrypt and decrypt the messages using RSA and AES keys for added security, like i did)
basically what i did was the following:
a socket connection is created between the client and the server.
if you're not going to use pycrypto skip the second and third parts - the server generates a pair of RSA private and public keys for himself and so does the client for himself. the client then sends its RSA public key to the server using the socket connection. the pair of RSA keys(the public key and private key) are essentialy two keys that can only decrypt the messages encrypted by the other key in the pair (public key encrypts-private key decrypts || private key encrypts-public key decrypts) (i would suggest this computerphile video for a little more in depth explanation, once you get how the keys suppose to theoretically work, the module itself is pretty straight forward, so don't worry about that).
the server now creates an AES key (a symmetrical key, that can decrypt the messages it encrypted), and then encrypts the AES key (which is basically a pretty long string) with the RSA public key he got from the client, and sends its to the client, so now only the client can decrypt this message with his private key and get the AES key that will be used for the communication between them.
the client enters a new username and password in order signup. the client script puts the data in a list like so: ["signup", newUserNameVar,NewPassWordVar], uses the json encoding on it using json.dumps() function (basically turns the entire list into a string that can be sent through a socket) then encrypts it using the AES key we got from the server earlier, and then sends it to the server through the socket connection.
so now the server receives an encrypted message only he can decrypt with his AES key (which is the same one the client holds, if you've followed my explanation earlier). so he decrypts the message using the AES key and gets the json message the client sent him. to turn the json back into the list the server uses the json.dumps() function (which turns the json object back into a list).
in my design, the first index of the list is the command so basically either "login" or "setup", so now the server deploys a switch case (or a bunch of ifs) on the first index of the list to see which command he needs to fulfill. if its signup then he takes the 2nd and 3rd index off of the list and creates the table in the database with rows corresponding to the users data, and if its login it takes the data from the database and if it matches the data that was sent by the client in the other indexes of the list then its sends a "correct" response back, and then you can tell your script to run that other program only when it gets a "correct" message from the server, which sends the message exactly as i mentioned in steps 4-5.
so to recap it goes something like this:
server: generate - RSA public key, RSA private key, AES key
client: generate - (other)RSA public key, (other)RSA private key
client ----> server, send clients RSA public key
server ----> client, encrypt the AES key using the clients RSA key and send it to him.
client: decrypt the AES key using the clients private RSA key.
continue to talk with each other in the following way: LIST -> JSON (json.dumps(LIST))-> AES encryption on the JSON object -> send through a socket.
AES decryption on the message you got -> UNJSON (json.loads(WHAT YOU GOT FROM THE DECRYPTION)) -> LIST ->deploy switch-case.
I'm trying to send username and password data from a web form to my server.
The password is sent as plain text over a https connection, then properly encrypted on the server (using python hashlib.sha224) before being stored, however I'm not sure how to transmit the password text to the server in an encrypted format.
My web client is written in javascript, and the server is written in python.
You'd have to encrypt it in the JavaScript. Unfortunately JavaScript does not offer any cryptographic tools out of the box, so you'd have to either use a third-party library or write something on your own.
As others have said, HTTPS is meant to encrypt the whole traffic, so perhaps you don't need anything extra? If you do however, you may want to look at this Article, which might shed some light on the problem. Good luck ! :)
Actually you transmit the password encrypted, because you use SSL.
Furthermore you do not encrypt the password, you hash the password on the server.
Anyway, you can use something like jCryption for it. crypt-js could also fit your purpose.
For python there is a Crypto Library called PyCrypto. But I have a Problem with the communication between Javascript and Python. I try to do something similar, but have a problem with it. I think my question will help you with yours.
Include nonce and block count in PyCrypto AES MODE_CTR
But in general, you have already solved the problem on your own, by using https.
https is an encrypted format. You're good.
If you want to do it clientside anyway I recommend hashing it with sha1. This guy seems to have some libs for that: http://pajhome.org.uk/crypt/md5/ - SHA1, sha256, md5, etc.
The HTTPS channel over which you send the password to the server provides encryption that is good enough.
However, you need a more secure storage mechanism for the password. Use an algorithm like "bcrypt" with many thousands of hash iterations (bcrypt calls this the cost factor, and it should be at least 16, meaning 216 iterations), and a random "salt". This works by deriving an encryption key from the password, which is a computationally expensive process, then using that key to encrypt some known cipher text, which is saved for comparison on future login attempts.
Also, using HTTPS on the login only is not sufficient. You should use it for any requests that require an authenticated user, or that carry an authentication cookie.
On the contrary to http, https is an encrypted protocol. You don't need additional encryption between the client and the server.
SHA224, SHA1 or MD5 are not a encryption, but a hashing function, which means they are irreversible.
Some answers suggest hashing passwords client-side.
However, irreversible doesn't mean uncrackable. Having plain password hashed, it is relatively easy to obtain the matching password from hash (see Rainbow tables for example).
Therefore you should not hash the password on the client side, but concatenate it with a some arbitrary string selected on the server side (usually called a salt) and hash the result.
I have a webapp with some functionality that I'd like to be made accessible via an API or webservice. My problem is that I want to control where my API can be accessed from, that is, I only want the apps that I create or approve to have access to my API. The API would be a web-based REST service. My users do not login, so there is no authentication of the user. The most likely use case, and the one to work with now, is that the app will be an iOS app. The API will be coded with django/python.
Given that it is not possible to view the source-code of an iOS app (I think, correct me if I'm wrong), my initial thinking is that I could just have some secret key that is passed in as a parameter to the API. However, anyone listening in on the connection would be able to see this key and just use it from anywhere else in the world.
My next though is that I could add a prior step. Before the app gets to use API it must pass a challenge. On first request, my API will create a random phrase and encrypt it with some secret key (RSA?). The original, unencrypted phrase will be sent to the app, which must also encrypt the phrase with the same secret key and send back the encrypted text with their request. If the encryptions match up, the app gets access but if not they don't.
My question is: Does this sound like a good methodology and, if so, are there any existing libraries out there that can do these types of things? I'll be working in python server-side and objective-c client side for now.
The easiest solution would be IP whitelisting if you expect the API consumer to be requesting from the same IP all the time.
If you want to support the ability to 'authenticate' from anywhere, then you're on the right track; it would be a lot easier to share an encryption method and then requesting users send a request with an encrypted api consumer handle / password / request date. Your server decodes the encrypted value, checks the handle / password against a whitelist you control, and then verifies that the request date is within some timeframe that is valid; aka, if the request date wasnt within 1 minute ago, deny the request (that way, someone intercepts the encrypted value, it's only valid for 1 minute). The encrypted value keeps changing because the request time is changing, so the key for authentication keeps changing.
That's my take anyways.
In addition to Tejs' answer, one known way is to bind the Product ID of the OS (or another unique ID of the client machine) with a specific password that is known to the user, but not stored in the application, and use those to encrypt/decrypt messages. So for example, when you get the unique no. of the machine from the user, you supply him with password, such that they complete each other to create a seed X for RC4 for example and use it for encryption / decryption. this seed X is known to the server as well, and it also use it for encryption / decryption. I won't tell you this is the best way of course, but assuming you trust the end-user (but not necessarily any one who has access to this computer), it seems sufficient to me.
Also, a good python library for cryptography is pycrypto
On first request, my API will create a random phrase and encrypt it with some secret key (RSA?)
Read up on http://en.wikipedia.org/wiki/Digital_signature to see the whole story behind this kind of handshake.
Then read up on
http://en.wikipedia.org/wiki/Lamport_signature
And it's cousin
http://en.wikipedia.org/wiki/Hash_tree
The idea is that a signature can be used once. Compromise of the signature in your iOS code doesn't matter since it's a one-use-only key.
If you use a hash tree, you can get a number of valid signatures by building a hash tree over the iOS binary file itself. The server and the iOS app both have access to the same
file being used to generate the signatures.
I have an application that should communicate status information to a server. This information is effectively a large dictionary with string keys.
The server will run a web application based on Turbogears, so the server-side method called accepts an arbitrary number of keyword arguments.
In addition to the actual data, some data related to authentication (id, password..) should be transmitted. One approach would be to simply urlencode a large dictionary containing all this and send it in a request to the server.
urllib2.urlencode(dataPlusId)
But actually, the method doing the authentication and accepting the data set does not have to know much about the data. The data could be transmitted and accepted transparently and handed over to another method working with the data.
So my question is: What is the best way to transmit a large dictionary of data to a server in general? And, in this specific case, what is the best way to deal with authentication here?
I agree with all the answers about avoiding pickle, if safety is a concern (it might not be if the sender gets authenticated before the data's unpickled -- but, when security's at issue, two levels of defense may be better than one); JSON is often of help in such cases (or, XML, if nothing else will do...!-).
Authentication should ideally be left to the webserver, as SpliFF recommends, and SSL (i.e. HTTPS) is generally good for that. If that's unfeasible, but it's feasible to let client and server share a "secret", then sending the serialized string in encrypted form may be best.
I think the best way is to encode your data in an appropriate transfer format (you should not use pickle, as it's not save, but it can be binary) and transfer it as a multipart post request
What I do not know if you can make it work with repoze.who. If it does not support sign in and function call in one step, you'll perhaps have to verify the credentials yourself.
If you can wrap your data in xml you could also use XML-RPC.
Why don't you serialize the dictionary to a file, and upload the file? This way, the server can read the object back into a dictionary .
Do a POST of your python data (use binary as suggested in other answers) and handle security using your webserver. Apache and Microsoft servers can both do authentication using a wide variety of methods (SSL client certs, Password, System accounts, etc...)
Serialising/Deserialising to text or XML is probably overkill if you're just going to turn it back to dictionary again).
I'd personally use SimpleJSON at both ends and just post the "file" (it would really just be a stream) over as multipart data.
But that's me. There are other options.
Have you tried using pickle on the data ?