Develop Reference

PGP encryption in azure databrick

I need your help badly :D
I wrote a code in python with PGP , I have a trusted public key and I could perfectly encrypt my massage with this code, but when I run it on data brick I faced problem :
gnupghome should be a directory and it isnt
I would like to know how can I access to a directory in databrick.
import gnupg
from pprint import pprint
import os
gpg = gnupg.GPG(gnupghome='/root/.pnugp')
key_data = open("/dbfs/mnt/xxxx/SCO/oracle/xxx/Files/publickey.asc").read()
import_result = gpg.import_keys(key_data)
pprint(import_result.results)
with open("/dbfs/mnt/xxxxx-storage/SCO/oracle/xxx/Files/FileToEncrypt.txt",'rb') as f:
status = gpg.encrypt_file(
f, recipients=['securxxxxfertuca#xx.ca'],
output='my-encrypted.txt.gpg')
print( 'ok: ', status.ok)
print ('status: ', status.status)
print ('stderr: ', status.stderr)

I suspect that this ran successfully locally. It doesn't work on databricks because it is looking for the .pnugp in the root which data bricks does not allow you to access.
I use the below snippet of code which doesn't need you to access anything from any directory other than the files you plan to encrypt and the keys.
In the code, I have my public key stored in the key vault as a secret named 'publicb64'. If you want to read the asc version from somewhere you can just read it into KEY_PUB. Don't forget to install pgpy using pip install pgpy.
#Encrypting a file using public key
import pgpy
from pgpy.constants import PubKeyAlgorithm, KeyFlags, HashAlgorithm, SymmetricKeyAlgorithm, CompressionAlgorithm
from timeit import default_timer as timer
import base64
import io
KEY_PUB = base64.b64decode(publicb64).decode("ascii").lstrip()
#print(KEY_PUB)
pub_key = pgpy.PGPKey()
pub_key.parse(KEY_PUB)
pass
# -READ THE FILE FROM MOUNT POINT-----------------
with io.open('/dbfs/mnt/sample_data/california_housing_test.csv', "r",newline='') as csv_file:
input_data = csv_file.read() # The io and newline retains the CRLF
t0 = timer()
#PGP Encryption start
msg = pgpy.PGPMessage.new(input_data)
###### this returns a new PGPMessage that contains an encrypted form of the original message
encrypted_message = pub_key.encrypt(msg)
pgpstr = str(encrypted_message)
with open('/dbfs/mnt/sample_data/california_housing_test.csv.pgp', "w") as text_file:
text_file.write(pgpstr)
print("Encryption Complete :" + str(timer()-t0))

encrypt password in python

I want to generate my Python code in a setup.exe. The user stores an email password in the script. My question: Do I have to additionally encrypt this password, even though I create an * .exe file.
def load_settings(self):
# print(__file__)
# print(os.path.dirname(__file__))
pf = os.path.dirname(__file__)
pa = os.path.join(pf, "settings.json")
# print(pa)
if os.path.exists(pa):
# print("Pfad existiert")
with open(pa, "r") as infile:
data = json.load(infile)
self.ein.pfadbez.setText(data["pfad"])
self.ein.name.setText(data["name"])
self.ein.mail.setText(data["mail"])
self.ein.ausgangserver.setText(data["smtp"])
self.ein.port.setText(data["port"])
self.ein.login.setText(data["login"])
self.ein.passwort.setText(data["pw"])

From the way you worded your question, it sounds like you want a user to store a password within the code itself, or in a text file. Variables are called variables because they vary - a password won't be saved between executions unless stored in plain text, which is where encryption will be needed.
Further, generating Python code from a Windows executable will still require that Python code to be put somewhere for execution, and since Python is fundamentally open-source, hiding it in a compiled package won't do much.
Going about text encryption is simple - since you're on Windows, you could use Pycryptodomex, which will simplify the process of encrypting text. This tutorial could help.

Here's my code:
from cryptography.fernet import Fernet
import pyperclip
print("For this program to work, please send the file named 'pwkey' and the encrypted code to the other user.")
key = Fernet.generate_key()
file = open('pwkey', 'wb')
file.write(key)
file.close()
print('File Generated')
original = input('Enter message>>>')
message = original.encode()
f = Fernet(key)
encrypted = f.encrypt(message)
encrypted = encrypted.decode("ascii")
print('Encrypted:', encrypted)
pyperclip.copy(encrypted)
print('Please tell the other user to input the encrypted code in the Decrypt program')
print('(Code copied to Clipboard)')
print("Note: Please delete the 'pwkey' file after sending it to the other user. It
is for one-time use only.")
And decrypting
# Decrypt
from cryptography.fernet import Fernet
print("For this program to work, make sure you have the file named 'pwkey' in your Python folder and the encrypted "
"code.")
file = open('pwkey', 'rb')
key = file.read()
file.close()
print('Key Retrieved')
encrypted = input('Please input your encrypted code>>>')
encrypted = bytes(encrypted, 'utf-8')
f = Fernet(key)
decrypted = f.decrypt(encrypted)
decrypted = decrypted.decode()
print('Decrypted Message:')
print(decrypted)
print("Note: Please delete the 'pwkey' file after getting the decrypted message.")

Problems to verify a file signed with python

I'm trying to create a signed file using OpenSSL and Python, and I'm not receiving any error mesage, but the proccess is not working properly and I can't find the reason.
Below is my step-by-step to sign the file and check the signature:
First I create the crt in command line
openssl req -nodes -x509 -sha256 -newkey rsa:4096 -keyout "cert.key" -out "cert.crt" -subj "/C=BR/ST=SP/L=SP/O=Company/OU=IT Dept/CN=cert"
At this point, I have two files: cert.key and cert.crt
Sign the file use a Python Script like below:
import os.path
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from Crypto.Hash import SHA256
from base64 import b64encode, b64decode
def __init__(self):
folder = os.path.dirname(os.path.realpath(__file__))
file_path = os.path.join(folder, '../static/cert.key')
self.key = open(file_path, "r").read()
def sign_data(self, my_file):
rsakey = RSA.importKey(self.key) # I opened the cert.key in __init__
signer = PKCS1_v1_5.new(rsakey)
digest = SHA256.new()
digest.update(my_file)
sign = signer.sign(digest)
return sign, b64encode(sign)
All works fine and after save the files, I have other three files: my_file.csv (the original one), my_file.txt.sha256 and my_file.txt.sha256.base64. At this point, I can decode the base64 file and compare with the signed one and both are fine.
The problem is when I try to verify the signature using the following command:
`openssl dgst -sha256 -verify <(openssl x509 -in "cert.crt" -pubkey -noout) -signature my_file.txt.sha256 my_file.csv`
At this point I always receive the "Verification Failure" and don't understand why.
Maybe the problem is my lack of Python's Knowledge, because when I sign the file using the following command (after step 1 and before use the Python script described in 2), the same verification works fine.
openssl dgst -sha256 -sign "cert.key" -out my_file.txt.sha256 my_file.csv
Am I doing anything wrong?
UPDATE
Based on the comments, I tried the script in a local virtualnv with python 2.7 and it worked, so the problem must be in the read/write operations.
I'm updating this quetion with the complete script, including the read/write operations 'cause I can run it locally, but I still don't get any error in the GAE environment and can't understand why.
The first step is the CSV creation and storage in the Google Storage (Bucket) with the script below
import logging
import string
import cloudstorage as gcs
from google.appengine.api import app_identity
def create_csv_file(self, filename, cursor=None):
filename = '/' + self.bucket_name + filename
try:
write_retry_params = gcs.RetryParams(backoff_factor=1.1)
# the cursor stores a MySQL result set
if cursor is not None:
gcs_file = gcs.open(filename,
'w',
content_type='text/csv',
retry_params=write_retry_params)
for row in cursor:
gcs_file.write(','.join(map(str, row)) + '\n')
gcs_file.close()
except Exception as ex:
logging.critical("Problem to write in th GC Storage with the exception:{}".format(ex))
raise ex
It works fine and store a CSV in the correct path inside Google storage. After that part, the next read/write operation is the signature of the file.
def cert_file(self, original_filename):
filename = '/' + self.bucket_name + original_filename
cert = Cert() # This class just has one method, that is that described in my original question and is used to sign the file.
with gcs.open(filename) as cloudstorage_file:
cloudstorage_file.seek(-1024, os.SEEK_END)
signed_file, encoded_signed_file = cert.sign_data(cloudstorage_file.read()) #the method to sign the file
signature_content = encoded_signed_file
signed_file_name = string.replace(original_filename, '.csv', '.txt.sha256')
encoded_signed_file_name = string.replace(signed_file_name, '.txt.sha256', '.txt.sha256.base64')
self.inner_upload_file(signed_file, signed_file_name)
self.inner_upload_file(encoded_signed_file, encoded_signed_file_name)
return signed_file_name, encoded_signed_file_name, signature_content
The inner_upload_file, just save the new files in the same bucket:
def inner_upload_file(self, file_data, filename):
filename = '/' + self.bucket_name + filename
try:
write_retry_params = gcs.RetryParams(backoff_factor=1.1)
gcs_file = gcs.open(filename,
'w',
content_type='application/octet-stream',
retry_params=write_retry_params)
gcs_file.write(file_data)
gcs_file.close()
except Exception as ex:
logging.critical("Problem to write in th GC Storage with the exception:{}".format(ex))
raise ex
Here is the app.yaml for reference. The cert.key and cert.crt generated by command line are stored in a static folder inside the app folder (the same directory where is my app.yaml).
UPDATE 2
Following the comments, I tried to run the signature proccess locally and then compare the files. Below is the step-by-setp and results.
First, I adapted the signature process to run as python sign.py file_name.
#!/usr/bin/python
import sys
import os
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from Crypto.Hash import SHA256
from base64 import b64encode, b64decode
file_path = 'static/cert.key'
key = open(file_path, "rb").read()
rsakey = RSA.importKey(key)
signer = PKCS1_v1_5.new(rsakey)
digest = SHA256.new()
file_object = open(sys.argv[1], "r")
digest.update(file_object.read())
sign = signer.sign(digest)
signed_path = "signed"
f = open(signed_path + '.txt.sha256', 'w')
f.write(sign)
f.close()
f2 = open(signed_path + '.txt.sha256.base64', 'w')
f2.write(b64encode(sign))
f2.close()
I ran the automatic proccess that saved the signed file in GCS's bucket (along with the original CSV file). After it I download both files through Google web panel for GCS.
I ran the command python sign.py gcs_file_original.csv in a virtualenv with python 2.7.10 using the CSV file I just downloaded.
After it, I compared the two signed files with cmp -b gcs_signed.txt.sha256 locally_signed.txt.sha256 resulting in:
gcs_signed.txt.sha256 locally_signed.txt.sha256 differ: byte 1, line 1 is 24 ^T 164 t
Using the VisualBinaryDiff, the result looks like two totally different files.
Now, I know the problem, but have no idea on how to fix it. This problem is beeing very trick.

I finally found the problem. I was so focused in find a problem in the openssl signature proccess and didn't pay attention to old Ctrl+C/Ctrl+V problem.
For test purposes, I copied the 'Read from GCS' example from this tutorial.
When I moved the test to the real world application, I didn't read the page again and didn't note the gcs_file.seek(-1024, os.SEEK_END).
As I said in the original question, I'm not Python specialist, but this line was reading just part of the GCS file, so the signature was indeed different from the original one.
Just cut that line of my reading methods and now all works fine.

Python 3.6: Reading a non-empty binary file is interpreted by Python to be empty

I have the following code, where 'password' is a string which is passed into the function. The issue is such that when I attempt to read the file created in the first half of the code, Python interprets it as being empty (despite the fact that File Explorer and text editors tell me it contains content). The 4 print statements are to assist with debugging (found here).
def encryptcredentials(username, password):
# Create key randomly and save to file
key = get_random_bytes(16)
keyfile = open("key.bin", "wb").write(key)
password = password.encode('utf-8')
path = "encrypted.bin"
# The following code generates a new AES128 key and encrypts a piece of data into a file
cipher = AES.new(key, AES.MODE_EAX)
ciphertext, tag = cipher.encrypt_and_digest(password)
file_out = open(path, "wb")
[file_out.write(x) for x in (cipher.nonce, tag, ciphertext)]
print("the path is {!r}".format(path))
print("path exists: ", os.path.exists(path))
print("it is a file: ", os.path.isfile(path))
print("file size is: ", os.path.getsize(path))
# At the other end, the receiver can securely load the piece of data back, if they know the key.
file_in = open(path, "rb")
nonce, tag, ciphertext = [file_in.read(x) for x in (16, 16, -1)]
The console output is as such:
the path is 'encrypted.bin'
path exists: True
it is a file: True
file size is: 0
Here's an image of how the file is displayed in File Explorer.
It appears that there's content in the .bin file produced at [file_out.write(x) for x in (cipher.nonce, tag, ciphertext)], but I can't get Python to read it.
Welcoming all suggestions. I'm running Python 3.6, 32-bit.

You have to close or even flush the file after file_out.write(x), so your data are writing from buffer to the file.
[file_out.write(x) for x in (cipher.nonce, tag, ciphertext)]
file_out.close()

Encrypt file Line by Line in Python using RSA and Compare it to another File

On my Linux Debian server, using python, I am trying to read a file of names line-by-line and encrypt it using the public RSA. Then I want to compare the encrypted line to another file that I have, which is also encrypted. If they are equal to each other I would like to print out the name in the decrypted and encrypted form. I have never used python before, so any help would be much appreciated.
#!/usr/bin/python
from Crypto.PublicKey import RSA
key = RSA.generate(2048)
names = open('names.txt')
cipher = open('ciphertext.txt',"r")
readname = names.readline()
readcipher = cipher.readlines()
while readname:
enc_name = pubkey.encrypt(names,0)
if enc_name == readcipher:
print readname
readname = names.readline()
names.close()
cipher.close()

I have no idea what your actual question is but maybe this will help?
for name_plaintext,name_encoded in zip(names,cipher):
if do_encode(name_plaintext) == name_encoded:
print name_decoded,"==>",name_encoded