Develop Reference

How can I SHA1-HMAC a string in Python

I've been trying to figure out how to use HMAC to hash a string in Python for a TOTP generator I've already made in PHP. In PHP, I use the command echo -n "0x00000000035362f9" | xxd -r -p | openssl dgst -sha1 -mac HMAC -macopt hexkey:0080700040024c8581c2 which returns the desired value 7289e7b135d54b86a462a53da93ef6ad28b902f8. However, when I use the hmac library in Python 3.10
from hashlib import sha1
from hmac import new
key = "00000000035362f9"
msg = "0080700040024c8581c2"
byte_key = bytes(key, "UTF-8")
message = msg.encode()
print(new(byte_key, message, sha1).hexdigest())
print(new(message, byte_key, sha1).hexdigest())
The printed values are b05fe172b6a8a20767c18e1bfba159f0ea54c2bd and 0fe109b32b17aeff840255558e6b5c8ff3d8a115, neither match what I want.
I've tried making the key and message hex values first, I've made them raw bytes using b'0080700040024c8581c2', encoding them using UTF-8 and ASCII, and none of the solutions have worked.
I've looked at other post relating to this, and none of them worked.
Python hmac (sha1) calculation
Python HMAC OpenSSL equivalent
hmac returning different hexdigest values to openssl
Why Python and Node.js's HMAC result is different in this code?
Implementing SHA1-HMAC with Python

#jasonharper's answer from the comments was correct.
from hashlib import sha1
from hmac import new
from binascii import unhexlify
msg = unhexlify("00000000035362f9")
key = unhexlify("0080700040024c8581c2")
print(new(key, msg, sha1).hexdigest())

How to import XOR function from Crypto.Cipher module?

cannot import name 'XOR' from 'Crypto.Cipher'
(/usr/local/lib/python3.8/dist-packages/Crypto/Cipher/__init__.py)
I just tried importing XOR function into my code & this is the error that i have got when i executed my code in the google colab.
Can i get the solution for this?
I just need to import XOR function using Crypto.Cipher module. My code is as follows
import Crypto
from Crypto import Cipher
from Crypto.Cipher import XOR
key = "abcdefghijklij"
xor = XOR.XORCipher(key) # To encrypt
xor1 = XOR.XORCipher(key) # To decrypt
def enc(sock, message, addr):
abcd = str_xor.encrypt(message)
print (message == dec(sock, abcd, addr))
sock.sendto(abcd, addr)
return abcd
def dec(sock, message, addr):
abcd = str_xor1.decrypt(message)
return abcd
#message = "dfjsdfjsdfjdsfdfsk"4
#print message
#newm = enc(1, message, message)
#print newm
#print dec(1, newm, newm)

pip install crypto
installs https://github.com/chrissimpkins/crypto which does not appear to be import-able class-library. Its examples and test scripts suggest crypto and decrypto should be executes as commands.
Readme: https://github.com/chrissimpkins/crypto
Tests/examples: https://github.com/chrissimpkins/crypto/tree/master/tests
Please specify which crypto-library did you install?
Make sure your installation matches the library you are supposed to install.

Error when serving attention_ocr model ("error": "Expected one or two output Tensors, found 17")

I'm trying to serve attention_ocr model on docker with tensorflow/serving image.
First, I trained this model with own dataset and get a good result with demo_inference.py
So, I'm export the trained model with export_model.py
python export_model.py --checkpoint=model.ckpt-111111 --export_dir=/tmp/mydir
Then, run docker container for serving the model.
docker run -it --rm -p 8501:8501 -v /tmp/mydir:/models/aocr -e MODEL_NAME=aocr --gpus all tensorflow/serving
And this is my python client script.
data_dir = '/root/src/models/research/attention_ocr/python/datasets/data/demo/'
data_files = os.listdir(data_dir)
with open(data_dir + "0.jpg", "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
## Some requests I tried ##
# predict_request = '{"examples": [{"inputs": ["%s"]}]}' % encoded_string
# predict_request = '{"examples": [{"inputs": "%s"}]}' % encoded_string
predict_request = '{"examples": [{"inputs": {"b64": ["%s"]}}]}' % encoded_string
r = requests.post('http://MY_IP_ADDR:8501/v1/models/aocr:classify', data=predict_request)
print(r.text)
Result.. "error": "Expected one or two output Tensors, found 17"
This is the first time using tensorflow/serving. I can't handle this error.
Please help this newbie.. Thanks in advance.

Thank you for reporting this issue. I filed a bug (#9264) on Github on your behalf. The issue is that the default signature includes all the endpoints that the model provides. If you want to use the Serving's Classification API, we need to modify the export_model script to export just the 2 tensors expected by the classification API (i.e., predictions and scores). In the meantime, you can use the Predict API, which supports an arbitrary number of output tensors. Please note that when using the predict API via GRPC you can specify output_filter, but the RESTful API does not have that option, so the response is pretty heavy, since it sends back all the attention masks and the raw image. In case somebody else is trying to figure out how to run inference, here are steps that worked for me.
Export the model:
wget http://download.tensorflow.org/models/attention_ocr_2017_08_09.tar.gz
tar -xzvf attention_ocr_2017_08_09.tar.gz
python model_export.py --checkpoint=model.ckpt-399731 \
--export_dir=/tmp/aocr_b1 --batch_size=1
Note that the --batch_size=1 is needed due to a bug in model_export.py. I'll take care of it when send the PR for the signature issue.
Run the docker container for serving the model.
sudo docker run -t --rm -p 8501:8501 \
-v /tmp/aocr_b1:/models/aocr/1 -e MODEL_NAME=aocr tensorflow/serving
Please note that the path needs to contain a version number /models/aocr/1. If you don't append /1 the server complains that it could not find any versions.
Run the script
python send_serving_request.py --image_file=testdata/fsns_train_00.png
Here are the results
Prediction: Rue de la Gare░░░░░░░░░░░░░░░░░░░░░░░
Confidence: 0.899479449
Here is the code:
send_serving_request.py
from absl import app
from absl import flags
import base64
import json
import os
from PIL import Image
import numpy as np
import requests
import tensorflow as tf
flags.DEFINE_string('image_file', None,
'Name of file containing image to request.')
def create_serialized_tf_example(image):
"""Create a serialized tf.Example proto for feeding the model."""
example = tf.train.Example()
example.features.feature['image/encoded'].float_list.value.extend(
list(np.reshape(image, (-1))))
return example.SerializeToString()
def main(_):
pil_image = Image.open(flags.FLAGS.image_file)
encoded_string = base64.b64encode(
create_serialized_tf_example(np.asarray(pil_image)))
predict_request = (
b'{"instances": [{"inputs": {"b64": "%s"}}]}') % encoded_string
r = requests.post(
'http://localhost:8501/v1/models/aocr:predict', data=predict_request)
data = json.loads(r.text)
print('Prediction:', data['predictions'][0]['predicted_text'])
print('Confidence:', data['predictions'][0]['normalized_seq_conf'])
if __name__ == '__main__':
flags.mark_flag_as_required('image_file')
app.run(main)

hmac returning different hexdigest values to openssl

For example when I run this command in my terminal:
echo -n 'something' | openssl dgst -sha256 -hmac 'NhqPtmdS'
This is returned:
caa686a03a502a0da2985dfea0b0b5798657fc30c2fd917db527d29ea5b23579
I'm trying to do this in Python, but I don't know why is returning something different.
This is my code:
import base64
from hashlib import sha256
import hmac
key = base64.b64decode('NhqPtmdS')
jsonBytes = bytes('something', "ascii")
hmac_result = hmac.new(key, jsonBytes, sha256).hexdigest()
print(hmac_result)
But I'm getting the next result:
6a964bd560a9dc763864ddf337d64e5f2ef958e6937ad296084166da0db83eb9
Also I tried this:
hmac_result = hmac.new(key, jsonBytes, sha256)
base64.b64encode(hmac_result.digest()).decode()
But it's not working either.
Any advice will be appreciated.

Non-ASCII character '\x97' in file, but no encoding declared

I'm trying to encode the contents of a Python script in Linux. I just started off with a simple script called test.py -
# !/app/logs/Python/bin/python3
# -*- coding: ascii -*-
print ("hi")
Once I have the script, I execute the vim -x test.py and enter the encryption key twice. Then save the file as normal and then execute the script using python test.py
I tried almost all the examples provided in the link here but still i end up getting the below error -
SyntaxError: Non-ASCII character '\x97' in file test.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
I checked the default encoding using print sys.getdefaultencoding() and it is acsii.
What am i missing here. kindly clarify. I know this question is a duplicate but none of the solutions helped.

Python knows how to execute clear text Python source code. If you encrypt the source file, it no longer contains valid Python source and cannot be directly executed.
There are 2 possible ways here. First is to only obfuscate your source. You should be aware that obfuscation is not security, and a user could recover some Python source with some work (not necessarily the original source, because comments and doc strings could have been stripped off and variable names could have been changed). You could read How do I protect Python code? and google for python obfuscate to find some possible ways of obfuscating Python source and their trade-offs.
The good news with obfuscated source is that it can be used by anybody without any password.
Second way is to encrypt the source. If you use a decent tool, you can assume that it will be impossible to read or execute the file without knowing the key. In that sense, vim crypto has not the highest possible reputation. In the simplest way (for example with your example vim -x) you will have to decrypt the file to execute it. Unfortunately, good crypto modules are not shipped in a standard Python installation and must be downloaded from pypi. Well known crypto modules include pycrypto and cryptography.
You can then encrypt the most part of the code, and then at run time ask for the key, decrypt it and execute it. Still a serious job but feasible.
Alternatively, you could build in another language (C/C++) a decryptor that decrypts the remaining of the file and feed it into a python interpretor, but functionally, this is only a variant of the above method.
As per your comment I assume that you want to encrypt the source code and decrypt it (with the password) at run time. The princips are:
build a Python script that will take another arbitrary Python script, encode it with a secure crypto module and prepend it with some decrypting code.
at run time, the prepended code will ask for the password, decrypt the encrypted code exec it
The builder could be (this code uses the cryptography module):
import cryptography.fernet
import cryptography.hazmat.primitives.kdf.pbkdf2
import cryptography.hazmat.primitives.hashes
import cryptography.hazmat.backends
import base64
import os
import sys
# the encryption function
def encrypt_source(infile, outfile, passwd):
with open(infile, 'rb') as fdin: # read original file
plain_data = fdin.read()
salt, key = gen_key(passwd) # derive a key from the password
f = cryptography.fernet.Fernet(key)
crypted_data = f.encrypt(plain_data) # encrypt the original code
with open(outfile, "w") as fdout: # prepend a decoding block
fdout.write("""#! /usr/bin/env python
import cryptography.fernet
import cryptography.hazmat.primitives.kdf.pbkdf2
import cryptography.hazmat.primitives.hashes
import cryptography.hazmat.backends
import base64
import os
def gen_key(passwd, salt): # key derivation
kdf = cryptography.hazmat.primitives.kdf.pbkdf2.PBKDF2HMAC(
algorithm = cryptography.hazmat.primitives.hashes.SHA256(),
length = 32,
salt = salt,
iterations = 100000,
backend = cryptography.hazmat.backends.default_backend()
)
return base64.urlsafe_b64encode(kdf.derive(passwd))
passwd = input("Password:") # ask for the password
salt = base64.decodebytes({})
key = gen_key(passwd.encode(), salt) # derive the key from the password and the original salt
crypted_source = base64.decodebytes( # decode (base64) the crypted source
b'''{}'''
)
f = cryptography.fernet.Fernet(key)
plain_source = f.decrypt(crypted_source) # decrypt it
exec(plain_source) # and exec it
""".format(base64.encodebytes(salt),
base64.encodebytes(crypted_data).decode()))
# derive a key from a password and a random salt
def gen_key(passwd, salt=None):
if salt is None: salt = os.urandom(16)
kdf = cryptography.hazmat.primitives.kdf.pbkdf2.PBKDF2HMAC(
algorithm = cryptography.hazmat.primitives.hashes.SHA256(),
length = 32,
salt = salt,
iterations = 100000,
backend = cryptography.hazmat.backends.default_backend()
)
return salt, base64.urlsafe_b64encode(kdf.derive(passwd))
if __name__ == '__main__':
if len(sys.argv) != 3:
print("Usage {} infile outfile".format(sys.argv[0]))
sys.exit(1)
passwd = input("Password:").encode() # ask for a password
encrypt_source(sys.argv[1], sys.argv[2], passwd) # and generates an encrypted Python script