Develop Reference

How can I SHA1-HMAC a string in Python

I've been trying to figure out how to use HMAC to hash a string in Python for a TOTP generator I've already made in PHP. In PHP, I use the command echo -n "0x00000000035362f9" | xxd -r -p | openssl dgst -sha1 -mac HMAC -macopt hexkey:0080700040024c8581c2 which returns the desired value 7289e7b135d54b86a462a53da93ef6ad28b902f8. However, when I use the hmac library in Python 3.10
from hashlib import sha1
from hmac import new
key = "00000000035362f9"
msg = "0080700040024c8581c2"
byte_key = bytes(key, "UTF-8")
message = msg.encode()
print(new(byte_key, message, sha1).hexdigest())
print(new(message, byte_key, sha1).hexdigest())
The printed values are b05fe172b6a8a20767c18e1bfba159f0ea54c2bd and 0fe109b32b17aeff840255558e6b5c8ff3d8a115, neither match what I want.
I've tried making the key and message hex values first, I've made them raw bytes using b'0080700040024c8581c2', encoding them using UTF-8 and ASCII, and none of the solutions have worked.
I've looked at other post relating to this, and none of them worked.
Python hmac (sha1) calculation
Python HMAC OpenSSL equivalent
hmac returning different hexdigest values to openssl
Why Python and Node.js's HMAC result is different in this code?
Implementing SHA1-HMAC with Python

#jasonharper's answer from the comments was correct.
from hashlib import sha1
from hmac import new
from binascii import unhexlify
msg = unhexlify("00000000035362f9")
key = unhexlify("0080700040024c8581c2")
print(new(key, msg, sha1).hexdigest())

'utf-8' codec can't decode byte: invalid continuation byte

i am trying to retrain a model with tensorflow. I used the code from Tensorflow Models while following a Tutorial on Youtube. The process ends with the following error:
File "C:\Users\Jeff\anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\util\compat.py", line 109, in as_text
return bytes_or_text.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd3 in position 9: invalid continuation byte
The Error happens to be in the following script compat.py in the highlighted line in the method "as_text":
def as_text(bytes_or_text, encoding='utf-8'):
if isinstance(bytes_or_text, _six.text_type):
return bytes_or_text
elif isinstance(bytes_or_text, bytes):
**return bytes_or_text.decode(encoding)**
else:
raise TypeError('Expected binary or unicode string, got %r' % bytes_or_text)
I googled for hours but couldn't find anything. I tried different pre-trained models and startet the installation and training configuration all over again. I am out of ideas and hope someone can help me here.
Can anybody tell me, what i can do to make it work? Thanks in advance!
Here the whole script compat.py :
# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Compatibility functions.
The `tf.compat` module contains two sets of compatibility functions.
## Tensorflow 1.x and 2.x APIs
The `compat.v1` and `compat.v2` submodules provide a complete copy of both the
`v1` and `v2` APIs for backwards and forwards compatibility across TensorFlow
versions 1.x and 2.x. See the
[migration guide](https://www.tensorflow.org/guide/migrate) for details.
## Utilities for writing compatible code
Aside from the `compat.v1` and `compat.v2` submodules, `tf.compat` also contains
a set of helper functions for writing code that works in both:
* TensorFlow 1.x and 2.x
* Python 2 and 3
## Type collections
The compatibility module also provides the following aliases for common
sets of python types:
* `bytes_or_text_types`
* `complex_types`
* `integral_types`
* `real_types`
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import numbers as _numbers
import numpy as _np
import six as _six
from tensorflow.python.util.tf_export import tf_export
try:
# This import only works on python 3.3 and above.
import collections.abc as collections_abc # pylint: disable=unused-import
except ImportError:
import collections as collections_abc # pylint: disable=unused-import
def as_bytes(bytes_or_text, encoding='utf-8'):
"""Converts `bytearray`, `bytes`, or unicode python input types to `bytes`.
Uses utf-8 encoding for text by default.
Args:
bytes_or_text: A `bytearray`, `bytes`, `str`, or `unicode` object.
encoding: A string indicating the charset for encoding unicode.
Returns:
A `bytes` object.
Raises:
TypeError: If `bytes_or_text` is not a binary or unicode string.
"""
if isinstance(bytes_or_text, bytearray):
return bytes(bytes_or_text)
elif isinstance(bytes_or_text, _six.text_type):
return bytes_or_text.encode(encoding)
elif isinstance(bytes_or_text, bytes):
return bytes_or_text
else:
raise TypeError('Expected binary or unicode string, got %r' %
(bytes_or_text,))
def as_text(bytes_or_text, encoding='utf-8'):
"""Converts any string-like python input types to unicode.
Returns the input as a unicode string. Uses utf-8 encoding for text
by default.
Args:
bytes_or_text: A `bytes`, `str`, or `unicode` object.
encoding: A string indicating the charset for decoding unicode.
Returns:
A `unicode` (Python 2) or `str` (Python 3) object.
Raises:
TypeError: If `bytes_or_text` is not a binary or unicode string.
"""
if isinstance(bytes_or_text, _six.text_type):
return bytes_or_text
elif isinstance(bytes_or_text, bytes):
return bytes_or_text.decode(encoding)
else:
raise TypeError('Expected binary or unicode string, got %r' % bytes_or_text)
def as_str(bytes_or_text, encoding='utf-8'):
if _six.PY2:
return as_bytes(bytes_or_text, encoding)
else:
return as_text(bytes_or_text, encoding)
tf_export('compat.as_text')(as_text)
tf_export('compat.as_bytes')(as_bytes)
tf_export('compat.as_str')(as_str)
#tf_export('compat.as_str_any')
def as_str_any(value):
"""Converts input to `str` type.
Uses `str(value)`, except for `bytes` typed inputs, which are converted
using `as_str`.
Args:
value: A object that can be converted to `str`.
Returns:
A `str` object.
"""
if isinstance(value, bytes):
return as_str(value)
else:
return str(value)
#tf_export('compat.path_to_str')
def path_to_str(path):
r"""Converts input which is a `PathLike` object to `str` type.
Converts from any python constant representation of a `PathLike` object to
a string. If the input is not a `PathLike` object, simply returns the input.
Args:
path: An object that can be converted to path representation.
Returns:
A `str` object.
Usage:
In case a simplified `str` version of the path is needed from an
`os.PathLike` object
Examples:
```python
$ tf.compat.path_to_str('C:\XYZ\tensorflow\./.././tensorflow')
'C:\XYZ\tensorflow\./.././tensorflow' # Windows OS
$ tf.compat.path_to_str(Path('C:\XYZ\tensorflow\./.././tensorflow'))
'C:\XYZ\tensorflow\..\tensorflow' # Windows OS
$ tf.compat.path_to_str(Path('./corpus'))
'corpus' # Linux OS
$ tf.compat.path_to_str('./.././Corpus')
'./.././Corpus' # Linux OS
$ tf.compat.path_to_str(Path('./.././Corpus'))
'../Corpus' # Linux OS
$ tf.compat.path_to_str(Path('./..////../'))
'../..' # Linux OS
```
"""
if hasattr(path, '__fspath__'):
path = as_str_any(path.__fspath__())
return path
def path_to_bytes(path):
r"""Converts input which is a `PathLike` object to `bytes`.
Converts from any python constant representation of a `PathLike` object
or `str` to bytes.
Args:
path: An object that can be converted to path representation.
Returns:
A `bytes` object.
Usage:
In case a simplified `bytes` version of the path is needed from an
`os.PathLike` object
"""
if hasattr(path, '__fspath__'):
path = path.__fspath__()
return as_bytes(path)
# Numpy 1.8 scalars don't inherit from numbers.Integral in Python 3, so we
# need to check them specifically. The same goes from Real and Complex.
integral_types = (_numbers.Integral, _np.integer)
tf_export('compat.integral_types').export_constant(__name__, 'integral_types')
real_types = (_numbers.Real, _np.integer, _np.floating)
tf_export('compat.real_types').export_constant(__name__, 'real_types')
complex_types = (_numbers.Complex, _np.number)
tf_export('compat.complex_types').export_constant(__name__, 'complex_types')
# Either bytes or text.
bytes_or_text_types = (bytes, _six.text_type)
tf_export('compat.bytes_or_text_types').export_constant(__name__,
'bytes_or_text_types')

php - need help converting a python command to php code sha256

I'm using the python hashlib to get a condensed sha256 value using the following command, can someone help me out with a compatible function?
import hashlib
biz_number = "000-00-00000"
total_amount = "45000"
external_auth_key = "00000000-0000-0000-0000-000000000000"
signature = hashlib.sha256(':'.join([biz_number, external_auth_key, total_amount]).encode()).hexdigest()
print(signature)
> b9c4510deb7e279687bac8503d12f0878e0955a0f247b9257cb600df10378895

Convert Bytes[] to Python Bytes Using PythonNet

I have a PythonNet C# bytes[] object and I need to convert it to Python's bytes.
Is there a way to do this using Python?
Here is what I'm doing to get the bytes. I need to go into the winstore's credential store and export out a client certificate.
import clr, os
import requests
from cryptography.hazmat.primitives.serialization.pkcs12 import (
load_key_and_certificates,
)
from cryptography.hazmat.primitives.serialization import Encoding, NoEncryption
from cryptography.hazmat.backends import default_backend
clr.AddReference("System")
clr.AddReference("System.Security.Cryptography.X509Certificates")
clr.AddReference("System.Security.Cryptography")
from System.Security.Cryptography.X509Certificates import (
X509Store,
StoreName,
StoreLocation,
OpenFlags,
X509CertificateCollection,
X509FindType,
X509Certificate2,
X509ContentType,
)
store = X509Store(StoreName.My, StoreLocation.CurrentUser)
store.Open(OpenFlags.ReadOnly)
serial_number = (
'SUPER SECRET SERIAL NUMBER'
)
collection = store.Certificates.Find(
X509FindType.FindBySerialNumber, serial_number, False
)
cert = collection.get_Item(0)
pkcs12 = cert.Export(X509ContentType.Pkcs12, "<Secret Password>")
# I have System.Bytes[] need python bytes to use with pyopenssl/cryptography
This code is using the pythonnet library to access the crypto API on windows. It then dumps a client certificate, found by a serial number, to bytes. When the clr library returns the value, it's a system.Bytes[]. This is incompatible with other libraries, so I need a way to convert this object to Python bytes.

You should be able to use the Python function bytes (as #JabberJabber mentioned). The following works for me (c_sharp_bytes actually being the result of a C# function) with Python 3.8:
from System import Byte, Array
python_bytes = b'Some bytes'
# Python -> C#
c_sharp_bytes = Array[Byte](python_bytes) # <System.Byte[] object at 0x000001A33CC2A3D0>
# C# -> Python
python_bytes = bytes(c_sharp_bytes) # b'Some bytes'

How to compress the string using gzip and encode in python 2.7

I'm trying to compress and encode my string using gGIP in python 2.7, I'm able to do that in Python 3 but I'm not getting same output in Python 2.7 version code:
Python 3:
import sys
import redis
import io import StringIO
import gzip
redisConn = redis.StrictRedis(host="127.0.0.1", port=6379, db=0)
myValue = "This is test data"
result = gzip.compress(myValue.encode())
redisConn.set("myKey", result)
Python 2.7:
import sys
import redis
import StringIO
import gzip
redisConn = redis.StrictRedis(host="127.0.0.1", port=6379, db=0)
myValue = "This is test data"
out = StringIO.StringIO()
gzip_s = gzip.GzipFile(fileobj=out, mode="w")
result = gzip_s.write(myValue.encode())
redisConn.set("myKey", result)
But Python 2.7 version code is breaking I'm getting an error:
'int' object has no attribute 'encode'
Can someone please help what's the equivalent code of Python 2.7 - my Python 3 version is working as expected.
Thanks for your help in advance.

Python 2 doesn't make the distinction between strings and bytes (even if the gzip stream is open as binary like here). You can write strings in a binary stream without needing to encode it. It has some drawbacks but in your case just remove the .encode() call:
gzip_s.write(myValue)
for a Python 2/3 agnostic code, I would simply do:
if bytes is str:
# python 2, no need to do anything
pass
else:
# python 3+: encode string as bytes
myValue = myValue.encode()
gzip_s.write(myValue)
EDIT: since you seem to issue a command redisConn.set("myKey", result), don't forget to call:
gzip_s.close()
before that, or it's not guaranteed that the file is fully flushed.

Here's a complete working example for Python 2.7. Note that gzip_s.write() returns the number of bytes written, so your code is passing an int to redisConn.set("myKey", result). Also, you probably should explicitly encode and decode the data to avoid unexpected encoding/decoding errors if you ever need to store non-ASCII data.
# -*- coding: utf-8 -*-
import redis
import StringIO
import gzip
redis_cxn = redis.StrictRedis(host='127.0.0.1', port=6379, db=0)
test_data = u'This is some test data with a non-ASCII character: ñ'
print 'Test data:\n ', test_data
out_file = StringIO.StringIO()
gzip_file = gzip.GzipFile(fileobj=out_file, mode='wb')
# If you don't encode the data yourself, it will be implicitly encoded as
# ASCII.
gzip_file.write(test_data.encode('utf-8'))
gzip_file.close()
# Get the bytes written to the underlying file object
value = out_file.getvalue()
print 'Setting value in Redis'
redis_cxn.set('key', value)
print 'Getting value from Redis'
retrieved_value = redis_cxn.get('key')
assert retrieved_value == value
in_file = StringIO.StringIO()
in_file.write(retrieved_value)
in_file.seek(0)
gzip_file = gzip.GzipFile(fileobj=in_file, mode='rb')
retrieved_data = gzip_file.read()
retrieved_data = retrieved_data.decode('utf-8')
gzip_file.close()
assert retrieved_data == test_data
print 'Data retrieved from Redis and unzippped:\n ', test_data