Develop Reference

Convert from ASCII to Hex in Python

I'm trying to convert a string with special characters from ASCII to Hex using python, but it doesn't seem that I'm getting the correct value, noting that it works just fine whenever I try to convert a string that has no special characters. So basically here is what I'm doing:
import binascii
s = "D`Cزف³›"
s_bytes = str.encode(s)
hex_value = str(binascii.hexlify(s_bytes),'ascii')
print (hex_value)
Output
446043d8b2d981c2b316e280ba
Where the output should be (using online converter https://www.rapidtables.com/convert/number/ascii-to-hex.html):
446043632641b3203a

str.encode(s) defaults to utf8 encoding, which doesn't give you the byte values needed to get the desired output. The values you want are simply Unicode ordinals as hexadecimal values, so get the ordinal, convert to hex and join them all together:
s = 'D`Cزف³›'
h = ''.join([f'{ord(c):x}' for c in s])
print(h)
446043632641b3203a
Just realize that Unicode ordinals can be 1-6 hexadecimal digits long, so there is no easy way to reverse the process since you have no spacing of the numbers.

How to Convert Each Character of String to ASCII and Pad to Given Length

Using Python 3.4, I am trying to convert a string to padded ASCII values (for each character).
Given a sting:
myString = "hello world"
Convert each character to ASCII and join back into a single string:
asciiString = ''.join(str(ord(c)) for c in myString)
This works great, returning:
'10410110810811132119111114108100'
I would like to pad EACH CHARACTER to 7 digits and then join. I've been trying to do this with format and have tried so many combinations of join, format, str, and ord that I can't even list them all. Can someone please help me figure out how to add format to the "asciiString =" line? Maybe this can't be done in a single line of code?

asciiString = ''.join(str('{num:07d}'.format(num=ord(c))) for c in myString)

Dealing with doubly escaped unicode string

I have a database of badly formatted database of strings. The data looks like this:
"street"=>"\"\\u4e2d\\u534e\\u8def\""
when it should be like this:
"street"=>"中华路"
The problem I have is that when that doubly escaped strings comes from the database they are not being decoded to the chinese characters as they should be. So suppose I have this variable; street="\"\\u4e2d\\u534e\\u8def\"" and if I print that print(street) the result is a string of codepoints "\u4e2d\u534e\u8def"
What can I do at this point to convert "\u4e2d\u534e\u8def" to actual unicode characters ?

First encode this string as utf8 and then decode it with unicode-escape which will handle the \\ for you:
>>> line = "\"\\u4e2d\\u534e\\u8def\""
>>> line.encode('utf8').decode('unicode-escape')
'"中华路"'
You can then strip the " if necessary

You could remove the quotation marks with strip and split at every '\\u'. This would give you the characters as strings representing hex numbers. Then for each string you could convert it to int and back to string with chr:
>>> street = "\"\\u4e2d\\u534e\\u8def\""
>>> ''.join(chr(int(x, 16)) for x in street.strip('"').split('\\u') if x)
'中华路'

Based on what you wrote, the database appears to be storing an eval-uable ascii representation of a string with non-unicode chars.
>>> eval("\"\\u4e2d\\u534e\\u8def\"")
'中华路'
Python has a built-in function for this.
>>> ascii('中华路')
"'\\u4e2d\\u534e\\u8def'"
The only difference is the use of \" instead of ' for the needed internal quote.

Python store non numeric string as number

I am currently trying to find a way to convert any sort of text to a number, so that it can later be converted back to text.
So something like this:
text = "some string"
number = somefunction(text)
text = someotherfunction(number)
print(text) #output "some string"

If you're using Python 3, it's pretty easy. First, convert the str to bytes in a chosen encoding (utf-8 is usually appropriate), then use int.from_bytes to convert to an int:
number = int.from_bytes(mystring.encode('utf-8'), 'little')
Converting back is slightly trickier (and will lose trailing NUL bytes unless you've stored how long the resulting string should be somewhere else; if you switch to 'big' endianness, you lose leading NUL bytes instead of trailing):
recoveredstring = number.to_bytes((number.bit_length() + 7) // 8, 'little').decode('utf-8')
You can do something similar in Python 2, but it's less efficient/direct:
import binascii
number = int(binascii.hexlify(mystring.encode('utf-8')), 16)
hx = '%x' % number
hx = hx.zfill(len(hx) + (len(hx) & 1)) # Make even length hex nibbles
recoveredstring = binascii.unhexlify(hx).decode('utf-8')
That's equivalent to the 'big' endian approach in Python 3; reversing the intermediate bytes as you go in each direction would get the 'little' effect.

You can use the ASCII values to do this:
ASCII to int:
ord('a') # = 97
Back to a string:
str(unichr(97)) # = 'a'
From there you could iterate over the string one character at a time and store these in another string. Assuming you are using standard ASCII characters, you would need to zero pad the numbers (because some are two digits and some three) like so:
s = 'My string'
number_string = ''
for c in s:
number_string += str(ord(c)).zfill(3)
To decode this, you will read the new string three characters at a time and decode them into a new string.
This assumes a few things:
all characters can be represented by ASCII (you could use Unicode code points if not)
you are storing the numeric value as a string, not as an actual int type (not a big deal in Python—saves you from having to deal with maximum values for int on different systems)
you absolutely must have a numeric value, i.e. some kind of hexadecimal representation (which could be converted into an int) and cryptographic algorithms won't work
we're not talking about GB+ of text that needs to be converted in this manner

Python3: Creating a string with an unescaped backslash

In Python 3.3, I am trying to rebuild unicode characters from truncated unicode values,
and then print the character to console.
For example, from '4E00' I want to form the string '\u4E00'. I have tried:
base = '4E00'
uni = r'\u' + base
print(uni) # getting '\u4E00', want: '一'
print(repr(uni)) # '\\u4E00'
Is there a way to form an unescaped string like '\u4E00' in this situation?

Keep in mind that \u followed by a Unicode character code is only a thing in string literals. r'\u' + '4E00' has no special meaning as a Unicode character because it's not all in one literal; it's just a six-character string.
So you're trying to take a Unicode escape code as it would appear in a Python string literal, then decode that into a Unicode character. You can do that:
base = '4E00'
uni = str(bytes(r'\u' + base, encoding="ascii"), encoding="unicode_escape")
But it's the long way around (especially since you have to convert it to bytes first since it's already Unicode). Your Unicode character spec is in hexadecimal. So convert it directly to an integer and then use chr() to turn it into a Unicode character.
base = '4E00'
uni = chr(int(base, 16))

Use:
chr(int(base, 16))
to turn a hex value into a Unicode character.
The \u escape sequence only works in string literals. You could use:
(br'\u' + base.encode('ascii')).decode('unicode_escape')
but that's much more verbose than this needs to be.
Demo:
>>> base = '4E00'
>>> chr(int(base, 16))
'一'
>>> (br'\u' + base.encode('ascii')).decode('unicode_escape')
'一'