Develop Reference

Python decode not converting to string

I am attempting to read a string from serial line using the code below, python keeps attaching the b' prefix and newline or return suffixes despite my telling it to convert to regular code and strip those out. Also, even if I send the text for 'FORWARD' to the device, it will not recognize the response.
Why wont python convert my text to regular format, and how do I get it to recognize my input.
#!/usr/bin/env python
import serial
ser = serial.Serial(port='/dev/ttyAMA0',baudrate = 9600,timeout=1)
while 1:
x=ser.readString()
x = x.decode()
x = x.strip()
print(x)
if x.find('FORWARD') >= 0:
print("FORWARD")
I expect it to show my serial input without the b' prefix or any \r\n suffixes, just the text I sent. I also expect it to recognize that the word "FORWARD" was in my input when I send that over the serial line. It dont do that, it shows b'FORWARD\r\n' and dont recognize that FORWARD is in the text

Hello I think decode('UTF-8') might help. I am working with binary data as well and you should specify which form you want it to be decoded to.
b'test'.decode('utf-8') == 'test' -> True
If the problem roots in that your string is binary which contains binary string, such as b'b"test"' then you can solve it with:
b'b"test"'.decode('utf-8').strip('b"') == 'test' -> True

Python encoding issue in script if string not hard-coded

I have an encoding issue with strings I get from an external source.
This source sends the strings encoded to me and I can decode them only if they are part of the script's code.
I've looked at several threads here and even some recommended tutorials (such as this one) but came up empty.
For example, if I run this:
python -c 'print "gro\303\237e"'
I get:
große
Which is the correct result.
But If I use it in a script, such as:
import sys
print sys.argv[1]
and call it like test.py "gro\303\237e", I get:
gro\303\237e
I intend to write the correct string to syslog, but I can't seem to get this to work.
Some data on my system:
- Python 2.7.10
- CentOS Linux
- LANG=en_US.UTF-8
- LC_CTYPE=UTF-8
I will appreciate any help, please let me know if you need more information.
Thanks!

If you really have the chars gro\303\237e which is something else as "gro\303\237e" (the first one are the chars g r o \ 3 0 3 \ 2 3 7, the second one is the chars g r o ß e) you can use decode("escape_string") as described in this SO answer
Note that this is probably an encoding error whoever produced the data. So it may contain other errors that you can not fix with this method.

This will work:
import sys
import ast
print ast.literal_eval('b"%s"' % sys.argv[1]).decode("utf-8")
But please read about literal_eval first to make sure it suits your needs (I think it should be safe to use but you should read and make sure).

base64 syntax in python is not working

this code works on the command line.
python -c 'import base64,sys; u,p=sys.argv[1:3]; print base64.encodestring("%s\x00%s\x00%s" % (u,u,p))' user pass
output is
dXNlcgB1c2VyAHBhc3M=
I am trying to get this to work in my script
test = base64.encodestring("{0}{0}{1}").format(acct_name,pw)
print test
output is
ezB9ezB9ezF9
anyone no what i am doing wrong ?
thank you.

You have a mistake in parenthesis. Instead of:
test = base64.encodestring("{0}{0}{1}").format(acct_name,pw)
(which first encodes "{0}{0}{1}" in base64 and then tries to substitute variables using format),
you should have
test = base64.encodestring("{0}{0}{1}".format(acct_name,pw))
(which first substitutes variables using format and then encodes in base64).

Thanks SZYM i am all set. This is the code that gets it to work
test = base64.encodestring("{0}\x00{0}\x00{1}".format(acct_name,pw))
Turns out the hex \x00 is needed so program getting the hash knows where username stops and password begins.
-ALF

Working with unicode encoded Strings from Active Directory via python-ldap

I already came up with this problem, but after some testing I decided to create a new question with some more specific Infos:
I am reading user accounts with python-ldap (and Python 2.7) from our Active Directory. This does work well, but I have problems with special chars. They do look like UTF-8 encoded strings when printed on the console. The goal is to write them into a MySQL DB, but I don't get those strings into proper UTF-8 from the beginning.
Example (fullentries is my array with all the AD entries):
fullentries[23][1].decode('utf-8', 'ignore')
print fullentries[23][1].encode('utf-8', 'ignore')
print fullentries[23][1].encode('latin1', 'ignore')
print repr(fullentries[23][1])
A second test with a string inserted by hand as follows:
testentry = "M\xc3\xbcller"
testentry.decode('utf-8', 'ignore')
print testentry.encode('utf-8', 'ignore')
print testentry.encode('latin1', 'ignore')
print repr(testentry)
The output of the first example ist:
M\xc3\xbcller
M\xc3\xbcller
u'M\\xc3\\xbcller'
Edit: If I try to replace the double backslashes with .replace('\\\\','\\) the output remains the same.
The output of the second example:
Müller
M�ller
'M\xc3\xbcller'
Is there any way to get the AD output properly encoded? I already read a lot of documentation, but it all states that LDAPv3 gives you strictly UTF-8 encoded strings. Active Directory uses LDAPv3.
My older question this topic is here: Writing UTF-8 String to MySQL with Python
Edit: Added repr(s) infos

First, know that printing to a Windows console is often the step that garbles data, so for your tests, you should print repr(s) to see the precise bytes you have in your string.
You need to find out how the data from AD is encoded. Again, print repr(s) will let you see the content of the data.
UPDATED:
OK, it looks like you're getting strange strings somehow. There might be a way to get them better, but you can adapt in any case, though it isn't pretty:
u.decode('unicode_escape').encode('iso8859-1').decode('utf8')
You might want to look into whether you can get the data in a more natural format.

sending raw bits to the terminal in python

As I understand it, files like /dev/urandom provide just a constant stream of bits. The terminal emulator then tries to interpret them as strings, which results in a mess of unrecognised characters.
How would I go about doing the same thing in python, send a string of ones and zeros to the terminal as "raw bits"?
edit
I may have to clarify:
Say for example the string I want to "print" is 1011100. On an ascii system, the output should be "\". If I cat /dev/urandom, it provides a constant stream of bits. Which get printed like this: "���c�g/�t]+__��-�;". That's what I want.

Stephano: the key is the incomplete answer by "#you" above - the chr function :
import random, sys
for i in xrange(500):
sys.stdout.write(chr(random.randrange(256)))

Use the chr function. I takes an input between 0 and 255 and returns a string containing the character corresponding to that value.

And from another question on StackOverflow you can get a _bin function.
def _bin(x, width):
return ''.join(str((x>>i)&1) for i in xrange(width-1,-1,-1))
Then simply put call _bin(ord(x), 8) where x is a character (string of length one)

import sys, random
while True:
sys.stdout.write(chr(random.getrandbits(8)))
sys.stdout.flush()