Develop Reference

How NOT to print emojis from comments or submission when using praw

Getting error messages when I am trying to print out comments or submission with emojis in it. How can I just disregard and print only letters and numbers?
Using Praw to webscrape
top_posts2 = page.top(limit = 25)
for post in top_posts2:
outputFile.write(post.title)
outputFile.write(' ')
outputFile.write(str(post.score))
outputFile.write('\n')
outputFile.write(post.selftext)
outputFile.write('\n')
submissions = reddit.submission(id = post.id)
comment_page = submissions.comments
top_comment = comment_page[0] #by default, this will be the best comment of the post
commentBody = top_comment.body
outputFile.write(top_comment.body)
outputFile.write('\n')
I want to output only letters and numbers. and maybe some special characters (or all)

There's a couple ways you can do this. I would recommend creating kind of a "text cleaning" function
def cleanText(text):
new_text = ""
for c in text: # for each character in the text
if c.isalnum(): # check if it is either a letter or number (alphanumeric)
new_text += c
return new_text
or if you want to include specific non-alphanumeric numbers
def cleanText(text):
valid_symbols = "!##$%^&*()" # <-- add whatever symbols you want here
new_text = ""
for c in text: # for each character in the text
if c.isalnum() or c in valid_symbols: # check if alphanumeric or a valid symbol
new_text += c
return new_text
so then in your script you can do something like
commentBody = cleanText(top_comment.body)

LDAP search with username as variable

I am using the Python-LDAP module and trying to make a query on the logged in user. The username will be passed into the query. When I simply type the username in as a string my results come out correctly.
But if I try to pass the (username) variable it returns
LDAPError - FILTER_ERROR: {'desc': u'Bad search filter'} I've tried a number of different combinations but continue to get the same error returned. Any insight here would be great!
Edited for Minimal, Complete, and Verifiable example:
import ldap
LDAP_SERVER = "ldap://myldapserver.domain.ad:389"
username = r"domain\serviceAccount"
password = "Password"
l = ldap.initialize(LDAP_SERVER)
def login(username, password):
try:
l.simple_bind_s(username, password)
base = "OU=Users,OU=Group,DC=domain,DC=ad"
criteria = "(&(objectClass=user)(sAMAccountName=anActualUsername))" #WORKS
criteria = '(&(objectClass=user)(sAMAccountName=%s))' % username #DOESNT WORK
criteria = "(&(objectClass=user)" + "(sAMAccountName=" + username + "))" #DOESNT WORK
attributes = ['displayName']
result = l.search_s(base, ldap.SCOPE_SUBTREE, criteria, attributes)
print result
except ldap.INVALID_CREDENTIALS:
return False
return True
login(username,password)

Did you try to encode your string ?
criteria = ('(&(objectClass=user)(sAMAccountName=%s))' % username).encode('utf8')

In the "WORKS" case, your filter string contains a simple name with no domain:
(&(objectClass=user)(sAMAccountName=bobsmith))
In the "DOESN'T WORK" case, you use a name with a domain:
(&(objectClass=user)(sAMAccountName=domain\serviceAccount)
The character \ is not allowed in a filter string unless it is escaped.
How to fix this depends upon the data present in your ldap server. Perhaps this:
criteria = '(&(objectClass=user)(sAMAccountName=%s))' % (
username if '\\' not in username else username.split('\\')[1])
Or perhaps this:
criteria = '(&(objectClass=user)(sAMAccountName=%s))' % (
ldap.filter.escape_filter_chars(username))

I needed to use ldap.filter.filter_format for proper character escaping.
import ldap.filter
criteria= ldap.filter.filter_format('(&(objectClass=user)(sAMAccountName=%s))', [username])

Try switching single quotes with double quotes.
criteria = "(&(objectClass=user)(sAMAccountName=anActualUsername))" #WORKS
criteria = '(&(objectClass=user)(sAMAccountName=%s))' % username #DOESNT WORK
the second criteria change it to this one (I didn't try with %s but only string):
criteria = "(&(objectClass=user)(sAMAccountName=%s))" % username #SHOULD WORK

Finding a recurrance of three ASCII families in a string

I'm trying to create a 'check' system for a password generator that will advsie whether or not three of the same types of character family are found in a row in a generated password, i.e
If the password is
y8kpBD8zcZLKRSh1j7vwCMDQ5orR8VEP
it will find 'ZLK' etc
I first thought lowercase_repeat = re.compile("[a-z]{3}") would for example find three lowercase repeats, but I can't seem to understand how this works exactly.
The password generator is below:
import random
import re
generator = random.SystemRandom()
password_characters = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ123456789!##$%^&*()'
password = ''.join(generator.choice(password_characters) for _ in range(32))
print password

If you just want to check for specific character sets; e.g: all uppercase, all lowercase, digit and non-alnum - you can create a non-capturing group for each set. For example:
import re
pattern = '(?:[a-z]{3}|[A-Z]{3}|\d{3}|[\x20-\x2F\x3A-\x40\x5B-\x60\x7B-\x7E]{3})'
password = 'y8kpBD8zcZLKRSh1j7vwCMDQ5orR8VEP!'
matches = re.search(pattern, password)
The variable matches returns None if there are no matches, indicating the password passes.
The pattern [\x20-\x2F\x3A-\x40\x5B-\x60\x7B-\x7E] is a (probably pretty gnarly) way to catch a set of all non-alnum ascii characters (hex codes). It represents the following set:
[space] ! " # $ % & ' ( ) * + , - . / : ; < = > ? # [ \ ] ^ _ ` { | } ~
I pulled it out of an old project, so YMMV. I'm sure there might be a more succinct way to express it - indeed, you might prefer to explicitly specify a set; e.g: [!?#] etc.
Quick sanity-check:
import re
def check_password(password):
pattern = '(?:[a-z]{3}|[A-Z]{3}|\d{3}|[\x20-\x2F\x3A-\x40\x5B-\x60\x7B-\x7E]{3})'
return re.search(pattern, password)
passwords = ['a', 'abc', 'ABC', 'aBc', '1bc', '123']
for password in passwords:
if check_password(password):
print 'password failed: ', password
else:
print 'password passed: ', password
Yields:
password passed: a
password failed: abc
password failed: ABC
password passed: aBc
password passed: 1bc
password failed: 123
Hope this helps :)

Python - encryption and decryption scripts produce occasional errors

I made a Python script to encrypt plaintext files using the symmetric-key algorithm described in this video. I then created a second script to decrypt the encrypted message. Here is the original text:
I came, I saw, I conquered.
Here is the text after being encrypted and decrypted:
I came, I saw, I conquerdd.
Almost perfect, except for a single letter. For longer texts, there will be multiple letters which are just off ie the numerical representation of the character which appears is one lower than the numerical representation of the original character. I have no idea why this is.
Here's how my scripts work. First, I generated a random sequence of digits -- my PAD -- and saved it in the text file "pad.txt". I won't show the code because it is so straightforward. I then saved the text which I want to be encrypted in "text.txt". Next, I run the encryption script, which encrypts the text and saves it in the file "encryptedText.txt":
#!/usr/bin/python3.4
import string
def getPad():
padString = open("pad.txt","r").read()
pad = padString.split(" ")
return pad
def encrypt(textToEncrypt,pad):
encryptedText = ""
possibleChars = string.printable[:98] # last two elements are not used bec
# ause they don't show up well on te
# xt files.
for i in range(len(textToEncrypt)):
char = textToEncrypt[i]
if char in possibleChars:
num = possibleChars.index(char)
else:
return False
encryptedNum = num + int(pad[(i)%len(pad)])
if encryptedNum >= len(possibleChars):
encryptedNum = encryptedNum - len(possibleChars)
encryptedChar = possibleChars[encryptedNum]
encryptedText = encryptedText + encryptedChar
return encryptedText
if __name__ == "__main__":
textToEncrypt = open("text.txt","r").read()
pad = getPad()
encryptedText = encrypt(textToEncrypt,pad)
if not encryptedText:
print("""An error occurred during the encryption process. Confirm that \
there are no forbidden symbols in your text.""")
else:
open("encryptedText.txt","w").write(encryptedText)
Finally, I decrypt the text with this script:
#!/usr/bin/python3.4
import string
def getPad():
padString = open("pad.txt","r").read()
pad = padString.split(" ")
return pad
def decrypt(textToDecrypt,pad):
trueText = ""
possibleChars = string.printable[:98]
for i in range(len(textToDecrypt)):
encryptedChar = textToDecrypt[i]
encryptedNum = possibleChars.index(encryptedChar)
trueNum = encryptedNum - int(pad[i%len(pad)])
if trueNum < 0:
trueNum = trueNum + len(possibleChars)
trueChar = possibleChars[trueNum]
trueText = trueText + trueChar
return trueText
if __name__ == "__main__":
pad = getPad()
textToDecrypt = open("encryptedText.txt","r").read()
trueText = decrypt(textToDecrypt,pad)
open("decryptedText.txt","w").write(trueText)
Both scripts seem very straightforward, and they obvious work almost perfectly. However, every once in a while there is an error and I cannot see why.

I found the solution to this problem. It turns out that every character that was not decrypted properly was encrypted to \r, which my text editor changed to a \n for whatever reason. Removing \r from the list of possible characters fixed the issue.

My python encryption software isn't working

So I wrote a little script in python that brings up a gui with 2 buttons and a text field. you type into the text area and click encrypt and the text all of a sudden looks crazy. coooooool.
but then when you hit decrypt, it doesn't go back to the original. here is the code
from Tkinter import *
import ttk
root = Tk()
textArea = Text(root, state = "normal")
def encrypt():
message = textArea.get('1.0', 'end')
newMessage = ''
lastChar = ''
for c in message:
if lastChar != '':
newMessage += chr((ord(c) + ord(lastChar)) % 256)
lastChar = chr((ord(c) + ord(lastChar)) % 256)
else:
newMessage += chr((ord(c) + 5) % 256)
lastChar = chr((ord(c) + 5) % 256)
textArea.delete('1.0', 'end')
textArea.insert('end', newMessage)
def decrypt():
message = textArea.get('1.0', 'end')
newMessage = ''
lastChar = ''
for c in message:
if lastChar != '':
newMessage += chr((ord(c) - ord(lastChar)) % 256)
lastChar = chr((ord(c) - ord(lastChar)) % 256)
else:
newMessage += chr((ord(c) - 5) % 256)
lastChar = chr((ord(c) - 5) % 256)
textArea.delete('1.0', 'end')
textArea.insert('end', newMessage)
encrypt = ttk.Button(root, text = "Encrypt", command = encrypt)
encrypt.pack({"side": "top"})
decrypt = ttk.Button(root, text = "Decrypt", command = decrypt)
decrypt.pack({"side": "top"})
textArea.pack({"side": "bottom"});
mainloop()
the problem is that it doesn't show the original text. It just seems to make it more cryptic. what is wrong here? Please help.
update:
a changed it so it just adds 5. and It works. So that tells me that it's the part where I add last characters code value. There is still one problem: it adds a new line and this strange line character (not the pipe ---> |).
now the code is fixed. thanks. here is the result:
WW1ueCVueCV1d2p5eX4laHR0cTMlWW1mc3AlfnR6JXh5ZmhwJXR7andrcXR8JWt0dyV5bWolZnN4fGp3Mw8=
after I made the decrypt so It doesn't take away five, it worked. Thanks again.

You can't put arbitrary bytes into a text area.
Your encryption algorithm is purely bytewise - it works on the numeric values of the characters. This isn't going to work, even for ASCII, because byte 0 is an ASCII NUL - treated specially, and byte 127 (what 'Z', character 122, becomes when shifted up by five) is a DEL character: not what you want! Moving beyond ASCII, the real world's UTF-8 has thousands of invalid byte sequences.
If you're going to be using bytewise encryption like this, you can't put the results into a text field - it's binary data. If you want it to be able to go into a text field, you must make sure it's valid text without control characters, NUL bytes, or invalid byte sequences.
One easy way of making the binary sanitary is to Base64-encode it on encryption, and have the first step of decryption be Base64 decoding.
Final note: you're also encrypting the newline at the end of the text area. Probably not something you wish to do.

This seems a rather trivial "encryption" to break. Everyone who has the source can read your messages. A secure cryptosystem should not depend on the algorithm being secure.
You could use the Python Cryptography Toolkit (www.dlitz.net/software/pycrypto/). It has a host of different encryption algorithms available.
You can find some examples of how to use it at: http://www.laurentluce.com/posts/python-and-cryptography-with-pycrypto/
If you don't care about security, why not use e.g. one of the standard encodings (see §7.8 'codec' of the standard library reference);
The Ceasar cipher:
>>> res = 'this is a test'.encode('rot_13')
>>> print res
guvf vf n grfg
>>> print res.decode('rot_13')
this is a test
Base64 encoding:
>>> res = 'this is a test'.encode('base64')
>>> print res
dGhpcyBpcyBhIHRlc3Q=
>>> print res.decode('base64')
this is a test