Develop Reference

What's the default encoding for Sha1 in Python 2?

I have an Api code from Python 2 is as below:
dataStr = request.POST['data']
data = json.loads(dataStr)
key_string = data.get('key')
seed = data.get('seed')
guid = data.get('guid')
sha1 = hashlib.sha1()
yesStr = seed + 'yes' + key_string + 'ke.yp.ad' + guid
sha1.update(yesStr)
yesValue = sha1.hexdigest()
resp = {
'data': yesValue,
'place_id': str(record.GroupNumber)
}
return HttpResponse(resp)
now I am upgrading it to python 3, but sha1.update must use an encoded string. but I don't know what the encoding format was. so far I tried
sha1.udpate(yesStr.encode("utf-8"))
sha1.udpate(yesStr.encode("ascii"))
but none of them is matching the old string. Can anyone help? thanks.

Finally I figured this out with my friend's help. The default encoding for Python 2 is 'latin1'. So what I did to fix the problem is to change
sha1.update(yesStr)
to:
sha1.update(yesStr.encode('latin1'))
and it worked in python 3.

How do I make it so I only need my api key referenced once?

I am teaching myself how to use python and django to access the google places api to make nearby searches for different types of gyms.
I was only taught how to use python and django with databases you build locally.
I wrote out a full Get request for they four different searches I am doing. I looked up examples but none seem to work for me.
allgyms = requests.get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=38.9208,-77.036&radius=2500&type=gym&key=AIzaSyDOwVK7bGap6b5Mpct1cjKMp7swFGi3uGg')
all_text = allgyms.text
alljson = json.loads(all_text)
healthclubs = requests.get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=38.9208,-77.036&radius=2500&type=gym&keyword=healthclub&key=AIzaSyDOwVK7bGap6b5Mpct1cjKMp7swFGi3uGg')
health_text = healthclubs.text
healthjson = json.loads(health_text)
crossfit = requests.get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=38.9208,-77.036&radius=2500&type=gym&keyword=crossfit&key=AIzaSyDOwVK7bGap6b5Mpct1cjKMp7swFGi3uGg')
cross_text = crossfit.text
crossjson = json.loads(cross_text)
I really would like to be pointed in the right direction on how to have the api key referenced only one time while changing the keywords.

Try this for better readability and better reusability
BASE_URL = 'https://maps.googleapis.com/maps/api/place/nearbysearch/json?'
LOCATION = '38.9208,-77.036'
RADIUS = '2500'
TYPE = 'gym'
API_KEY = 'AIzaSyDOwVK7bGap6b5Mpct1cjKMp7swFGi3uGg'
KEYWORDS = ''
allgyms = requests.get(BASE_URL+'location='+LOCATION+'&radius='+RADIUS+'&type='+TYPE+'&key='+API_KEY) all_text = allgyms.text
alljson = json.loads(all_text)
KEYWORDS = 'healthclub'
healthclubs = requests.get(BASE_URL+'location='+LOCATION+'&radius='+RADIUS+'&type='+TYPE+'&keyword='+KEYWORDS+'&key='+API_KEY)
health_text = healthclubs.text
healthjson = json.loads(health_text)
KEYWORDS = 'crossfit'
crossfit = requests.get(BASE_URL+'location='+LOCATION+'&radius='+RADIUS+'&type='+TYPE+'&keyword='+KEYWORDS+'&key='+API_KEY)
cross_text = crossfit.text
crossjson = json.loads(cross_text)
as V-R suggested in a comment you can go further and define function which makes things more reusable allowing you to use the that function in other places of your application
Function implementation
def makeRequest(location, radius, type, keywords):
BASE_URL = 'https://maps.googleapis.com/maps/api/place/nearbysearch/json?'
API_KEY = 'AIzaSyDOwVK7bGap6b5Mpct1cjKMp7swFGi3uGg'
result = requests.get(BASE_URL+'location='+location+'&radius='+radius+'&type='+type+'&keyword='+keywords+'&key='+API_KEY)
jsonResult = json.loads(result)
return jsonResult
Function invocation
json = makeRequest('38.9208,-77.036', '2500', 'gym', '')
Let me know if there is an issue

How can I talk to UniProt over HTTP in Python?

I'm trying to get some results from UniProt, which is a protein database (details are not important). I'm trying to use some script that translates from one kind of ID to another. I was able to do this manually on the browser, but could not do it in Python.
In http://www.uniprot.org/faq/28 there are some sample scripts. I tried the Perl one and it seems to work, so the problem is my Python attempts. The (working) script is:
## tool_example.pl ##
use strict;
use warnings;
use LWP::UserAgent;
my $base = 'http://www.uniprot.org';
my $tool = 'mapping';
my $params = {
from => 'ACC', to => 'P_REFSEQ_AC', format => 'tab',
query => 'P13368 P20806 Q9UM73 P97793 Q17192'
};
my $agent = LWP::UserAgent->new;
push #{$agent->requests_redirectable}, 'POST';
print STDERR "Submitting...\n";
my $response = $agent->post("$base/$tool/", $params);
while (my $wait = $response->header('Retry-After')) {
print STDERR "Waiting ($wait)...\n";
sleep $wait;
print STDERR "Checking...\n";
$response = $agent->get($response->base);
}
$response->is_success ?
print $response->content :
die 'Failed, got ' . $response->status_line .
' for ' . $response->request->uri . "\n";
My questions are:
1) How would you do that in Python?
2) Will I be able to massively "scale" that (i.e., use a lot of entries in the query field)?

question #1:
This can be done using python's urllibs:
import urllib, urllib2
import time
import sys
query = ' '.join(sys.argv)
# encode params as a list of 2-tuples
params = ( ('from','ACC'), ('to', 'P_REFSEQ_AC'), ('format','tab'), ('query', query))
# url encode them
data = urllib.urlencode(params)
url = 'http://www.uniprot.org/mapping/'
# fetch the data
try:
foo = urllib2.urlopen(url, data)
except urllib2.HttpError, e:
if e.code == 503:
# blah blah get the value of the header...
wait_time = int(e.hdrs.get('Retry-after', 0))
print 'Sleeping %i seconds...' % (wait_time,)
time.sleep(wait_time)
foo = urllib2.urlopen(url, data)
# foo is a file-like object, do with it what you will.
foo.read()

You're probably better off using the Protein Identifier Cross Reference service from the EBI to convert one set of IDs to another. It has a very good REST interface.
http://www.ebi.ac.uk/Tools/picr/
I should also mention that UniProt has very good webservices available. Though if you are tied to using simple http requests for some reason then its probably not useful.

Let's assume that you are using Python 2.5.
We can use httplib to directly call the web site:
import httplib, urllib
querystring = {}
#Build the query string here from the following keys (query, format, columns, compress, limit, offset)
querystring["query"] = ""
querystring["format"] = "" # one of html | tab | fasta | gff | txt | xml | rdf | rss | list
querystring["columns"] = "" # the columns you want comma seperated
querystring["compress"] = "" # yes or no
## These may be optional
querystring["limit"] = "" # I guess if you only want a few rows
querystring["offset"] = "" # bring on paging
##From the examples - query=organism:9606+AND+antigen&format=xml&compress=no
##Delete the following and replace with your query
querystring = {}
querystring["query"] = "organism:9606 AND antigen"
querystring["format"] = "xml" #make it human readable
querystring["compress"] = "no" #I don't want to have to unzip
conn = httplib.HTTPConnection("www.uniprot.org")
conn.request("GET", "/uniprot/?"+ urllib.urlencode(querystring))
r1 = conn.getresponse()
if r1.status == 200:
data1 = r1.read()
print data1 #or do something with it
You could then make a function around creating the query string and you should be away.

check this out bioservices. they interface a lot of databases through Python.
https://pythonhosted.org/bioservices/_modules/bioservices/uniprot.html
conda install bioservices --yes

in complement to O.rka answer:
Question 1:
from bioservices import UniProt
u = UniProt()
res = u.get_df("P13368 P20806 Q9UM73 P97793 Q17192".split())
This returns a dataframe with all information about each entry.
Question 2: same answer. This should scale up.
Disclaimer: I'm the author of bioservices

There is a python package in pip which does exactly what you want
pip install uniprot-mapper

XMODEM for python

I am writing a program that requires the use of XMODEM to transfer data from a sensor device. I'd like to avoid having to write my own XMODEM code, so I was wondering if anyone knew if there was a python XMODEM module available anywhere?

def xmodem_send(serial, file):
t, anim = 0, '|/-\\'
serial.setTimeout(1)
while 1:
if serial.read(1) != NAK:
t = t + 1
print anim[t%len(anim)],'\r',
if t == 60 : return False
else:
break
p = 1
s = file.read(128)
while s:
s = s + '\xFF'*(128 - len(s))
chk = 0
for c in s:
chk+=ord(c)
while 1:
serial.write(SOH)
serial.write(chr(p))
serial.write(chr(255 - p))
serial.write(s)
serial.write(chr(chk%256))
serial.flush()
answer = serial.read(1)
if answer == NAK: continue
if answer == ACK: break
return False
s = file.read(128)
p = (p + 1)%256
print '.',
serial.write(EOT)
return True

There is XMODEM module on PyPi. It handles both sending and receiving of data with XModem. Below is sample of its usage:
import serial
try:
from cStringIO import StringIO
except:
from StringIO import StringIO
from xmodem import XMODEM, NAK
from time import sleep
def readUntil(char = None):
def serialPortReader():
while True:
tmp = port.read(1)
if not tmp or (char and char == tmp):
break
yield tmp
return ''.join(serialPortReader())
def getc(size, timeout=1):
return port.read(size)
def putc(data, timeout=1):
port.write(data)
sleep(0.001) # give device time to prepare new buffer and start sending it
port = serial.Serial(port='COM5',parity=serial.PARITY_NONE,bytesize=serial.EIGHTBITS,stopbits=serial.STOPBITS_ONE,timeout=0,xonxoff=0,rtscts=0,dsrdtr=0,baudrate=115200)
port.write("command that initiates xmodem send from device\r\n")
sleep(0.02) # give device time to handle command and start sending response
readUntil(NAK)
buffer = StringIO()
XMODEM(getc, putc).recv(buffer, crc_mode = 0, quiet = 1)
contents = buffer.getvalue()
buffer.close()
readUntil()

I think you’re stuck with rolling your own.
You might be able to use sz, which implements X/Y/ZMODEM. You could call out to the binary, or port the necessary code to Python.

Here is a link to XMODEM documentation that will be useful if you have to write your own. It has detailed description of the original XMODEM, XMODEM-CRC and XMODEM-1K.
You might also find this c-code of interest.

You can try using SWIG to create Python bindings for the C libraries linked above (or any other C/C++ libraries you find online). That will allow you to use the same C API directly from Python.
The actual implementation will of course still be in C/C++, since SWIG merely creates bindings to the functions of interest.

There is a python module that you can use -> https://pypi.python.org/pypi/xmodem
You can see the transfer protocol in http://pythonhosted.org//xmodem/xmodem.html

Using DPAPI with Python?

Is there a way to use the DPAPI (Data Protection Application Programming Interface) on Windows XP with Python?
I would prefer to use an existing module if there is one that can do it. Unfortunately I haven't been able to find a way with Google or Stack Overflow.
EDIT: I've taken the example code pointed to by "dF" and tweaked it into a standalone library which can be simply used at a high level to crypt and decrypt using DPAPI in user mode. Simply call dpapi.cryptData(text_to_encrypt) which returns an encrypted string, or the reverse decryptData(encrypted_data_string), which returns the plain text. Here's the library:
# DPAPI access library
# This file uses code originally created by Crusher Joe:
# http://article.gmane.org/gmane.comp.python.ctypes/420
#
from ctypes import *
from ctypes.wintypes import DWORD
LocalFree = windll.kernel32.LocalFree
memcpy = cdll.msvcrt.memcpy
CryptProtectData = windll.crypt32.CryptProtectData
CryptUnprotectData = windll.crypt32.CryptUnprotectData
CRYPTPROTECT_UI_FORBIDDEN = 0x01
extraEntropy = "cl;ad13 \0al;323kjd #(adl;k$#ajsd"
class DATA_BLOB(Structure):
_fields_ = [("cbData", DWORD), ("pbData", POINTER(c_char))]
def getData(blobOut):
cbData = int(blobOut.cbData)
pbData = blobOut.pbData
buffer = c_buffer(cbData)
memcpy(buffer, pbData, cbData)
LocalFree(pbData);
return buffer.raw
def Win32CryptProtectData(plainText, entropy):
bufferIn = c_buffer(plainText, len(plainText))
blobIn = DATA_BLOB(len(plainText), bufferIn)
bufferEntropy = c_buffer(entropy, len(entropy))
blobEntropy = DATA_BLOB(len(entropy), bufferEntropy)
blobOut = DATA_BLOB()
if CryptProtectData(byref(blobIn), u"python_data", byref(blobEntropy),
None, None, CRYPTPROTECT_UI_FORBIDDEN, byref(blobOut)):
return getData(blobOut)
else:
return ""
def Win32CryptUnprotectData(cipherText, entropy):
bufferIn = c_buffer(cipherText, len(cipherText))
blobIn = DATA_BLOB(len(cipherText), bufferIn)
bufferEntropy = c_buffer(entropy, len(entropy))
blobEntropy = DATA_BLOB(len(entropy), bufferEntropy)
blobOut = DATA_BLOB()
if CryptUnprotectData(byref(blobIn), None, byref(blobEntropy), None, None,
CRYPTPROTECT_UI_FORBIDDEN, byref(blobOut)):
return getData(blobOut)
else:
return ""
def cryptData(text):
return Win32CryptProtectData(text, extraEntropy)
def decryptData(cipher_text):
return Win32CryptUnprotectData(cipher_text, extraEntropy)

I have been using CryptProtectData and CryptUnprotectData through ctypes, with the code from
http://article.gmane.org/gmane.comp.python.ctypes/420
and it has been working well.

Also, pywin32 implements CryptProtectData and CryptUnprotectData in the win32crypt module.

The easiest way would be to use Iron Python.