Develop Reference

Display list sorted alphabetically

We're trying to create a function that takes the input, some data containing the following information: ID number, Name, as well as a number of columns containing the grades for different assignments, and then sorts the data alphabetically (according to the name) and then displays the data with a column added that also displays the final grade (that we calculate with another function we made). We've tried writing the following code, but can't get it to work... The error-message given is "names = GRADESdata[:,1].tolist() TypeError: string indices must be integers".
Can anyone help us to figure out how to get it working?
def listOfgrades(GRADESdata):
names = GRADESdata[:,1].tolist()
names = names.sort(names)
assignments = GRADESdata[:,2::]
final_grades = computeFinalGrades(GRADESdata)
final_grades = np.array(final_grades.reshape(len(final_grades),1))
List_of_grades = np.hstack((GRADESdata, final_grades))
NOofColumns = np.size(GRADESdata,axis = 1)
display = np.zeros(NOofColumns)
for i in names:
display = np.vstack((display,GRADESdata[GRADESdata[:,1] == i]))
grades = display[1::,2:-1]
gradesfinal = display[1::,-1]
#Column titles
c = {"Student ID": GRADESdata[1::,0], "Name": GRADESdata[1::,1]}
for i in range(GRADESdata.shape[1]):
c["Assign.{}".format(i+1)] = GRADESdata[:,i]
c["Final grade"] = final_grades
d = pd.DataFrame(c)
print(d.to_string())
display = np.array([student_list, names, assignments, final_grades])
return display
The expected output is something like this (with the data below ofc):
ID number Name Assignment 1 Assignment 2 Final Grade
EDIT: the data input is a .csv file containing the following data:ID number,Name,Assignment 1,Assignment 2, etc.

The comma in
names = GRADESdata[:,1].tolist()
is not a valid character. the part between [: and ] must be an integer

From looking at .tolist(), I assume the data structure you're supposed to use is numpy.ndarray.
I managed to replicate the error with the following code:
print("12354"[:,1].tolist())
which makes sense if you're using a file name as input - and that's your mistake.
In order to fix this problem, you need to implement a string parser at the beginning or outside the function.
Add the following to your code at the beginning:
file=open(GRADESdata,"r")
data=file.read()
file.close()
list1=data.split("\n")#Replace \n with appropriate line separator
list2=[e.split(",") for e in list1]
GRADESdata=numpy.array(list2)

ArcMap Field Calculator Program to create Unique ID's

I'm using the Field Calculator in ArcMap and
I need to create a unique ID for every storm drain in my county.
An ID Should look something like this: 16-I-003
The first number is the municipal number which is in the column/field titled "Munic"
The letter is using the letter in the column/field titled "Point"
The last number is simply just 1 to however many drains there are in a municipality.
So far I have:
rec=0
def autoIncrement()
pStart=1
pInterval=1
if(rec==0):
rec=pStart
else:
rec=rec+pInterval
return "16-I-" '{0:03}'.format(rec)
So you can see that I have manually been typing in the municipal number, the letter, and the hyphens. But I would like to use the fields: Munic and Point so I don't have to manually type them in each time it changes.
I'm a beginner when it comes to python and ArcMap, so please dumb things down a little.

I'm not familiar with the ArcMap, so can't directly help you, but you might just change your function to a generator as such:
def StormDrainIDGenerator():
rec = 0
while (rec < 99):
rec += 1
yield "16-I-" '{0:03}'.format(rec)
If you are ok with that, then parameterize the generator to accept the Munic and Point values and use them in your formatting string. You probably should also parameterize the ending value as well.
Use of a generator will allow you to drop it into any later expression that accepts an iterable, so you could create a list of such simply by saying list(StormDrainIDGenerator()).
Is your question on how to get Munic and Point values into the string ID? using .format()?

I think you can use following code to do that.
def autoIncrement(a,b):
global rec
pStart=1
pInterval=1
if(rec==0):
rec=pStart
else:
rec=rec+pInterval
r = "{1}-{2}-{0:03}".format(a,b,rec)
return r
and call
autoIncrement( !Munic! , !Point! )
The r = "{1}-{2}-{0:03}".format(a,b,rec) just replaces the {}s with values of variables a,b which are actually the values of Munic and Point passed to the function.

listproperty GAE to string method

So my problem below is that when it sends the json object lets say I have multiple items in the lists? well it sends that many objects except changing that field once in the dict that i created above. For some reason , even though I use the for loops above to create a string from the list it doesn't actually make it so and i get 15 objects instead of 1 object with maybe a field that has 15 things in it separated by commas.
my expected output should be
name,special1,special2,special3..etc, review1,review2
instead i get
name,special1,review1
name,special2,review1
name,special3,review1
etc..
name,special1,review2
name,special1,review3
My main question is how do I convert the list to an array or even just to a string(like a toString Method) so that my json dump doesn't spit out multiples?
class store(db.Model):
mykey = db.StringProperty(db.Key)
storeSpecial = db.ListProperty(item_type=str)
reviews = db.ListProperty(item_type=str)
storeName = db.StringProperty()
#later in the code
qy1 = GqlQuery("SELECT storeName,storeSpecial,reviews FROM store WHERE mykey =:1",dataInput)
records_to_send = []
for i in qy1:
rev =""
for k in i.review:
rev = rev + str(k) + ","
spec = ""
for k2 in i.storeSpecial:
spec = spec + str(k2) + ","
output = {"store name": i.storeName,"specials": spec,"reviews":rev}
records_to_send.append(output)
self.response.out.write(json.dumps(records_to_send))

I'm not really sure what you want, your example is confusing. If you use JSON, why not just keep it as list? Why do you change it to a string?
It seems you want something like this
for store in qy1:
records_to_send.append({'store name': store.storeName, 'specials': store.storeSpecial}, 'reviews': store.reviews})

How to filter with "contains"?

I try to filter and get some set of objects using this segment.
baseSet = ThreadedComment.objects.filter(tree_path__contains = baseT.comment_ptr_id)
but it brings some objects that are not supposed to be there.
For example, my baseT.comment_ptr_id is 1, it brought items with these tree_path.
comment_ptr_id=1 treepath = 0000000001
comment_ptr_id=3 treepath = 0000000001/0000000003
comment_ptr_id=4 treepath = 0000000001/0000000003/0000000004
comment_ptr_id=8 treepath = 0000000001/0000000003/0000000004/0000000008
comment_ptr_id=10 treepath = 0000000006/0000000010
comment_ptr_id=11 treepath = 0000000011
The last 2 ones are not supposed to be here. But since their tree_path contains "1"
filter brings those as well.
How can I write regex to create a filter that does not bring those items?

Why not do
baseSet = ThreadedComment.objects.filter(tree_path__contains = ('%010i' % int(baseT.comment_ptr_id)))
so that the search string for id=1 will be "0000000001" and won't be a substring of "0000000011"?
EDIT: As per the comment below, it might be better to use COMMENT_PATH_DIGITS. This is a little messier because you're using formatting to set a formatting tag. It looks like this:
tree_path__contains = ('%%0%ii' % COMMENT_PATH_DIGITS % int(baseT.comment_ptr_id))

the regexp would be '(^|/)0*%d(/|$)' % baseT.comment_ptr_id and you use it with tree_path__regex
read about MPTT for alternatives to this approach.

How to make unique short URL with Python?

How can I make unique URL in Python a la http://imgur.com/gM19g or http://tumblr.com/xzh3bi25y
When using uuid from python I get a very large one. I want something shorter for URLs.

Edit: Here, I wrote a module for you. Use it. http://code.activestate.com/recipes/576918/
Counting up from 1 will guarantee short, unique URLS. /1, /2, /3 ... etc.
Adding uppercase and lowercase letters to your alphabet will give URLs like those in your question. And you're just counting in base-62 instead of base-10.
Now the only problem is that the URLs come consecutively. To fix that, read my answer to this question here:
Map incrementing integer range to six-digit base 26 max, but unpredictably
Basically the approach is to simply swap bits around in the incrementing value to give the appearance of randomness while maintaining determinism and guaranteeing that you don't have any collisions.

I'm not sure most URL shorteners use a random string. My impression is they write the URL to a database, then use the integer ID of the new record as the short URL, encoded base 36 or 62 (letters+digits).
Python code to convert an int to a string in arbitrary bases is here.

Python's short_url is awesome.
Here is an example:
import short_url
id = 20 # your object id
domain = 'mytiny.domain'
shortened_url = "http://{}/{}".format(
domain,
short_url.encode_url(id)
)
And to decode the code:
decoded_id = short_url.decode_url(param)
That's it :)
Hope this will help.

Hashids is an awesome tool for this.
Edit:
Here's how to use Hashids to generate a unique short URL with Python:
from hashids import Hashids
pk = 123 # Your object's id
domain = 'imgur.com' # Your domain
hashids = Hashids(salt='this is my salt', min_length=6)
link_id = hashids.encode(pk)
url = 'http://{domain}/{link_id}'.format(domain=domain, link_id=link_id)

This module will do what you want, guaranteeing that the string is globally unique (it is a UUID):
http://pypi.python.org/pypi/shortuuid/0.1
If you need something shorter, you should be able to truncate it to the desired length and still get something that will reasonably probably avoid clashes.

This answer comes pretty late but I stumbled upon this question when I was planning to create an URL shortener project. Now that I have implemented a fully functional URL shortener(source code at amitt001/pygmy) I am adding an answer here for others.
The basic principle behind any URL shortener is to get an int from long URL then use base62(base32, etc) encoding to convert this int to a more readable short URL.
How is this int generated?
Most of the URL shortener uses some auto-incrementing datastore to add URL to datastore and use the autoincrement id to get base62 encoding of int.
The sample base62 encoding from string program:
# Base-62 hash
import string
import time
_BASE = 62
class HashDigest:
"""Base base 62 hash library."""
def __init__(self):
self.base = string.ascii_letters + string.digits
self.short_str = ''
def encode(self, j):
"""Returns the repeated div mod of the number.
:param j: int
:return: list
"""
if j == 0:
return [j]
r = []
dividend = j
while dividend > 0:
dividend, remainder = divmod(dividend, _BASE)
r.append(remainder)
r = list(reversed(r))
return r
def shorten(self, i):
"""
:param i:
:return: str
"""
self.short_str = ""
encoded_list = self.encode(i)
for val in encoded_list:
self.short_str += self.base[val]
return self.short_str
This is just the partial code showing base62 encoding. Check out the complete base62 encoding/decoding code at core/hashdigest.py
All the link in this answer are shortened from the project I created

The reason UUIDs are long is because they contain lots of information so that they can be guaranteed to be globally unique.
If you want something shorter, then you'll need to do something like generate a random string, checking whether it is in the universe of already generated strings, and repeating until you get an unused string. You'll also need to watch out for concurrency here (what if the same string gets generated by a separate process before you inserted into the set of strings?).
If you need some help generating random strings in Python, this other question might help.

It doesn't really matter that this is Python, but you just need a hash function that maps to the length you want. For example, maybe use MD5 and then take just the first n characters. You'll have to watch out for collisions in that case, though, so you might want to pick something a little more robust in terms of collision detection (like using primes to cycle through the space of hash strings).

I don't know if you can use this, but we generate content objects in Zope that get unique numeric ids based on current time strings, in millis (eg, 1254298969501)
Maybe you can guess the rest. Using the recipe described here:
How to convert an integer to the shortest url-safe string in Python?, we encode and decode the real id on the fly, with no need for storage. A 13-digit integer is reduced to 7 alphanumeric chars in base 62, for example.
To complete the implementation, we registered a short (xxx.yy) domain name, that decodes and does a 301 redirect for "not found" URLs,
If I was starting over, I would subtract the "starting-over" time (in millis) from the numeric id prior to encoding, then re-add it when decoding. Or else when generating the objects. Whatever. That would be way shorter..

You can generate a N random string:
import string
import random
def short_random_string(N:int) -> str:
return ''.join(random.SystemRandom().choice(
string.ascii_letters + \
string.digits) for _ in range(N)
)
so,
print (short_random_string(10) )
#'G1ZRbouk2U'
all lowercase
print (short_random_string(10).lower() )
#'pljh6kp328'

Try this http://code.google.com/p/tiny4py/ ... It's still under development, but very useful!!

My Goal: Generate a unique identifier of a specified fixed length consisting of the characters 0-9 and a-z. For example:
zcgst5od
9x2zgn0l
qa44sp0z
61vv1nl5
umpprkbt
ylg4lmcy
dec0lu1t
38mhd8i5
rx00yf0e
kc2qdc07
Here's my solution. (Adapted from this answer by kmkaplan.)
import random
class IDGenerator(object):
ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyz"
def __init__(self, length=8):
self._alphabet_length = len(self.ALPHABET)
self._id_length = length
def _encode_int(self, n):
# Adapted from:
# Source: https://stackoverflow.com/a/561809/1497596
# Author: https://stackoverflow.com/users/50902/kmkaplan
encoded = ''
while n > 0:
n, r = divmod(n, self._alphabet_length)
encoded = self.ALPHABET[r] + encoded
return encoded
def generate_id(self):
"""Generate an ID without leading zeros.
For example, for an ID that is eight characters in length, the
returned values will range from '10000000' to 'zzzzzzzz'.
"""
start = self._alphabet_length**(self._id_length - 1)
end = self._alphabet_length**self._id_length - 1
return self._encode_int(random.randint(start, end))
if __name__ == "__main__":
# Sample usage: Generate ten IDs each eight characters in length.
idgen = IDGenerator(8)
for i in range(10):
print idgen.generate_id()