I am having problems understanding how to work with query results. I asked about half a dozen questions about this but I still do not understand. I copy from previous code and I make it work somehow but since I don't understand the underlying concept the code breaks down if I make a minor change. I would really appreciate if you could tell me how you visualize what is happenning here and explain it to me. Thank you.
class ReceiveEmail(InboundMailHandler):
def receive(self, message):
logging.info("Received email from %s" % message.sender)
plaintext = message.bodies(content_type='text/plain')
for text in plaintext:
txtmsg = ""
txtmsg = text[1].decode()
logging.info("Body is %s" % txtmsg)
logging.info("CC email is %s" % ((message.cc).split(",")[1]))
query = User.all()
query.filter("userEmail =", ((message.cc).split(",")[1]))
results = query.fetch(1)
for result in results:
result.userScore += 1
um = results[0]
um.userScore = result.userScore
um.put()
In this code, as I understand it, the query takes the second email address from the cc list and fetches the result.
Then I increment the userScore by 1.
Next, I want to update this item in Datastore so I say
um = results[0]
um.userScore = result.userScore
um.put()
But this gives an index out of range error:
um = results[0]
IndexError: list index out of range
Why? I am imagining that results[0] is the zeroeth item of the results. Why is it out of range? Only thing I can think of is that, the list may be None. But I don't understand why. It must have the 1 item that was fetched.
Also, if I try to test for the first email address by changing the index from [1] to [0]
query.filter("userEmail =", ((message.cc).split(",")[0]))
then I don't get the IndexError.
What am I doing wrong here?
Thanks!
EDIT
See comments:
(message.cc).split(",")[0])
left a space in front of the emails (starting with the second email), so the query was not matching them;
>>> cc.split(",")
['cc12#example.com', ' cc13#example.com', ' cc13#example.com']
adding a space after comma fixed the problem:
>>> listcc = cc.split(", ")
>>> listcc
['cc12#example.com', 'cc13#example.com', 'cc13#example.com']
>>>
To understand the code break it down and look at it piece by piece:
class ReceiveEmail(InboundMailHandler):
def receive(self, message):
logging.info("Received email from %s" % message.sender)
# Get a list of CC addresses. This is basically a for loop.
cc_addresses = [address.strip() for address in message.cc.split(",")]
# The CC list goes with the message, not the bodies.
logging.info("CC email is %s" % (cc_addresses))
# Get and iterate over all of the *plain-text* bodies in the email.
plaintext = message.bodies(content_type='text/plain')
for text in plaintext:
txtmsg = ""
txtmsg = text[1].decode()
logging.info("Body is %s" % txtmsg)
# Setup a query object.
query = User.all()
# Filter the user objects to get only the emails in the CC list.
query.filter("userEmail IN", cc_addresses)
# But, only get at most 10 users.
users = query.fetch(10)
logging.info('Got %d user entities from the datastore.' % len(users))
# Iterate over each of the users increasing their score by one.
for user in users:
user.userScore += 1
# Now, write the users back to the datastore.
db.put(users)
logging.info('Wrote %d user entities.' % len(users))
I would make an adjustment to your model structure. When you create the User entity, I would set the key_name to the email address. You will be able to make your queries much more efficient.
Some references:
List Comprehension.
Query Object.
db.put().
Related
New to automation, I have a couple of months under my belt and learning when I have free time.
I'm trying to confirm that a user name appears inside a table and the name will appear on several rows.
I'm using something like this:
#step('Users homepage my lists created by is only user "{username}"')
def step_impl(context, username):
users_name = context.browser.find_elements_by_xpath(
"//*[#id='apollo-table_wrapper']][contains(text(),'%s')]" % username)
I know the xpath is correct for the table but if I want to verify that only a specific username is visible on the screen I'm running into an issue.
In this image below I want to have a test that makes sure only the user of "mike" is present on the page. I will call out "mike" in the feature file...
General idea of the UI
You have a typo change ] to //:
users_name = context.browser.find_elements_by_xpath(
"//*[#id='apollo-table_wrapper']//[contains(text(),'%s')]" % username)
To loop through you can do something like this:
users_names = context.browser.find_elements_by_xpath("//*[#id='apollo-table_wrapper']")
print([i.text for i in users_names if i.text == "Mike"])
# Or you can append to list:
res = []
[res.append(i.text) for i in users_names if i.text == "Mike"]
print(res)
If i understood you, you wan't to verify all the names in the grid, and detect if any of these names isn't "Mike". You can use the code below to select all the names from the grid, and then verify if any of them is different from the expected
#This will only work if the Xpath bellow selects only the names, if it
#selects other fields from the table it will verify if all the fields are
#"Mike", instead of only the Name fields.
users_name = context.browser.find_elements_by_xpath("//*[#id='apollo-table_wrapper']")
result = true
expectedValue = "Mike"
for x in users_name:
if x.text != expectedValue
result = false
break
#Assert result here
Let me know if this doesn't work.
I'm trying to find the best way to get a list of all LDAP user accounts that belong to groups which are members of a groupOfNames using python-ldap. This is on an OpenLDAP server, not AD. I wrote the function below, which does the job but takes forever to run. I'm hoping either python-ldap has some builtin function that I'm not aware of, or there's something I can modify to make this run more quickly. If not, hopefully someone else will find this code useful. Thanks in advance for any help!
def get_nested_members(con, dn):
"""
Parameters
----------
con : LDAPObject
An authenticated python-ldap connection object
dn : string
The dn of the groupOfNames to be checked
Returns
-------
members : list
A list of all accounts that are members of the given dn
"""
members = []
searched = []
to_search = [dn]
while len(to_search) > 0:
current_dn = to_search.pop()
cn = current_dn.split(',')[0]
r = con.search_s(base_dn, ldap.SCOPE_SUBTREE, cn, [])[0][1]
if 'groupOfNames' in r['objectClass']:
if 'member' in r:
for i in r['member']:
if((i != current_dn) and (i not in searched)):
to_search.append(i)
searched.append(current_dn)
elif 'posixGroup' in r['objectClass']:
if 'memberUid' in r:
for i in r['memberUid']:
members.append(i)
searched.append(current_dn)
elif 'posixAccount' in r['objectClass']:
if 'uid' in r:
members.append(r['uid'][0])
else:
print('ERROR: encountered record of unknown type:')
pprint(str([current_dn, r]))
return list(set(members))
I realized that running ldapsearch repeatedly was the limiting factor, so I made a new version which builds a dictionary of ALL group and groupOfNames records first. It takes up a bit more memory than the old solution, but is less taxing on the LDAP server and runs significantly faster (down from ~15 minutes to <1 second for my application). I'll leave the original code below the new version for a reference of what not to do. Credit for the merge_dicts() function goes to Aaron Hall.
import ldap
def merge_dicts(*dict_args):
"""Given any number of dicts, shallow copy and merge into a new dict,
precedence goes to key value pairs in latter dicts.
"""
result = {}
for dictionary in dict_args:
result.update(dictionary)
return result
def get_nested_members(con, dn, base_dn='dc=example'):
"""Search a groupOfNames and return all posixAccount members from all its subgroups
Parameters
----------
con: LDAPObject
An authenticated LDAP connection object
dn: string
The dn of the groupOfNames to be searched for members
(optional) base_dn: string
The base dn to search on. Make sure to change the default value to fit your LDAP server
Returns
-------
members: list
A list of all nested members from the provided groupOfNames
"""
logging.info('Getting nested members of ' + str(dn))
print('Getting nested members of ' + str(dn))
if type(dn) is list:
to_search = [] + dn
elif type(dn) is str:
to_search = [dn]
else:
print('ERROR: Invalid dn value. Please supply either a sting or list of strings.')
return []
members = []
searched = []
groupOfNames_list = con.search_s(base_dn, ldap.SCOPE_SUBTREE, 'objectClass=groupOfNames', ['dn', 'member', 'cn'])
groupOfNames_dict = {}
for g in range(len(groupOfNames_list)):
groupOfNames_dict[groupOfNames_list[g][0]] = groupOfNames_list[g][1]
groupOfNames_list = None #To free up memory
group_list = con.search_s(base_dn, ldap.SCOPE_SUBTREE, 'objectClass=posixGroup', ['dn', 'memberUid', 'cn'])
group_dict = {}
for g in range(len(group_list)):
group_dict[group_list[g][0]] = group_list[g][1]
group_list = None #To free up memory
all_groups = merge_dicts(groupOfNames_dict, group_dict)
group_dict = None #To free up memory
groupOfNamesdict = None #To free up memory
while len(to_search) > 0:
search_dn = to_search.pop()
try:
g = all_groups[search_dn]
if 'memberUid' in g:
members += g['memberUid']
searched.append(search_dn)
elif 'member' in g:
m = g['member']
for i in m:
if i.startswith('uid='):
members.append((i.split(',')[0]).split('=')[1])
elif i.startswith('cn='):
if i not in searched:
to_search.append(i)
searched.append(search_dn)
else:
searched.append(search_dn)
except:
searched.append(search_dn)
return list(set(members))
I'm trying to keep this as simple as possible. Basically I want the data to be saved to a file, and the retrieve it so that questor.py works and can "remember" everything it was ever taught on your machine. The original code is available on the web at http://www.strout.net/info/coding/python/questor.py
If I'm reading the code right, you end up with an object that looks something like {key:{key:{key:class instance},class instance},class instance} . (rough estimate)
Please ignore the unfished method Save, I'm working on that as soon as I figure out how to pickle the dictionary without losing any of the imbedded instances.
The following is my attempt at trying to save the dict via pickler. Some of the code is unfinished, but you should be able to get an idea of what I was trying to do. So far all I am able to do is retrieve the last question/answer set. Either my pickle isn't saving the imbedded instances, or they're not actually there when I save the pickle. I've followed the spaghetti lines as much as possible, but can't seem to figure out how to set up a way to save to file without losing anything.
Also my file doesn't have to be .txt originally I was going to use .data for the pickle.
# questor.py
# define some constants for future use
kQuestion = 'question'
kGuess = 'guess'
questfile = 'questfile.txt'
## Added
import cPickle as p
# create a file for questor
def questor_file():
try:
questor = open(questfile,'rb')
try:
q = p.Unpickler(questor)
quest = q.load()
questor.close()
return quest
except:
print 'P.load failed'
except:
print 'File did not open'
questor = open('questfile.data', 'wb')
questor.close()
return Qnode('python')
# define a function for asking yes/no questions
def yesno(prompt):
ans = raw_input(prompt)
return (ans[0]=='y' or ans[0]=='Y')
# define a node in the question tree (either question or guess)
class Qnode:
# initialization method
def __init__(self,guess):
self.nodetype = kGuess
self.desc = guess
##Added
## Not sure where I found this, but was going to attempt to use this as a retreival method
## haven't gotten this to work yet
def Load(self):
f = open(self.questfile,'rb')
tmp_dict = cPickle.load(f)
f.close()
self.__dict__.update(tmp_dict)
##Added
# was going to use this as a save method, and call it each time I added a new question/answer
def Save(self,node):
f = open(self.questfile,'wb')
quest = p.pickler(f)
# get the question to ask
def query(self):
if (self.nodetype == kQuestion):
return self.desc + " "
elif (self.nodetype == kGuess):
return "Is it a " + self.desc + "? "
else:
return "Error: invalid node type!"
# return new node, given a boolean response
def nextnode(self,answer):
return self.nodes[answer]
# turn a guess node into a question node and add new item
# give a question, the new item, and the answer for that item
def makeQuest( self, question, newitem, newanswer ):
# create new nodes for the new answer and old answer
newAnsNode = (Qnode(newitem))
oldAnsNode = (Qnode(self.desc))
# turn this node into a question node
self.nodetype = kQuestion
self.desc = question
# assign the yes and no nodes appropriately
self.nodes = {newanswer:newAnsNode, not newanswer:oldAnsNode}
self.save(self.nodes)
def traverse(fromNode):
# ask the question
yes = yesno( fromNode.query() )
# if this is a guess node, then did we get it right?
if (fromNode.nodetype == kGuess):
if (yes):
print "I'm a genius!!!"
return
# if we didn't get it right, return the node
return fromNode
# if it's a question node, then ask another question
return traverse( fromNode.nextnode(yes) )
def run():
# start with a single guess node
# This was supposed to assign the data from the file
topNode = questor_file()
done = 0
while not done:
# ask questions till we get to the end
result = traverse( topNode )
# if result is a node, we need to add a question
if (result):
item = raw_input("OK, what were you thinking of? ")
print "Enter a question that distinguishes a",
print item, "from a", result.desc + ":"
q = raw_input()
ans = yesno("What is the answer for " + item + "? ")
result.makeQuest( q, item, ans )
print "Got it."
# repeat until done
print
done = not yesno("Do another? ")
# Added
# give me the dictionary
return result
# immediate-mode commands, for drag-and-drop or execfile() execution
if __name__ == '__main__':
print "Let's play a game."
print 'Think of something, just one thing.'
print 'It can be anything, and I will try to guess what it is.'
raw_input('Press Enter when ready.')
print
questdata = run()
print
# Added
# Save the dictionary
questor = open(questfile,'wb')
q = p.Pickler(questor)
q.dump(questdata)
questor.close()
raw_input("press Return>")
else:
print "Module questor imported."
print "To run, type: questor.run()"
print "To reload after changes to the source, type: reload(questor)"
# end of questor.py
one way that comes to mind is creating a list of all the nodes and saving that ... they should keep their internal pointers on their own.
declare a list of nodes at the top of your file (and use pickle... just cause Im more familiar with that)
import pickle
kQuestion = 'question'
kGuess = 'guess'
questfile = 'questfile.txt'
nodes = []
....
change your load method to something like
def questor_file():
global nodes
try:
questor = open(questfile,'rb')
try:
nodes= pickle.load(questor)
quest = nodes[0]
questor.close()
return quest
except:
print 'P.load failed'
nodes = []
except:
print 'File did not open'
nodes = []
return Qnode('python')
change your class constructor so that it adds each node to nodes
class Qnode:
# initialization method
def __init__(self,guess):
self.nodetype = kGuess
self.desc = guess
nodes.append(self)
at the end where it says #added save dictionary , save your list of nodes
questor = open(questfile,'wb')
q = pickle.dump(nodes,questor)
make sure you exit the program by typing no when prompted ...
you could also save it to a database or whatever but you would still have to store each node and it might be more complicated... this method should really be fine I think , (although there may be a more natural way to save a tree structure) ...
I'm trying to write a program that creates an address book with contact names, emails, phone numbers, etc. I store each contact as a dictionary and then place each person (dictionary) into a global list. I then convert the list to a string using repr() and write it to a file. When I try to reload the list and write what it contains, I get a list of empty dictionaries. Please help me figure out what is wrong.
Here is my code:
list = []
listfile = 'phonebook.txt'
class bookEntry(dict):
total = 0
def __init__(self):
bookEntry.total += 1
self.d = {}
def __del__(self):
bookEntry.total -= 1
class Person(bookEntry):
def __init__(self, n):
bookEntry.__init__(self)
self.n = n
print '%s has been created' % (self.n)
def addnewperson(self, n, e = '', ph = '', note = ''):
f = file(listfile, 'w')
self.d['name'] = n
self.d['email'] = e
self.d['phone'] = ph
self.d['note'] = note
list.append(self)
listStr = repr(list)
f.write(listStr)
f.close()
I start the program with a startup() function:
def startup():
aor = raw_input('Hello! Would you like to add an entry or retrieve one?')
if aor == 'add':
info = raw_input('Would you like to add a person or a company?')
if info == 'person':
n = raw_input('Please enter this persons name:')
e = raw_input('Please enter this persons email address:')
ph = raw_input('Please enter this persons phone number:')
note = raw_input('Please add any notes if applicable:')
X = Person(n)
X.addnewperson(n, e, ph, note)
startup()
I add these answers to the prompts:
'''
Hello! Would you like to add an entry or retrieve one?add
Would you like to add a person or a company?person
Please enter this persons name:Pig
Please enter this persons email address:pig#brickhouse.com
Please enter this persons phone number:333-333-3333
Please add any notes if applicable:one of three
Pig has been created
'''
When I open phonebook.txt, this is what I see:
[{}]
Why are empty dictionaries being returned?
You're deriving from dict, but storing all the elements in a member d. Hence, repr gives you a string representing an empty dict. If you want to use a bookEntry as a dict, insert the info with
self['name'] = n
instead of
self.d['name'] = n
(But really, you shouldn't be inheriting from dict here. Also, please don't use list as an identifier, it's the name of a builtin.)
you should save self.d instead of self:
alist.append(self.d)
listStr = repr(alist)
f.write(listStr)
btw don't use list as the name of a variable, you are overwritting the keyword list
Your problem is that the X.d dictionary is not the same as the dictionary "bookEntry" is inheriting from. Therefore repr(X) is not showing X.d
A solution might be to override repr in BookEntry:
e.g.
def __repr___(self):
return repr(self.d)
I am trying to access the group name of all entries in my contact list. How can I access the details of a group given the href link?
This is the current output:
9 Albert Gandhi 2011-03-07T09:48:19.824Z
I was thrown out of college for cheating on the metaphysics exam; I looked into the soul of the boy sitting next to me.
albert.gandhi#company1.net
Member of group: http://www.google.com/m8/feeds/groups/blah.blah%40blah.com/base/4c8d4c8d8d218d21
But I would like something like this:
9 Albert Gandhi 2011-03-07T09:48:19.824Z
I was thrown out of college for cheating on the metaphysics exam; I looked into the soul of the boy sitting next to me.
albert.gandhi#company1.net
Member of group: GroupName
Below is the code I am using to list the contact feed (taken mainly from the examples provided with the API)
def PrintFeed(self, feed, ctr=0):
"""Prints out the contents of a feed to the console.
Args:
feed: A gdata.contacts.ContactsFeed instance.
ctr: [int] The number of entries in this feed previously printed. This
allows continuous entry numbers when paging through a feed.
Returns:
The number of entries printed, including those previously printed as
specified in ctr. This is for passing as an argument to ctr on
successive calls to this method.
"""
if not feed.entry:
print '\nNo entries in feed.\n'
return 0
for i, entry in enumerate(feed.entry):
print '\n%s %s %s' % (ctr+i+1, entry.title.text, entry.updated.text)
if entry.content:
print ' %s' % (entry.content.text)
for email in entry.email:
if email.primary and email.primary == 'true':
print ' %s' % (email.address)
# Show the contact groups that this contact is a member of.
for group in entry.group_membership_info:
print ' Member of group: %s' % (group.href)
# Display extended properties.
for extended_property in entry.extended_property:
if extended_property.value:
value = extended_property.value
else:
value = extended_property.GetXmlBlobString()
print ' Extended Property %s: %s' % (extended_property.name, value)
return len(feed.entry) + ctr
For reference, you can find a solution to this issue here