I am having a lot of trouble trying to iterate over a mongo collection that have other ""subdictionaries" on it.
Basically, inside the structure of my collection I have an object that represents other Strings:
For instance:
I am trying to get the stats over this collection:
This is the code I am using for now:
cursor = mycol.find(
{}, {'_id': 1, 'stats.total': 1, 'stats.additions': 1, 'stats.deletions': 1})
with open('commits.csv', 'w') as outfile:
fields = ['id', 'stats.total', 'stats.additions', 'stats.deletions']
write = csv.DictWriter(outfile, fieldnames=fields)
write.writeheader()
for commits in cursor:
id = commits['_id']
for stats in commits['stats']:
flattened_record = {
'_id': id,
'stats.total': stats['total'],
'stats.additions': stats['additions'],
'stats.deletions': stats['deletions']
}
write.writerow(flattened_record)
I keep getting the TypeError: string indices must be integers and the type of stats object seem to be an unicode.
Does anyone know how to fix it?
Thanks for your time,
I have solved this by iterating over the stats first and then checking the total, additions and deletions separately:
if 'stats' in commits:
stats = commits['stats']
for key in stats:
if key == 'deletions':
deletions = stats[key]
elif key == 'additions':
additions = stats[key]
else:
total = stats[key]
I've got an Opportunity table and an Owner table. The Opportunity table has a many to one relationship with Owner. I'm using Python, SQLAlchemy, FLask-Alchemy and SQLite on this project. I need to do an inner join on the Opportunity table against the Owner table and return a single result set (with fields from both tables).
I need to build a dictionary out of the result set so that I can easily generate Json. The following code all works... but... It took a ton of debugging to figure out how to transform the result set returned (which, in this case is a generator object) into a dictionary.
Did I make this harder than I needed to? Is there a better way to do this using SQLAlchemy? (such as using the expression language instead of the ORM)
owner = db.session.query(Owner).filter(Owner.id == owner_id).first()
if owner is None:
return None
result = {'data': []}
report_date = datetime.datetime.strptime('2016-05-01', '%Y-%m-%d')
rows = db.session.query(Opportunity)\
.filter(Opportunity.status == 'Won',
Opportunity.owner == owner,
or_(Opportunity.actual_close_date_iso_year_week == '2016/16',\
Opportunity.actual_close_date_iso_year_week == '2016/17'),
Opportunity.report_date == report_date)\
.order_by(Opportunity.account)\
.join(Owner, Owner.id == Opportunity.owner_id)\
.values(Opportunity.account,
Opportunity.report_date_iso_year_week,
Opportunity.report_date,
Owner.first_name)
for row in rows:
result_row = {}
fields = getattr(row, '_fields')
for field in fields:
result_row[field] = getattr(row, field)
result['data'].append(result_row)
I think that basically this is the way to get what you need, however I would suggest few minor changes:
You don't really need to join, as after .filter(Opportunity.owner==owner) all the rows that you get from the Opportunity table have the same owner_id.
I think it's better to define the list of needed fields once, instead of trying to get it from each row tuple.
So, the code may be like this:
required_fields = ('account','report_date_iso_year_week','report_date')
owner = db.session.query(Owner).filter(Owner.id == owner_id).first()
if owner is None:
return None
result = {'data': []}
report_date = datetime.datetime.strptime('2016-05-01', '%Y-%m-%d')
rows = db.session.query(Opportunity)\
.filter(Opportunity.status == 'Won',
Opportunity.owner == owner,
or_(Opportunity.actual_close_date_iso_year_week == '2016/16',\
Opportunity.actual_close_date_iso_year_week == '2016/17'),
Opportunity.report_date == report_date)\
.order_by(Opportunity.account)\
.values(*[getattr(Opportunity,f) for f in required_fields])
for row in rows:
result_row = {'first_name':owner.first_name}
for field in required_fields:
result_row[field] = getattr(row, field)
result['data'].append(result_row)
How do I look up the 'id' associated with the a person's 'name' when the 2 are in a dictionary?
user = 'PersonA'
id = ? #How do I retrieve the 'id' from the user_stream json variable?
json, stored in a variable named "user_stream"
[
{
'name': 'PersonA',
'id': '135963'
},
{
'name': 'PersonB',
'id': '152265'
},
]
You'll have to decode the JSON structure and loop through all the dictionaries until you find a match:
for person in json.loads(user_stream):
if person['name'] == user:
id = person['id']
break
else:
# The else branch is only ever reached if no match was found
raise ValueError('No such person')
If you need to make multiple lookups, you probably want to transform this structure to a dict to ease lookups:
name_to_id = {p['name']: p['id'] for p in json.loads(user_stream)}
then look up the id directly:
id = name_to_id.get(name) # if name is not found, id will be None
The above example assumes that names are unique, if they are not, use:
from collections import defaultdict
name_to_id = defaultdict(list)
for person in json.loads(user_stream):
name_to_id[person['name']).append(person['id'])
# lookup
ids = name_to_id.get(name, []) # list of ids, defaults to empty
This is as always a trade-off, you trade memory for speed.
Martijn Pieters's solution is correct, but if you intend to make many such look-ups it's better to load the json and iterate over it just once, and not for every look-up.
name_id = {}
for person in json.loads(user_stream):
name = person['name']
id = person['id']
name_id[name] = id
user = 'PersonA'
print name_id[user]
persons = json.loads(...)
results = filter(lambda p:p['name'] == 'avi',persons)
if results:
id = results[0]["id"]
results can be more than 1 of course..
I do a project about Timesheet in OpenERP. I have this problem:
this is x_luong table.
class x_luong(osv.osv):
_name = 'x_luong'
_description = 'Luong'
_columns = {'name': fields.many2one('x_nhanvien', 'Mã nhân viên', size=10, required='1'),
'ma_luong': fields.integer('ma luong', size=10, required='1'),
'giolam': fields.float('Giờ làm', size=100, required='1'),
'giolamthuc': fields.char('Gio lam thuc te', size=5, required='1'),
'time_in': fields.char('Gio vào', size=20),
'time_out' :fields.char('Gio về', size=20),
'state' :fields.selection([('dangnhap','Đẳng nhập.'),('rave','Ra về')]),
'test': fields.integer('Kiem tra', size=20),
'phutvao': fields.integer('Phut vao ', size=20),
'phutra': fields.integer('phut ra', size=20),
}
_defaults = {'state':'dangnhap',
}
and this some function in it:
this 2 function mean get time when the staff sign_in or sign_out the system:
def get_timein(self,cr,uid,ids,context={}):
obj = self.browse(cr,uid,ids,context=context)[0]
timein = str(datetime.now())
self.write(cr, uid, ids, {'time_in':timein }, context=context)
return 1
def get_timeout(self,cr,uid,ids,context={}):
obj = self.browse(cr,uid,ids,context=context)[0]
timeout = str(datetime.now())
self.write(cr, uid, ids, {'time_out':timeout }, context=context)
return 1
and this 2 function for button sign_in and sign_out:
def cho_dangnhap(self,cr,uid,ids,context={}):
self.pool.get('x_luong').write(cr,uid,ids,{'state':'dangnhap'})
self.get_timein(cr,uid,ids)
return 1
def cho_rave(self,cr,uid,ids,context={}):
self.pool.get('x_luong').write(cr,uid,ids,{'state':'rave'})
self.get_timeout(cr,uid,ids)
self.tinh_thoigian(cr,uid,ids)
self.insert(cr,uid,ids)
function tinh_thoigian mean cut the string time for get ... hour or min for calculation
def _thoigianlam(self,cr,uid,ids,context={}):
obj = self.browse(cr,uid,ids,context=context)[0]
hour_den = int(obj.time_in[12:13])
hour_di = int(obj.time_out[12:13])
min_den = int(obj.time_in[15:16])
min_di = int(obj.time_out[15:16])
gl = int(hour_di)-int(hour_den)
pl = min_di-min_den
thucte = str(gl)+':'+pl
self.write(cr, uid, ids, {'giolam':gl }, context=context)
self.write(cr, uid, ids, {'giolamthuc':thucte }, context=context)
return 1
and last function insert() get ma_luong(i think this same the primary key in sql) and giolam(the hour of the staff work in company), time_in, time_out and this is function insert()
def insert(self,cr,uid,ids,context={}):
obj = self.browse(cr,uid,ids,context=context)
values = {'ma_luong':obj.name.id,
'giolam':obj.giolam,
'time_in':time_in,
'time_out':time_out,
self.pool.get('x_giolam').create(cr,uid,values,context=context)
with this function i want insert data in table x_giolam because when the staff sign in or sign out the system in day ... the data of it with save in this table and a other day when they do it again it with save it again ... and last month if you want calculation about salary of them you just select ma_luong=ma_luong(of table x_luong) and this table x_giolam:
class x_giolam(osv.osv):
_name = 'x_giolam'
_description = 'Gio Lam'
_columns = {'name': fields.integer('Lọai',size=64,required="true"),
'giolam' : fields.float('Gio lam',size=64,required="True"),
'time_in': fields.char('Gio vào',size=20),
'time_out' :fields.char('Gio về',size=20),
}
and i have 3 question with my project:
1) function insert have aerror:
AttributeError: 'browse_record_list' object has no attribute 'name'
How can i fix it ??? i data of it is save in table x_giolam
2) how can i select many row of table x_giolam which of thte employee' own.. give me some example about this function
3) how i can organization field.Xml when i show rows in
Sorry for your troubles because it is so long ... but i hope every body in here can help me. Python and open Erp so difference with c++ or c#. And this my project"research and write a module timesheet with OpenErp" of me and next week is deadline.
English of me not good, i'm sory about it!!!
Thanks!!
I can help with your first question. The problem is in this code:
def insert(self,cr,uid,ids,context={}):
obj=self.browse(cr,uid,ids,context=context)
values={'ma_luong':obj.name.id,
The error message was like this:
AttributeError: 'browse_record_list' object has no attribute 'name'
If you call orm.browse() with a list of ids, you will get back a list of browse records. You then have to enumerate through the list, or get a single entry from the list to work with.
For example:
for luong in self.browse(cr,uid,ids,context=context):
print luong.name
Or:
luongs = self.browse(cr,uid,ids,context=context)
luong = luongs[0]
print luong.name
Why don't you take a look at the standard hr_attendance module and go on from there?
For your model, the name is a reserved field name, so it would be best to keep it as achar. Try that change and see if it solves your error message.
For the other two questions, I think you should try to rephrase them a little better...
The type of obj is list of records, so for browse the list of records, you must define a one element.
in your case, you can type : obj[0].giolam --> for the giolam of the first record of obj.
forgive me for my bad english
I'm trying to build a simple search engine for a small website. My initial thought is to avoid using larger packages such as Solr, Haystack, etc. because of the simplistic nature of my search needs.
My hope is that with some guidance I can make my code more pythonic, efficient, and most importantly function properly.
Intended functionality: return product results based on full or partial matches of item_number, product name, or category name (currently no implementation of category matching)
Some code:
import pymssql
import utils #My utilities
class Product(object):
def __init__(self, item_number, name, description, category, msds):
self.item_number = str(item_number).strip()
self.name = name
self.description = description
self.category = category
self.msds = str(msds).strip()
class Category(object):
def __init__(self, name, categories):
self.name = name
self.categories = categories
self.slug = utils.slugify(name)
self.products = []
categories = (
Category('Food', ('123', '12A')),
Category('Tables', ('354', '35A', '310', '31G')),
Category('Chemicals', ('845', '85A', '404', '325'))
)
products = []
conn = pymssql.connect(...)
curr = conn.cursor()
for Category in categories:
for c in Category.categories:
curr.execute('SELECT item_number, name, CAST(description as text), category, msds from tblProducts WHERE category=%s', c)
for row in curr:
product = Product(row[0], row[1], row[2], row[3], row[4])
products.append(product)
Category.products.append(product)
conn.close()
def product_search(*params):
results = []
for product in products:
for param in params:
name = str(product.name)
if (name.find(param.capitalize())) != -1:
results.append(product)
item_number = str(product.item_number)
if (item.number.find(param.upper())) != -1:
results.append(product)
print results
product_search('something')
MS SQL database with tables and fields I cannot change.
At most I will pull in about 200 products.
Some things that jump out at me. Nested for loops. Two different if statements in the product search which could result in duplicate products being added to the results.
My thought was that if I had the products in memory (the products will rarely change) I could cache them, reducing database dependence and possibly providing an efficient search.
...posting for now... will come back and add more thoughts
Edit:
The reason I have a Category object holding a list of Products is that I want to show html pages of Products organized by Category. Also, the actual category numbers may change in the future and holding a tuple seemed like simple painless solution. That and I have read-only access to the database.
The reason for a separate list of products was somewhat of a cheat. I have a page that shows all products with the ability to view MSDS (safety sheets). Also it provided one less level to traverse while searching.
Edit 2:
def product_search(*params):
results = []
lowerParams = [ param.lower() for param in params ]
for product in products:
item_number = (str(product.item_number)).lower()
name = (str(product.name)).lower()
for param in lowerParams:
if param in item_number or param in name:
results.append(product)
print results
Prepare all variables outside of the loops and use in instead of .find if you don't need the position of the substring:
def product_search(*params):
results = []
upperParams = [ param.upper() for param in params ]
for product in products:
name = str(product.name).upper()
item_number = str(product.item_number).upper()
for upperParam in upperParams:
if upperParam in name or upperParam in item_number:
results.append(product)
print results
If both the name and number matches the search parameters, the product will appear twice on the result list.
Since the products count is a small number, I recommend constructing a SELECT query like:
def search(*args):
import operator
cats = reduce(operator.add, [list(c.categories) for c in categories], [])
query = "SELECT * FROM tblProducts WHERE category IN (" + ','.join('?' * len(cats)) + ") name LIKE '%?%' or CAST(item_number AS TEXT) LIKE '%?%' ..."
curr.execute(query, cats + list(args)) # Not actual code
return list(curr)