I'm trying to conditionally change field value during lookup - I have some specific order in mind and I do not want to overwrite field value, just to sort it my way. Let's say, I have classProduct and every class object has product_code field. Now I want to get less than or equal, but it's not trivial - product_code is for most of the time like this A01, B02 and so on and Django lookup lte would work. But now I have fields 0001C01 which I would like to be the biggest value. So during lookup I would like to add 0000 at the begining of every string that does not have this prefix, so it would look like 0001C01, 0000B02, 0000A01.
You can conditionally annotate your queryset in order to get a new field that has de desired value and then use this field in your filter or order_by clause. For example you could do the following:
from django.db.models import CharField, Value as V, F, Q, Case, When
from django.db.models.functions import Concat
Product.objects.annotate(
new_product_code=Case(
When(product_code__iregex=r'^[A-Z]+.*', # If it starts with letters
then=Concat(V('0000'), 'product_code', output_field=CharField()) # Then prepend four 0's
),
default=F('product_code') # Else, the original value
)
).filter(new_product_code__lte='whatever you like') # Now filter by using your new value
Relevant parts of the documentation are conditional expressions, database functions and QuerySet API reference
This sounds fairly straightforward. Fetch the desired Product objects, and for each one, prepend 0000 to product_code if it doesn't start with that string.
products = Product.objects.filter(some_query_expression)
for product in products:
if not product.product_code.startswith('0000'):
product.product_code = '0000' + product.product_code
It's not clear if you want to save this value back to the database, or just use it for temporary comparisons. If you do want to save it, call product.save().
I have a queryset of Products with JSONField attributes containing dict
class Product(models.Model):
attributes = JSONField(
pgettext_lazy('Product field', 'attributes'),
encoder=DjangoJSONEncoder, default={})
I want to filter Products where attributes['12'] == '31'
Following one works:
qs.filter(attributes__contains={'12': '31'})
Following one does not:
qs.filter(attributes__12='31')
Is this something I can achieve with PostgreSQL or should I move it to ES?
EDIT:
Unfortunately I cannot use first solution, as this dict may contain more keys.
First solution works well. Given we have:
product.attributes = {'333': ['6', '1']}
We can filter it out by:
Product.objects.filter(attributes__contains={'333': ['6']}
etc. Totally overlooked it.
You should be able to use the second format, i.e. qs.filter(attributes__key='value').
Your issue in this case, as explained in the docs, is that when using an integer as a key in a JSON query, that key will be used as the index of an array, thus it's interpreted as attributes[12] instead of attributes['12'].
As long as you stick to string keys, you should be fine.
An example:
class MyModel(models.Model)
json = JSONField(default=dict)
p = MyModel.objects.create(json={'0': 'something', 'a': 'something else'})
MyModel.objects.filter(json__0='something') # returns empty queryset
MyModel.objects.filter(json__a='something else') # returns the object created above
I would like to use the in_ operator in sqlalchemy using two values, one of them being NULL (mysql NULL), I don't know how to pass it via Python?
So I have a Python cgi that contains a bunch of parameters that I format then finally store inside a dict queryValues (the key being the column name and the value being a value sent by the user stored inside a fieldStorage)
for attr,value in queryValues.items() : #queryValues is a dict of parameters
valueWithNone = value.append(None) #I want to includ NULL
and_args_of_cr = [(and_(getattr(TableCR.c,attr).in_(valueWithNone)))]
I tried None and sqlalchemy.sql.null(), also tried putting directly in_(value,None) but value has the form ['Yes'] so I don't know how to do this.
But it's not working, how can I do this please?
The line value.append(None) is an in-place modification and does not return anything, so valueWithNone will be None. This is probably what you're after:
for attr,value in queryValues.items():
queryvalues = value[:] # Create a copy of list
queryvalues.append(None)
and_args_of_cr = [(and_(getattr(TableCR.c,attr).in_(queryvalues)))]
I have the following code:
query = """
SELECT Coalesce((SELECT sp.param_value
FROM sites_params sp
WHERE sp.param_name = 'ci'
AND sp.site_id = s.id
ORDER BY sp.id DESC
LIMIT 1), -1) AS ci
FROM sites s
WHERE s.deleted = 0
AND s.id = 10
"""
site = db_session.execute(query)
# print site
# <sqlalchemy.engine.result.ResultProxy object at 0x033E63D0>
site = db_session.execute(query).fetchone()
print site # (u'375')
print list(site) # [u'375']
Why does SQLAlchemy return tuples, not dicts, for this query? I want to use the following style to access the results of the query:
print site.ci
# u'375'
This is an old question, but still relevant today. Getting SQL Alchemy to return a dictionary is very useful, especially when working with RESTful based APIs that return JSON.
Here is how I did it using the db_session in Python 3:
resultproxy = db_session.execute(query)
d, a = {}, []
for rowproxy in resultproxy:
# rowproxy.items() returns an array like [(key0, value0), (key1, value1)]
for column, value in rowproxy.items():
# build up the dictionary
d = {**d, **{column: value}}
a.append(d)
The end result is that the array a now contains your query results in dictionary format.
As for how this works in SQL Alchemy:
Thedb_session.execute(query) returns a ResultProxy object
The ResultProxy object is made up of RowProxy objects
The RowProxy object has an .items() method that returns key, value tuples of all the items in the row, which can be unpacked as key, value in a for operation.
And here a one-liner alternative:
[{column: value for column, value in rowproxy.items()} for rowproxy in resultproxy]
From the docs:
class sqlalchemy.engine.RowProxy(parent, row, processors, keymap)
Proxy values from a single cursor row.
Mostly follows “ordered dictionary” behavior, mapping result values to the string-based column name, the integer position of the result in the row, as well as Column instances which can be mapped to the original Columns that produced this result set (for results that correspond to constructed SQL expressions).
has_key(key)
Return True if this RowProxy contains the given key.
items()
Return a list of tuples, each tuple containing a key/value pair.
keys()
Return the list of keys as strings represented by this RowProxy.
Link: http://docs.sqlalchemy.org/en/latest/core/connections.html#sqlalchemy.engine.RowProxy.items
Did you take a look at the ResultProxy docs?
It describes exactly what #Gryphius and #Syed Habib M suggest, namely to use site['ci'].
The ResultProxy does not "return a tuple" as you claim - it is (not surprisingly) a proxy that behaves (e.g. prints) like a tuple but also supports dictionary-like access:
From the docs:
Individual columns may be accessed by their integer position,
case-insensitive column name, or by schema.Column object. e.g.:
row = fetchone()
col1 = row[0] # access via integer position
col2 = row['col2'] # access via name
col3 = row[mytable.c.mycol] # access via Column object.
I've built a simple class to work like a database interface in our processes. Here it goes:
from sqlalchemy import create_engine
class DBConnection:
def __init__(self, db_instance):
self.db_engine = create_engine('your_database_uri_string')
self.db_engine.connect()
def read(self, statement):
"""Executes a read query and returns a list of dicts, whose keys are column names."""
data = self.db_engine.execute(statement).fetchall()
results = []
if len(data)==0:
return results
# results from sqlalchemy are returned as a list of tuples; this procedure converts it into a list of dicts
for row_number, row in enumerate(data):
results.append({})
for column_number, value in enumerate(row):
results[row_number][row.keys()[column_number]] = value
return results
You can easily convert each result row to a dictionary by using dict(site).
Then site['ci'] would be available if ci column is exists.
In order to have site.ci (according to https://stackoverflow.com/a/22084672/487460):
from collections import namedtuple
Site = namedtuple('Site', site.keys())
record = Site(*site)
This may help solve the OPs question. I think the problem he was having is that the row object only contained column values, but not the column names themselves, as is the case with ORM queries where the results have a dict attribute with both keys and values.
python sqlalchemy get column names dynamically?
The easiest way that I found is using list comprehension with calling dict() func on every RowProxy:
site = db_session.execute(query)
result = [dict(row) for row in site]
Based on Essential SQLAlchemy book:
A ResultProxy is a wrapper around a DBAPI cursor object, and its main
goal is to make it easier to use and manipulate the results of a
statement
Simple select example:
from sqlalchemy.sql import select
stmnt = select([cookies])
result_proxy = connection.execute(stmnt)
results = result_proxy.fetchall()
Results going to be like this:
# ID, cookie_name, quantity, amount
[
(1, u'chocolate chip', 12, Decimal('0.50')),
(2, u'dark chocolate chip', 1, Decimal('0.75')),
(3, u'peanut butter', 24, Decimal('0.25')),
(4, u'oatmeal raisin', 100, Decimal('1.00'))
]
It makes handling query results easier by allowing access using an index, name, or Column object.
Accessing cookie_name in different ways:
first_row = results[0]
first_row[1]
first_row.cookie_name
first_row[cookies.c.cookie_name]
These all result in u'chocolate chip' and they each reference the exact same data element in the first record of our results variable. This flexibility in access is only part of the power of the ResultProxy.
We can also leverage the ResultProxy as an iterable:
result_proxy = connection.execute(stmnt)
for record in result_proxy:
print(record.cookie_name)
This method uses list comprehensions, it receives a sql alchemy rowset object and returns the same items as a list of dictionaries:
class ResultHelper():
#classmethod
def resultproxy_to_list(cls, sql_alchemy_rowset):
return [{tuple[0]: tuple[1] for tuple in rowproxy.items()}
for rowproxy in sql_alchemy_rowset]
As you call db.execute(sql).fetchall(), you can easily use the following function to parse the return data to a dict:
def query_to_dict(ret):
if ret is not None:
return [{key: value for key, value in row.items()} for row in ret if row is not None]
else:
return [{}]
I have a list of tuples of foreign keys of the form [(3,2),(2,3)]. And I want to insert the items into a ManyToMany table within a model:
class Place(models.Model):
data=models.IntegerField()
connected_to=models.ManyToManyField('self')
class PlaceMeta(models.Model):
place=models.ForeignKey("places.Place")
and I am inserting the list (connections) with:
places=Place.objects.all()
for conn_1, conn_2 in connections:
for place in places:
if place.data == conn_1 and conn_1 != conn_2:
place.connected_to.add(conn_1, conn_2)
elif place.data == conn_2 and conn_1 != conn_2:
fixture.connected_to.add(conn_2, conn_1)
When I print the list it prints [(3L, 2L),(2L, 3L)] (for example) but after I insert the table shows that (2,2),(3,2),(2,3),and (3,3) have been inserted.
I've tried at multiple points in the code to check for if a tuple (a,a) exists and when I print prior to inserting it shows no such tuple. So how do I avoid inserting such tuples seeing as they don't even appear to exist in the list before I insert?
You have one parameter too much in you call to add(). It should look like this:
if place.data == conn_1 and conn_1 != conn_2:
# place is the Place instance described by conn_1.
# Let's connect it to conn_2!
place.connected_to.add(conn_2)
And you don't need to iterate through all the Places, instead use objects.get or objects.filter depending if data is unique or not. For example, if it is unique, use:
for source, target in connections:
Place.objects.get(data=source).connected_to.add(Place.objects.get(data=target))
(and probably add the unique=True attribute to the data field)