How in SQLAlchemy ORM to make analogue of the following raw sql?
SELECT "http://example.com/page/"||table.pagename as pageUrl
Need get value from table, modify using ORM/Python (here just a string concatenation), and output in result of the SQLAlchemy query as additional field.
The SQLAlchemy string types have operator overloads that allow you to treat them like you'd treat Python strings in this case (string concatenation), but produce SQL expressions:
session.query(
Table,
("http://example.com/page/" + Table.pagename).label("pageUrl"))
You can read more about SQLAlchemy's operator paradigm here: http://docs.sqlalchemy.org/en/latest/core/tutorial.html#operators
This can make via select, but there is almost no ORM:
from sqlalchemy.sql import select, text
q = select([text('"http://example.com/page/"||pagename as pageUrl')]).select_from(Table)
session.execute(q).fetchall()
Results will a list of objects in the RowProxy class.
For me seems that the solve via session.query (the answer above) is more convenient. It is short, and there results in result class that can be easy converting to dict.
Related
PostgreSQL has functions like jsonb_to_record or jsonb_populate_record which allow you convert data stored in a JSONB column to separate columns. however, I'm using SQLAlchemy and jsonb_to_record requires a predefined structure for the output (see example below from the PostgreSQL docs)
select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar"}') as x(a int, b text, d text)
Is there a way to use these functions from SQLAlchemy?
you can use any postgres function in SQLAlchemy with func like so:
from SQLAlchemy import func
session.query(func.json_to_record(Model.jsonb_field))
Do note that set-returning functions are a bit of a pain to deal with with SQLAlchemy, but if you want you can get it to work.
The simplest possible way to get "raw" SQL for any query is just print it (actually, convert it to str).
But, this is not working for count() queries because count() is "firing" method - a method which is stated as "This results in an execution of the underlying query". Other "firing" methods include all(), first() and so on.
How to get SQL for such methods?
I'm especially interested in count() because it transforms underlying query in some way (actually, this way is described explicitly in docs, but things may vary). Other methods can alter resulting SQL as well, for example, first().
So, sometimes it is useful to get raw SQL of such queries in order to investigate how thing goes under the hood.
I read answers about "getting raw SQL" but this case is special because such methods don't return Query objects.
Note that I mean that I need a SQL of existing Query objects which are already constructed in some way.
The following example will return a count of any query object, which you should then be able to convert to a string representation:
from sqlalchemy import func
...
existing_query = session.query(Something)\
.join(OtherThing)\
.filter(OtherThing.foo = 'FOO')\
.subquery()
query = session.query(func.count(existing_query.c.bar).label('bar_count'))
print(query)
actual_count = query.as_scalar() # Executes query
Notice that you have to specify a field from the query output to count. In the example defined by existing_query.c.bar.
Look at this query:
select 'DEFAULT_STRING' as category from foo
Here, we don't really have the column category in the table, but we just use a string literal to represent this column value in all rows returned by the query. In place of a string literal, we could have any valid SQL expression above. Is there some way to have this query in the django ORM?
A handy way of doing this is to annotate with RawSQL query expression.
from django.db.models.expressions import RawSQL
qset = Foo.objects.annotate(category=RawSQL("select 'DEFAULT_CATEGORY'", ()))
Please let me know in the answers if there are better ways of doing this.
I'm struggling to convert this to an ORM filter query:
select count(*) from issues WHERE pending_notifications ? 'flooby';
pending_notifications is a JSONB field containing a simple JSON array.
I'm not sure how to structure the filter using the question mark operator
I believe a Postgres ARRAY would work like this:
query.filter(pending_notifications.any('flooby'))
But I'm using JSONB and the filter syntax is not the same.
Any suggestions
You can use the .op() method to use any verbatim operator:
query.filter(pending_notifications.op("?")("flooby"))
Given that you're using JSONB as column data type, use the has_key() method:
query.filter(pending_notifications.has_key('flooby'))
which maps to the ? operator. The method name is misleading in this context, but PostgreSQL documentation for jsonb operators describes ? thusly:
Does the string exist as a top-level key within the JSON value?
and so has_key() is somewhat aptly named.
An example:
In [21]: t = Table('t', metadata, Column('json', postgresql.JSONB))
In [28]: print(t.c.json.has_key('test'))
t.json ? :json_1
I can use raw sql in filter
from sqlalchemy import text
db.session.query(Issue).filter(text("pending_notifications ? 'flooby'")
I'm having a problem with this sqlalchemy query:
def bvalue(value):
if isinstance(value, unicode):
value = re.sub('[^\w]', "", value).lower()
return value
basicValue = bvalue(someVariable)
q = self.session.query(sheet.id).\
filter(bvalue(sheet.column) == basicValue)
The bvalue function works. I'm trying to match values after stripping them from any special characters and capitalisation. The stripped variable does match with the stripped db value, but still the query is not retrieving any results.
What am I doing wrong? Can't you use custom methods in filters?
You are aware that SQLAlchemy translates your queries into plain SQL statements that are then emitted to your configured database?
So naturally you can't simply add arbitrary python functions, since they would have to be translated into SQL which can't be done in a generic way.
Aside from this general issue, bvalue(sheet.column) will simply return sheet.column (since it's not a unicode instance) and it is evaluated before creating the query. So your query is in fact equivalent to:
q = self.session.query(sheet.id).\
filter(sheet.column == basicValue)
How to get the regex into SQL depends on the database you're using. Check e.g.
REGEXP_LIKE in SQLAlchemy
for a some suggestions.