Can I implement the following in SQLAlchemy,
SELECT *
FROM TABLE
WHERE RIGHT(COLUMN_CODE, 2) = 'AX'
here RIGHT( ) returns the right part of a character string with the specified number of characters.
Is there a SQLAlchemy implementation of the RIGHT function?
You'd be better off using the .endswith() method instead:
select([tabledef]).where(tabledef.c.column_code.endswith('AX'))
or, when filtering with a mapper config and a session:
session.query(mappedobject).filter(mappedobject.column_code.endswith('AX'))
The column_code.endswith() method will be translated to whatever SQL is best for your particular engine, matching column values that end with AX.
You can always use the function generator to create arbitrary SQL functions if you have to use the RIGHT() sql function directly:
from sqlalchemy.sql.expression import func
select([tabledef]).where(func.right(tabledef.c.column_code, 2) == 'AX')
and the func.right() call will be translated to RIGHT(column_code, 2) by the SQL generation layer.
The documentation does not make it clear, but you can write any function using func.funcname sytle. funcname does not have to be defined natively by SQLAlchemy module. SQLAlchemy knows about common functions like min, max etc. and if there is dialect to dialect variation amongst those functions, SQLAlchemy takes care of that for you.
But the functions that SQLAlchemy does not know about are passed as is. So you can create your query that generates a SQL statement with the required RIGHT function like so
>>> from sqlalchemy import func
>>> select([table]).where(func.RIGHT(users.c.column_code, 2)='AX')
Related
I'm learning how to use SQL Alchemy, and I'm trying to re-implement a previously defined API but now using Python.
The REST API has the following query parameter:
myService/v1/data?range=time:2015-08-01:2015-08-02
So I want to map something like field:FROM:TO to filter a range of results, like a date range, for example.
This is what I'm using at this moment:
rangeStatement = range.split(':')
if(len(rangeStatement)==3):
query = query.filter(text('{} BETWEEN "{}" AND "{}"'.format(*rangeStatement)))
So, this will produce the following WHERE condition:
WHERE time BETWEEN "2015-08-01" AND "2015-08-02"
I know SQL Alchemy is a powerful tool that allows creating queries like Query.filter_by(MyClass.temp), but I need the API request to be as open as possible.
So, I'm worried that someone could pass something like DROP TABLE in the range parameter and exploit the text function
If queries are constructed using string formatting then sqlalchemy.text will not prevent SQL injection - the "injection" will already be present in the query text. However it's not difficult to build queries dynamically, in this case by using getattr to get a reference to the column. Assuming that you are using the ORM layer with model class Foo and table foos you can do
import sqlalchemy as sa
...
col, lower, upper = 'time:2015-08-01:2015-08-02'.split(':')
# Regardless of style, queries implement a fluent interface,
# so they can be built iteratively
# Classic/1.x style
q1 = session.query(Foo)
q1 = q1.filter(getattr(Foo, col).between(lower, upper))
print(q1)
or
# 2.0 style (available in v1.4+)
q2 = sa.select(Foo)
q2 = q2.where(getattr(Foo, col).between(lower, upper))
print(q2)
The respective outputs are (parameters will be bound at execution time):
SELECT foos.id AS foos_id, foos.time AS foos_time
FROM foos
WHERE foos.time BETWEEN ? AND ?
and
SELECT foos.id, foos.time
FROM foos
WHERE foos.time BETWEEN :time_1 AND :time_2
SQLAlchemy will delegate quoting of the values to the connector package being used by the engine, so you're protection against injection will be as good as that provided by the connector package*.
* In general I believe correct quoting should be a good defence against SQL injections, however I'm not sufficiently expert to confidently state that it's 100% effective. It will be more effective than building queries from strings though.
I want to pass in parameters to a sql query when using GeoPandas from_postgis functionality with SQLAlchemy.
classmethod GeoDataFrame.from_postgis(sql, con, geom_col='geom', crs=None, index_col=None, coerce_float=True, params=None)
I have looked at a previous similar question and also here which suggests to use SQLAlchemy's textual SQL option. However, this requires access to con.execute which isn't included in the GeoDataFrame from_postgis option.
Could you suggest the best way to pass the parameters to SQL? If not directly in from_postgis, then how best to construct the full SQL string separately and passing it in as the first sql argument to from_postgis.
For textual SQL, you can add parameters by using .bindparams:
query = text("select * from foo where bar = :a").bindparams(a=1)
For queries you construct in SQLAlchemy, bind parameters are automatically included:
foo = Table(...) # you need to define foo
query = select(["*"]).select_from(foo).where(foo.c.bar == 1)
You can also directly pass parameters via the params parameter of from_postgis, if that's more natural:
df.from_postgis(text("select * from foo where bar = :a"), params={"a": 1})
Do not use str.format as the other answer suggests because it's vulnerable to SQL injection.
I would like to set a random seed when shuffling a sqlalchmy query result using order_by to get always the same query result:
my_query.order_by(func.random())
I could not find how to do it in the documentation. I tried it with
my_query.order_by(func.random(random_seed=1))
my_query.order_by(func.random(random_state=1))
but both didn't work.
Thank you in advance for any idea/hint!
All best,
Lea
The SQLAlchemy func object just renders functions in SQL.
>>> from sqlalchemy import func
>>> print(func.random())
random()
In some cases (including this one), it will render a different underlying function on different database backends, e.g. on MySQL:
>>> from sqlalchemy.dialects import mysql
>>> print(func.random().compile(dialect=mysql.dialect()))
rand()
This means that the database you're using is what produces the random number, and so you must call a database specific function to set the seed, e.g. on PostgreSQL you use setseed, which is called in the same way as other SQL functions:
>>> func.setseed(0.123...)
I need to pass a partial raw sql query into sqlalchemy filter, like
s.query(account).filter("coordinate <#> point(%s,%s) < %s"%(lat,long,distance))
Yes, I'm trying to use earthdistance function in postgresql.
Of course, I could use PostGis and GeoAlchemy2, but I want to know the general solution to this kind of problems.
I know sqlalchemy can safely pass raw sql query .
result = db.engine.execute("select * coordinate <#> point(:lat,:long) < :distance",**params)
Is there any similar function that can be used to bind parameter of partial(?) sql query? I guess someone who implements custom sql function like func.ll_to_earth have used the function.
There is .params() on query. Try this:
query = s.query(account).filter(
"coordinate <#> point(:lat, :long_) < :dist").params(
lat=lat, long_=long_, dist=distance)
And there is the documentation on it.
Note: I renamed your long param, because there is alread a __builtin__ named long (long int) in python, it's good practice to not overwrite already used words for obvious reasons.
I need to retrieve what's the current date and time for the database I'm connected with SQLAlchemy (not date and time of the machine where I'm running Python code). I've seen this functions, but they don't seem to do what they say:
>>> from sqlalchemy import *
>>> print func.current_date()
CURRENT_DATE
>>> print func.current_timestamp()
CURRENT_TIMESTAMP
Moreover it seems they don't need to be binded to any SQLAlchemy session or engine. It makes no sense...
Thanks!
I foud the solution: these functions cannot be used in the way I used (print...), but need to be called inside of the code that interacts with the database. For instance:
print select([my_table, func.current_date()]).execute()
or assigned to a field in an insert operation.
Accidentally I discovered that exists at least a couple of parameters for these functions:
type_ that indicates the type of the value to return, I guess
bind that indicates a binding to an SQLAlchemy engine
Two examples of use:
func.current_date(type_=types.Date, bind=engine1)
func.current_timestamp(type_=types.Time, bind=engine2)
Anyway my tests seems to say these parameters are not so important.