I'm working on a very complex raw sql statement that has to be run with sqlalchemy. One part of the statement is a JOIN with a temporary table that is filled with data from a csv file.
The select is looking as follows:
(SELECT * FROM (VALUES ('1','xyz#something.com','+99123456798')) AS t (id, email, phone))
To prevent any sql injections I cannot simply copy paste everything from within the csv into the select.
I know that sqlalchemy has the option of inserting values with :x and then pass the actual value in the execute method, but I have A LOT of values and substituting them will be impossible.
Is there a way to build this temporary table from the csv with the necessary sql injection protection using sqlalchemy?
Related
I'm trying to find a way to dynamically create table in database (SQLite3). What I'm trying to do is to get file from user and based on rows in file create table with the same amount of columns. Everything I found is from 6-10 years so it doesn't work anymore.
I suspect that you are not finding many examples are Django is abstracting out raw SQL.
Have you looked at using Raw SQL queries, specifically Executing custom SQL directly
table_name = 'Your_Name'
with connection.cursor() as cursor:
cursor.execute(f"create table if not exists {table_name} ( id integer PRIMARY KEY )")
Will create a table called 'Your_Name' with a column called id, you will need to read the CSV, there is an example of how to do that here, if you follow that example you could add the DDL into views.py
I wonder if there's anyways to upload a dataframe and create a new table in Exasol? import_from_pandas assumes the table already exists. Do we need to run a SQL separately to create the table? for other databases, to_sql can just create the table if it doesn't exist.
Yes, As you mentioned import_from_pandas requires a table. So, you need to create a table before writing to it. You can run a SQL create table ... script by connection.execute before using import_from_pandas. Also to_sql needs a table since based on the documentation it will be translated to a SQL insert command.
Pandas to_sql allows to create a new table if it does not exist, but it needs an SQLAlchemy connection, which is not supported for Exasol out of the box. However, there seems to be a SQLAlchemy dialect for Exasol you could use (haven't tried it yet): sqlalchemy-exasol.
Alternatively, I think you have to use a create table statement and then populate the table via pyexasol's import_from_pandas.
I'm trying to execute this query, it's working fine in MySQL workbench, but It's not working in python shell.
And I have to export that result in an Excel sheet.
As de error suggests the SELECT clause must contain the primary key in order to be able to build the ORM object. By default, the primary key is a column named pk. I advise using "SELECT *" in this case if possible, maybe the "AS" could give problems too.
I am using Python to_sql function to insert data in a database table from Pandas dataframe.
I am able to insert data in database table but I want to know in my code how many records are inserted .
How to know record count of inserts ( i do not want to write one more query to access database table to get record count)?
Also, is there a way to see logs for this function execution. like what were the queries executed etc.
There is no way to do this, since python cannot know how many of the records being inserted were already in the table.
Working on Postgres DB within python using pscopg2, with an ORM of pewee. I created the initial tables using pewee and I needed to perform an ALTER statement:
improt psycopg2
cur.execute("ALTER TABLE Test_Table ADD COLUMN filename VARCHAR(100)")
conn.commit()
Which after executed, I do a select * from Test_Table from and the table is present.
However, when I do a select using the pewee ORM, that column filename does not exist in the Test_Table.
What do I need to do in order for that ALTER statement to show up using peewee?
Peewee models are not dynamically-created based on the state of the database schema. They are declarative.
So if you are adding a column to your database, you would add a corresponding field instance to the model class. Typically this is done offline (e.g., not in the middle of while your application is running).
Refer here for docs on Peewee's schema migration utilities:
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#migrate