I tried to insert row values for code column in statements table as a foreign key from companies Table. i took the following steps:
Creating Tables
cur.execute("CREATE TABLE IF NOT EXISTS companies (code INT NOT NULL PRIMARY KEY, short_name VARCHAR(255) NOT NULL, long_name VARCHAR(255) NOT NULL)")
cur.execute("CREATE TABLE IF NOT EXISTS statements (statement_id SERIAL NOT NULL PRIMARY KEY, statement_name VARCHAR(255) NOT NULL, code INT NOT NULL, FOREIGN KEY (code) REFERENCES companies_list (code))")
What code column contains in companies table ( i.e. )
code |
-----------
113
221
344
The next step is inserting wanted data to statements table as below :
statement_name = ["balance_sheet", "income_statement", "cash_flow"]
code = "SELECT code FROM companies_list WHERE code IS NOT NULL"
statements = [tuple((t,)) for t in zip(statement_name, code)]
query = "INSERT INTO statements (statement_name, code) VALUES %s"
cur.executemany(query, statements)
i got the following error :
psycopg2.DataError: invalid input syntax for integer: "S"
LINE 1: ...ents (statement_name, code) VALUES ('balance_sheet', 'S')
The Final result i want to get is like below :
statement_id | statement_name | code
---------------------------------------------
1 balance_sheet 113
2 income_statement 113
3 cash_flow 113
4 balance_sheet 221
5 income_statement 221
6 cash_flow 221
The error arises from this line:
code = "SELECT code FROM companies_list WHERE code IS NOT NULL"
This does not perform an actual query, it assigns the SQL select statement string to the code variable. The next line then zips the statement names with code which, because code is a string (an iterable), results in the first 3 characters of code being zipped with the items from statement_name, the result being:
[(('balance_sheet', 'S'),), (('income_statement', 'E'),), (('cash_flow', 'L'),)]
So that's where the 'S' is coming from - it's the first character of "SELECT" in the code string. 'S' is a string, not an integer as defined in the schema for the statements table, hence the error.
You can see the queries generated with cursor.mogrify():
>>> statement_name = ["balance_sheet", "income_statement", "cash_flow"]
>>> code = "SELECT code FROM companies_list WHERE code IS NOT NULL"
>>> statements = [tuple((t,)) for t in zip(statement_name, code)]
>>> query = "INSERT INTO statements (statement_name, code) VALUES %s"
>>> for args in statements:
... print(cur.mogrify(query, args))
...
INSERT INTO statements (statement_name, code) VALUES ('balance_sheet', 'S')
INSERT INTO statements (statement_name, code) VALUES ('income_statement', 'E')
INSERT INTO statements (statement_name, code) VALUES ('cash_flow', 'L')
One way of fixing this is to execute the query contained in code to get a list of company codes, then use that to construct the INSERT query:
import itertools
cur.execute("SELECT code FROM companies_list WHERE code IS NOT NULL")
codes = [row[0] for row in cur.fetchall()]
query = 'INSERT INTO statements (statement_name, code) VALUES (%s, %s)'
args = itertools.product(statement_name, codes)
cur.executemany(query, args)
Here itertools.product() is used to form the Cartesian product of the statement names and the company codes. This is mimicking database join functionality, so if the statement types are available in your database, it might be better to do it in SQL rather than Python.
Related
cursor.execute('''CREATE TABLE PEDIDO(
CPEDIDO INT GENERATED AS IDENTITY PRIMARY KEY,
CCLIENTE INT NOT NULL,
FECHA DATE NOT NULL
)''')
valores=[]
for i in range(10):
print(i)
x=datetime.date(year=2022,month=11,day=i+1)
valores.append((i,x))
cursor.executemany("INSERT INTO PEDIDO VALUES(?,?);", valores) #doesn't work writing [valores] instead of valores
That results in:
pyodbc.Error: ('HY000', '[HY000] [Devart][ODBC][Oracle]ORA-00947: not enough values\n (0) (SQLExecDirectW)') #when inserting the instances
I have tried to save data in two different tuples: cclients = (...) and dates=(...) and then write:
cursor.executemany("INSERT INTO PEDIDO VALUES(?,?);", [cclients, dates]).
But doesn't work
Name the columns you are inserting into (and you may not need/require including the statement terminator ; in the query):
INSERT INTO PEDIDO (CCLIENTE, FECHA) VALUES (?,?)
If you do not then Oracle will expect you to provide a value for every column in the table (including CPEDIDO).
You created a table with three columns, but you only provide two values in your SQL: INSERT INTO PEDIDO VALUES(?,?). The column CPEDIDO is defined as GENERATED AS IDENTITY. This means that you can provide a value for this column, but you don't have to. But if you leave out this column your SQL statement has to be adjusted.
INSERT INTO PEDIDO (CCLIENTE, FECHA) VALUES(?,?)
I use sqlalchemy engine for insertion data in postgresql table. I won't to insert list in one row as if list be a string with many value.
query = text('INSERT INTO table (list_id, list_name) VALUES ({}, {}) RETURNING'.format(my_list,'list_name'))
result_id = self.engine.execute(query)
when i tried execute my code I received error:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax error at or near "["
LINE 1: ...INTO table (list_id, list_name) VALUES (['str1... ^
[SQL: INSERT INTO table (list_id, list_name) VALUES (['str1', 'str1', 'str1'], 'list_name') RETURNING id]
I tried to represent my list as str(my list) but result was same. Also i try str(['str1', 'str1', 'str1']).replace('[', '{').replace(']', '}')
My table query:
CREATE TABLE api_services (
id SERIAL PRIMARY KEY,
list_id VARCHAR,
list_name VARCHAR(255) NOT NULL
);
I am trying to insert dates into a MySql database using pymysql and parameters. Some of the rows have a date field but for some others that particular date field is missing. The empty rows give an error of the type "pymysql.err.InternalError: (1292, "Incorrect date value:". Below is a piece of code that reproduces the error:
import pymysql
db=pymysql.connect("localhost","testuser","test1234","TESTDB")
cursor = db.cursor()
cursor.execute("SELECT VERSION()")
data = cursor.fetchone()
print ("Database version : %s " % data)
query = "DROP TABLE IF EXISTS birthdays"
cursor.execute(query)
query="CREATE TABLE birthdays(id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,\
name VARCHAR(20),\
birthday DATE NULL DEFAULT NULL)"
cursor.execute(query)
db.commit()
birthdays={'David':'2014-05-22','Larry':'014-05-22', 'Barry':''}
for k,v in birthdays.items():
print(k,v)
query="INSERT INTO birthdays (name,birthday) VALUES ('%s','%s')"%(k,v)
cursor.execute(query)
db.commit()
db.close()
The problem is with Barry and its empty date. I have tried setting Barry's date to None but it is not working. If I set it up to "NULL" and remove the quotes from the date parameter (('%s',%s) instead of ('%s','%s')) it works for Barry but note for the others.
thank you very much in advance,
Gabriel Vidal
You can use below code i have just change in your array and set 'Barry':None because When using mysqldb and cursor.execute(), pass the value None:
birthdays={'David':'2014-05-22','Larry':'014-05-22', 'Barry':None}
for k,v in birthdays.items():
print(k,v)
query="INSERT INTO birthdays (name,birthday) VALUES ('%s','%s')"%(k,v)
cursor.execute(query)
db.commit()
db.close()
For further detail here
In case someone drops by:
The following code snippet solves the problem you had. Note that you should keep your execute statements outside of the for loop to minimize the number of connections you do with the database.
birthdays = {'David':'2014-05-22','Larry':'2014-05-22', 'Barry':'NULL'}
values = ""
for k,v in birthdays.items():
values += "('%s', '%s')," % (k,v)
values = values[:-1].replace("'NULL'", "NULL") # Drop the final comma and the quotes of NULL values
query = f"INSERT INTO birthdays (name,birthday) VALUES {values}" # >= 3.6
cursor.execute(query) # Interaction with the database occurs only once
db.commit()
This generates the following statement:
INSERT INTO birthdays (name,birthday) VALUES ('David', '2014-05-22'),('Larry', '2014-05-22'),('Barry', NULL)
which is a valid sql statement
How do I insert a python dictionary into a Postgresql2 table? I keep getting the following error, so my query is not formatted correctly:
Error syntax error at or near "To" LINE 1: INSERT INTO bill_summary VALUES(To designate the facility of...
import psycopg2
import json
import psycopg2.extras
import sys
with open('data.json', 'r') as f:
data = json.load(f)
con = None
try:
con = psycopg2.connect(database='sanctionsdb', user='dbuser')
cur = con.cursor(cursor_factory=psycopg2.extras.DictCursor)
cur.execute("CREATE TABLE bill_summary(title VARCHAR PRIMARY KEY, summary_text VARCHAR, action_date VARCHAR, action_desc VARCHAR)")
for d in data:
action_date = d['action-date']
title = d['title']
summary_text = d['summary-text']
action_date = d['action-date']
action_desc = d['action-desc']
q = "INSERT INTO bill_summary VALUES(" +str(title)+str(summary_text)+str(action_date)+str(action_desc)+")"
cur.execute(q)
con.commit()
except psycopg2.DatabaseError, e:
if con:
con.rollback()
print 'Error %s' % e
sys.exit(1)
finally:
if con:
con.close()
You should use the dictionary as the second parameter to cursor.execute(). See the example code after this statement in the documentation:
Named arguments are supported too using %(name)s placeholders in the query and specifying the values into a mapping.
So your code may be as simple as this:
with open('data.json', 'r') as f:
data = json.load(f)
print(data)
""" above prints something like this:
{'title': 'the first action', 'summary-text': 'some summary', 'action-date': '2018-08-08', 'action-desc': 'action description'}
use the json keys as named parameters:
"""
cur = con.cursor()
q = "INSERT INTO bill_summary VALUES(%(title)s, %(summary-text)s, %(action-date)s, %(action-desc)s)"
cur.execute(q, data)
con.commit()
Note also this warning (from the same page of the documentation):
Warning: Never, never, NEVER use Python string concatenation (+) or string parameters interpolation (%) to pass variables to a SQL query string. Not even at gunpoint.
q = "INSERT INTO bill_summary VALUES(" +str(title)+str(summary_text)+str(action_date)+str(action_desc)+")"
You're writing your query in a wrong way, by concatenating the values, they should rather be the comma-separated elements, like this:
q = "INSERT INTO bill_summary VALUES({0},{1},{2},{3})".format(str(title), str(summery_text), str(action_date),str(action_desc))
Since you're not specifying the columns names, I already suppose they are in the same orders as you have written the value in your insert query. There are basically two way of writing insert query in postgresql. One is by specifying the columns names and their corresponding values like this:
INSERT INTO TABLE_NAME (column1, column2, column3,...columnN)
VALUES (value1, value2, value3,...valueN);
Another way is, You may not need to specify the column(s) name in the SQL query if you are adding values for all the columns of the table. However, make sure the order of the values is in the same order as the columns in the table. Which you have used in your query, like this:
INSERT INTO TABLE_NAME VALUES (value1,value2,value3,...valueN);
I need to write an INSERT statement that first checks to see if the data already exists. The current code is inside python using psycopg2 to connect to a postgresql db:
sql = """IF NOT EXISTS (SELECT * FROM table \
WHERE col_1 = (%s) AND col_2 = (%s) ) \
INSERT INTO table (col1, col2) \
VALUES (%s, %s);"""
data = ( col1_data, col2_data, col1_data, col2_data)
try:
CURSOR.execute(sql, data)
DB.commit()
except:
print "Cursor failed INSERT INTO table.\n"
which does not work (and I haven't done quality error handling so I don't get any good information).
So, I went into psql and tried just:
IF NOT EXISTS (SELECT * FROM t WHERE c1=d1 AND c2=d2)
INSERT INTO t (c1, c2) VALUES (d1,d2);
and I got the following error:
ERROR: syntax error at or near "IF"
LINE 1: IF NOT EXISTS (SELECT * FROM table WHERE c1 = d1...
^
So I BELIEVE my error is in the sql not the python (though I could be wrong) since this works:
sql = """INSERT INTO t2 (col_0, col_1, col_2) \
VALUES (%s, %s, %s);"""
data = (d1, d2, time.time())
try:
CURSOR.execute(sql, data)
DB.commit()
except:
print "Cursor failed to INSERT INTO t2.\n"
For table 1, my CREATE was:
db=> CREATE TABLE table (
col_0 SERIAL PRIMARY KEY,
col_1 varchar(16),
col_2 smallint
);
NOTICE: CREATE TABLE will create implicit sequence "pm_table_ip_id_seq" for serial column "pm_table.ip_id"
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "pm_table_pkey" for table "pm_table"
CREATE TABLE
I am grateful for any help and guidance.
I used plpgsql for such a requirement in my project
insert_function = """
CREATE LANGUAGE plpgsql;
CREATE FUNCTION insert_if_unique (sql_insert TEXT)
RETURNS VOID
LANGUAGE plpgsql
AS $$
BEGIN
EXECUTE sql_insert;
RETURN;
EXCEPTION WHEN unique_violation THEN
RETURN;
-- do nothing
END;
$$;
"""
cursor.execute(insert_function);
You can use something like below to use it:
cursor.execute("insert_if_unique("+sql+")"%data)
The above query is not parameterized. So please be wary of SQL injection if you are getting the input from an external source.
Note: You can use cursor.mogrify() to evade SQL injection attacks.
sql = cursor.mogrify(sql,data)
cursor.execute("insert_if_unique("+sql+")")
Try reversing those. NOT EXISTS condition with a subquery:
INSERT INTO t (c1, c2) VALUES (d1,d2)
WHERE NOT EXISTS (SELECT * FROM t WHERE c1=d1 AND c2=d2)