Develop Reference

How can I insert a None/Null value for a int column?

I'm using fast API and I have an optional input. In the event it's not provided, I just do not want to insert anything, here's my class:
class NetZeroAddGoalsRequest(BaseRequest):
target: int
description: str
target_date: Optional[str] = "1/1/2050"
update_frequency_month: Optional[int] = None
then I update it into my db:
company, target, description, target_date, update_frequency_month = body.company.upper(), body.target, body.description, body.target_date, body.update_frequency_month
# insert into DB
res = await main_db_instance.fetch_rows(f"INSERT INTO company.transition_company (company_name, target_carbon, description, target_date, update_frequency_month)"
f" VALUES ('{company}', '{target}', '{description}', '{target_date}', '{update_frequency_month}')"
f" RETURNING *")
but I am getting this error:
invalid input syntax for type integer: "None"
my DB column type is int, and I've tried to set the class variable to "" but then I get an error that I'm passing a string.
How do I have an optional int in fast API?
edit: if it helps, I'm using asyncpg - I tried to isolate it and it seems to be with the insert statement. If I comment it out then I do not get the error.

I suggest to use some ORM, like SQLalchemy ORM or even something that helps us remove boilerplate with fastapi/sql models (tortoise is great here!) with them you could do something as simple as
session.add(MyModel(**body.dict()))
But if you just want to make your example working, convert None to null, as db is not aware of Nones
update_frequency_month = 'null' if update_frequency_month is None else update_frequency_month
And do not use '' in sql statement

my DB column type is int, and I've tried to set the class variable to
"" but then I get an error that I'm passing a string.
None (Python type, not a postgres/SQL type) is not int, "" is not int. If you are using asyncpg it should convert the value automatically to null from None. I suspect that is because you insert the values directly into the SQL using a formatted string.
Try using native query argument syntax:
('''
INSERT INTO company.transition_company (company_name, target_carbon..)
VALUES ($1, $2,..)
''', company, target,..)

How to use date time as a cluster in where clause in Cassandra using python driver?

I am using python driver and Cassandra, I have created the following schema
CREATE TABLE channelFollowers(
channelID BIGINT,
isfavorite BOOLEAN,
userID BIGINT,
followDate TIMESTAMP,
PRIMARY KEY (userID, channelID, followDate)
);
my question is, how can i use the followDate in where clause of select and update query, I have tried but it does not work it gives the following error
typeerror: not enough arguments for format string
can any body help me please?
Here is my code
channelLike = channelSession.execute("update channelfollowers set isblocked=%s, isfavorite=%s where userid=%s and channelid=%s and followdate=%s",[int(userid),int(channel_id),followDate]

From what I see in the error message & in your code - you have 5 placeholders in format string (%s), but pass only 3 values. I believe that you omit placeholders for isblocked & isfavorite parameters - either pass them, or replace %s with true/false values (as I can see from query, they should have boolean type)
P.S. you also don't have isblocked in the table's definition

psycopg2 TypeError: not all arguments converted during string formatting

I'm trying execute a simple query, but getting this error no matter how I pass the parameters.
Here is the query (I'm using Trac db object to connect to a DB):
cursor.execute("""SELECT name FROM "%s".customer WHERE firm_id='%s'""" % (schema, each['id']))
schema and each['id'] both are simple strings
print("""SELECT name FROM "%s".customer WHERE firm_id='%s'""" % (schema, each['id']))
Result:
SELECT name FROM "Planing".customer WHERE firm_id='135'
There is on error is a remove quote after firm_id=, but that way parameter is treated a an integer and ::text leads to the very same error.

In my case I didn't realize that you had to pass a tuple to cursor.execute. I had this:
cursor.execute(query, (id))
But I needed to pass a tuple instead
cursor.execute(query, (id,))

I got this same error and couldn't for the life of me work out how to fix, in the end it was my mistake because I didn't have enough parameters matching the number of elements in the tuple:
con.execute("INSERT INTO table VALUES (%s,%s,%s,%s,%s)",(1,2,3,4,5,6))
Note that I have 5 elements in the values to be inserted into the table, but 6 in the tuple.

It is recommended to not use string interpolation for passing variables in database queries, but using string interpolation to set the table name is fine as long as it's not an external input or you restrict the allowed value. Try:
cursor.execute("""
SELECT name FROM %s.customer WHERE firm_id=%%s
""" % schema, (each['id'],))
Rules for DB API usage provides guidance for programming against the database.

Use AsIs
from psycopg2.extensions import AsIs
cursor.execute("""
select name
from %s.customer
where firm_id = %s
""",
(AsIs(schema), each['id'])
)

You could try this:
cursor.execute("INSERT INTO table_name (key) VALUES(%s)",(value1,))
You will get an error if you are missing a (,) after value1.

The correct way to pass variables in a SQL command is using the second argument of the execute() method. And i think you should remove single quotes from second parameter, read about it here - http://initd.org/psycopg/docs/usage.html#the-problem-with-the-query-parameters.
Note that you cant pass table name as parameter to execute and it considered as bad practice but there is some workarounds:
Passing table name as a parameter in psycopg2
psycopg2 cursor.execute() with SQL query parameter causes syntax error
To pass table name try this:
cursor.execute("""SELECT name FROM "%s".customer WHERE firm_id=%s""" % (schema, '%s'), (each['id'],))

Every time I have this kind of error, I am passing the wrong amount of values. Try check it

Issues with pyodbc numeric values being labelled as 'None' instead of NULL or empty

I am currently taking numeric values (amongst many other string and numeric values) from a set of access databases and uploading them to a single MS SQL Server database.
I am using 32-bit Python 3.3 and the respective pyodbc package.
I was wondering if there is a way to capture the fact that the numeric field is empty in the
Access database without the driver returning the string 'None' instead*. The syntax used is as follows:
access_con = pyodbc.connect(connection_string)
access_cur = access_con.cursor()
access_SQL = 'SELECT * FROM ' + source_table
rows = access_cur.execute(access_SQL).fetchall()
for row in rows:
[Statement uploading each row to SQL Server using an INSERT INTO statement]
Any help would be appreciated; whether as a solution or as a more direct way to transfer the data.
*EDIT: 'None' is only a string because I turned it into one to add it to the INSERT INTO statement. Using row.replace('None','NULL') replaced all of the 'None' instances and replaced it with 'NULL' which was interpreted as a NULL value by the ODBC driver.

None is a Python object, not a string. It is the equivalent of NULL in SQL Server, or an "empty" column value in Access.
For example, if I have an Access table with the following definition:
That contains the following values (note that the first value of the Number column is empty):
Relevant Python code produces:
...
>>> cursor = connection.cursor()
>>> rows = cursor.execute('select * from Table1').fetchall()
>>> print(rows)
[(None, ), (1, )]
This sample confirms the empty Access value is returned as None.
This PyODBC Documentation provides a good explanation of how ODBC and Python data types are mapped.

python, mysql, inserting string into table, error 1054

I am having the problem
OperationalError: (1054, "Unknown column 'Ellie' in 'field list'")
With the code below, I'm trying to insert data from json into a my sql database. The problem happens whenever I try to insert a string in this case "Ellie" This is something do to with string interpolation I think but I cant get it to work despite trying some other solutions I have seen here..
CREATE TABLE
con = MySQLdb.connect('localhost','root','','tweetsdb01')
cursor = con.cursor()
cursor.execute("CREATE TABLE IF NOT EXISTS User(user_id BIGINT NOT NULL PRIMARY KEY, username varchar(25) NOT NULL,user varchar(25) NOT NULL) CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE=InnoDB")
con.commit()
INSERT INTO
def populate_user(a,b,c):
con = MySQLdb.connect('localhost','root','','tweetsdb01')
cursor = con.cursor()
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(%s,%s,%s)"%(a,b,c))
con.commit()
cursor.close()
READ FILE- this calls the populate method above
def read(file):
json_data=open(file)
tweets = []
for i in range(10):
tweet = json.loads(json_data.readline())
populate_user(tweet['from_user_id'],tweet['from_user_name'],tweet['from_user'])

Use parametrized SQL:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES (%s,%s,%s)", (a,b,c))
(Notice the values (a,b,c) are passed to the function execute as a second argument, not as part of the first argument through string interpolation). MySQLdb will properly quote the arguments for you.
PS. As Vajk Hermecz notes, the problem occurs because the string 'Ellie' is not being properly quoted.
When you do the string interpolation with "(%s,)" % (a,) you get
(Ellie,) whereas what you really want is ('Ellie',). But don't bother doing the quoting yourself. It is safer and easier to use parametrized SQL.

Your problem is that you are adding the values into the query without any escaping.... Now it is just broken. You could do something like:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(\"%s\",\"%s\",\"%s\")"%(a,b,c))
But that would just introduce SQL INJECTION into your code.
NEVER construct SQL statements with concatenating query and data. Your parametrized queries...
The proper solution here would be:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(%s,%s,%s)", (a,b,c))
So, the problem with your code was that you have used the % operator which does string formatting, and finally you just gave one parameter to cursor.execute. Now the proper solution, is that instead of doing the string formatting yourself, you give the query part to cursor.execute in the first parameter, and provide the tuple with arguments in the second parameter.