Develop Reference

Getting "sqlite3.OperationalError: no such column:"

I am new to python (as I am sure you can tell from my code) and have basically written a sample script to test some core commands so I can better understand how things work. I have gotten everything running as designed except the last "insert" command -- after hours of googling and experimenting I cannot figure out what's wrong, so I would appreciate it if anyone can show me what needs to be changed and helping me understand why (I'm sure it's basic, but am stumped!).
Below is the line that is giving me trouble:
c.execute("INSERT OR IGNORE INTO {tn} ({cn1}, {cn2}, {cn3}, {cn4}) VALUES ({VID}, {VSnu}, {VIN}, {VName})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, VID=ID, VSnu=Snu, VIN=IN, VName=Name))
Here's the entire script for context, if that helps:
import sqlite3
sqlite_file = '/test_database.sqlite' # name of the sqlite database file
table_name = 'test_table'
column1 = 'my_1st_column'
column2 = 'my_2nd_column'
column3 = 'my_3rd_column'
column4 = 'my_4th_column'
ID = int(123456)
Base = 'Arnold'
Snu = 'test'
conn = sqlite3.connect(sqlite_file)
c = conn.cursor()
c.execute("UPDATE {tn} SET {cn2}=('Snu'), {cn3}=('Muh'), {cn4}=('Arnold_A') WHERE {cn1}=({NID})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, NID=ID))
i = 1
while(i<15):
if i == 1: IN = 'B'
if i == 2: IN = 'C'
if i == 3: IN = 'D'
if i == 4: IN = 'E'
if i == 5: IN = 'F'
if i == 6: IN = 'G'
if i == 7: IN = 'H'
if i == 8: IN = 'I'
if i == 9: IN = 'J'
ID = ID+1
i = i+1
Name = Base + '_' + IN
params = (Snu, IN, Name)
c.execute("INSERT OR IGNORE INTO {tn} ({cn1}, {cn2}, {cn3}, {cn4}) VALUES ({VID}, {VSnu}, {VIN}, {VName})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, VID=ID, VSnu=Snu, VIN=IN, VName=Name))
if(i == 10): break
conn.commit()
conn.close()
It will write the "VID" item just fine if isolated (which is an integer column and the primary key of the table), but everything after that it interprets as a column and I get the "no such column: [value]" error.
I get this error regardless of what comes after "VID" -- be it any variable, as demonstrated with the command above, or if I try to insert a direct string value. These other columns are simply text columns, if that helps.

I know you are just learning, so don't be scared by what I'm going to say next :-)
You are improperly forming SQL commands. There's a special feature known as parameter substitution that is present in many database APIs, including that of SQLite in Python.
You don't want to concatenate values like you are doing, because it opens up room for something called SQL injection. In simple scenarios like yours it may not a problem, but when people do it in services exposed on the internet, it can cause great damage (data theft, data loss, data corruption etc). This comic illustrates the problem: https://xkcd.com/327/
In your case, writing the SQL command with parameter substitution is not only going to make your code safer and easier to read, it will also solve the problem you are seeing.
See the example below:
import sqlite3
conn = sqlite3.connect(":memory:")
c = conn.cursor()
# Here we don't need parameter substitution
c.execute("CREATE TABLE mytable (name text, quantity int)")
# We set the values somewhere else. For example, this could come from a web
# form
entry = ("myname", 2)
# Now we execute the query, with the proper parameter substitution.
# Note the `?` characters. Python will automatically replace those with the
# values in `entry`, making sure that things are right when forming the
# command before passing to SQLite.
c.execute("INSERT OR IGNORE INTO mytable VALUES (?, ?)", entry)
print(c.execute("SELECT * from mytable").fetchall())
The example above assumes you don't need to parametrize the column names. In your case, you are apparently reading all columns, so you don't really need to pass all the names, just do like I did in the example above and it will read all columns. If you really need column names to be parameters to read a subset of data, then you'll have to resort to a concatenation mechanism similar to what you were using. But in that case, as always, be really careful with user input to make sure it's not forming bad queries and commands.
More details here: https://docs.python.org/3/library/sqlite3.html

Using WHERE in query PyMySQL

I am trying to create a program where a user can enter an operator i.e. <> or = and then a number for a database in pymysql. I have tried a number of different ways of doing this but unfortunately unsuccessful. I have two documents with display being one and importing display into the other document.
Docuemnt 1
def get_pop(op, pop):
if (not conn):
connect();
query = "SELECT * FROM city WHERE Population %s %s"
with conn:
cursor = conn.cursor()
cursor.execute(query, (op, pop))
x = cursor.fetchall()
return x
Document two
def city():
op = input("Enter < > or =: ")
population = input("Enter population: ")
pop = display.get_pop(op, population)
for p in pop:
print(pop)
I am getting the following error.
pymysql.err.ProgrammingError: (1064,......
Please help thanks

You can't do this. Parameterization works for values only, not operators or table names, or column names. You'll need to format the operator into the string. Do not confuse the %s placeholder here with Python string formatting; MySQL is awkward in that it uses %s for binding parameters, which clashes with regular Python string formatting.
The MySQL %s in a query string escapes the user input to protect against SQL Injection. In this case, I set up a basic test to see if the operation part submitted by the user was in a list of accepted operations.
def get_pop(op, pop):
query = "SELECT * FROM city WHERE Population {} %s" # Add a placeholder for format
with conn: # Where does this come from?
cursor = conn.cursor()
if op in ['=', '!=']:
cursor.execute(query.format(op), (pop,))
x = cursor.fetchall()
return x
You'll want to come up with some reasonable return value in the case that if op in ['=', '!='] is not True but that depends entirely on how you want this to behave.

After checking that op indeed contains either "<>" or "=" and that pop indeed contains a number you could try:
query = "SELECT * FROM city WHERE Population " + op + " %s";
Beware of SQL injection.
Then
cursor.execute(query, (pop))

psycopg2 Programming Error when using .format()

Im not sure exactly whats happening here but it might have something to do with format in python.
Running this causes an error.
x = '00000201000012EB'
sql = """ SELECT * FROM table WHERE id = {} """.format(x)
conn.execute(sql)
I get an error saying: syntax error near "EB"
however when i run the command like this:
sql = """ SELECT * FROM table WHERE id = '00000201000012EB' """
conn.execute(sql)
It works fine.
Is there something wrong with the way im formatting this sql statement?

Use the variable as an argument to execute():
cur.execute(""" SELECT * FROM my_table WHERE id = %s """, (x,))
If you are determined to use format(), you should add single quotes around the placeholder:
sql = """ SELECT * FROM my_table WHERE id = '{}' """.format(x)

Believe it or not it was fixed by adding more quotes to the string.
this finally worked.
x = '00000201000012EB'
sql = """ SELECT * FROM table WHERE id = {} """.format("'" + x + "'")
Since the sql statement required another set of quotes i just added them to ensure it was treated as its own string.

How can I insert a Python variable in a sqlite query?

I'm looking for the way to be able to compare what is in a Python variable with something in a SQLite query.
For example:
num = input ("enter a number")
cur.execute("SELECT a.Name,a.Number FROM Ads a WHERE a.Number = (num)")
row = cur.fetchone()
while row:
print(row)
row = cur.fetchone()
The cur.execute line doesn't work. I don't know how to be able to compare content of a Python variable with data in a SQLite database via a SQlite query.

You put a '?' as a place holder for each Python variable, something like
num = 5
cur.execute("SELECT Name, Number FROM Ads WHERE Number = ?", (num,))
The execute() method expects a sequence as a second argument, since several '?' positional placeholders can be used in the SQL statement.

How to get the numbers of data rows from sqlite table in python

I am trying to get the numbers of rows returned from an sqlite3 database in python but it seems the feature isn't available:
Think of php mysqli_num_rows() in mysql
Although I devised a means but it is a awkward: assuming a class execute sql and give me the results:
# Query Execution returning a result
data = sql.sqlExec("select * from user")
# run another query for number of row checking, not very good workaround
dataCopy = sql.sqlExec("select * from user")
# Try to cast dataCopy to list and get the length, I did this because i notice as soon
# as I perform any action of the data, data becomes null
# This is not too good as someone else can perform another transaction on the database
# In the nick of time
if len(list(dataCopy)) :
for m in data :
print("Name = {}, Password = {}".format(m["username"], m["password"]));
else :
print("Query return nothing")
Is there a function or property that can do this without stress.

Normally, cursor.rowcount would give you the number of results of a query.
However, for SQLite, that property is often set to -1 due to the nature of how SQLite produces results. Short of a COUNT() query first you often won't know the number of results returned.
This is because SQLite produces rows as it finds them in the database, and won't itself know how many rows are produced until the end of the database is reached.
From the documentation of cursor.rowcount:
Although the Cursor class of the sqlite3 module implements this attribute, the database engine’s own support for the determination of “rows affected”/”rows selected” is quirky.
For executemany() statements, the number of modifications are summed up into rowcount.
As required by the Python DB API Spec, the rowcount attribute “is -1 in case no executeXX() has been performed on the cursor or the rowcount of the last operation is not determinable by the interface”. This includes SELECT statements because we cannot determine the number of rows a query produced until all rows were fetched.
Emphasis mine.
For your specific query, you can add a sub-select to add a column:
data = sql.sqlExec("select (select count() from user) as count, * from user")
This is not all that efficient for large tables, however.
If all you need is one row, use cursor.fetchone() instead:
cursor.execute('SELECT * FROM user WHERE userid=?', (userid,))
row = cursor.fetchone()
if row is None:
raise ValueError('No such user found')
result = "Name = {}, Password = {}".format(row["username"], row["password"])

import sqlite3
conn = sqlite3.connect(path/to/db)
cursor = conn.cursor()
cursor.execute("select * from user")
results = cursor.fetchall()
print len(results)
len(results) is just what you want

Use following:
dataCopy = sql.sqlExec("select count(*) from user")
values = dataCopy.fetchone()
print values[0]

When you just want an estimate beforehand, then simple use COUNT():
n_estimate = cursor.execute("SELECT COUNT() FROM user").fetchone()[0]
To get the exact number before fetching, use a locked "Read transaction", during which the table won't be changed from outside, like this:
cursor.execute("BEGIN") # start transaction
n = cursor.execute("SELECT COUNT() FROM user").fetchone()[0]
# if n > big: be_prepared()
allrows=cursor.execute("SELECT * FROM user").fetchall()
cursor.connection.commit() # end transaction
assert n == len(allrows)
Note: A normal SELECT also locks - but just until it itself is completely fetched or the cursor closes or commit() / END or other actions implicitely end the transaction ...

I've found the select statement with count() to be slow on a very large DB. Moreover, using fetch all() can be very memory-intensive.
Unless you explicitly design your database so that it does not have a rowid, you can always try a quick solution
cur.execute("SELECT max(rowid) from Table")
n = cur.fetchone()[0]
This will tell you how many rows your database has.

I did it like
cursor.execute("select count(*) from my_table")
results = cursor.fetchone()
print(results[0])

this code worked for me:
import sqlite3
con = sqlite3.connect(your_db_file)
cursor = con.cursor()
result = cursor.execute("select count(*) from your_table").fetchall() #returns array of tupples
num_of_rows = result[0][0]

A simple alternative approach here is to use fetchall to pull a column into a python list, then count the length of the list. I don't know if this is pythonic or especially efficient but it seems to work:
rowlist = []
c.execute("SELECT {rowid} from {whichTable}".\
format (rowid = "rowid", whichTable = whichTable))
rowlist = c.fetchall ()
rowlistcount = len(rowlist)
print (rowlistcount)

The following script works:
def say():
global s #make s global decleration
vt = sqlite3.connect('kur_kel.db') #connecting db.file
bilgi = vt.cursor()
bilgi.execute(' select count (*) from kuke ') #execute sql command
say_01=bilgi.fetchone() #catch one query from executed sql
print (say_01[0]) #catch a tuple first item
s=say_01[0] # assign variable to sql query result
bilgi.close() #close query
vt.close() #close db file