Develop Reference

Python sqlite3 conn.commit() not saving data to the db file

Here's my code
import sqlite3
conn = sqlite3.connect('arbitrary.db')
c = conn.cursor()
data = ['sampletext',42]
c.execute("CREATE TABLE IF NOT EXISTS t1(link TEXT,story_id INTEGER)")
c.execute("INSERT INTO t1 (link, story_id) VALUES (?,?)", data)
conn.commit()
conn.close()
It runs without error and doesn't throw any parsing errors, but once the script closes and I go to open the db file, It's still empty. am I making a mistake somewhere?
Appreciate the help in advance :)

Is commit used on the connect or cursor?

The book named "Practical Programming: 2nd Edition" has conflicting code. This is the start of my code:
import sqlite3
con = sqlite3.connect('stackoverflow.db')
cur = conn.cursor()
To commit, would I use con.commit(), cur.commit() or are there different times to use each? From the book:
con.commit() :
cur.commit() :
Documentation shows con.commit() :

I took unutbu's advice and tried it myself.
Sample code:
import sqlite3
con = sqlite3.connect('db.db')
cur = con.cursor()
data = [('data', 3), ('data2', 69)]
cur.execute('CREATE TABLE Density(Name TEXT, Number INTEGER)')
for i in data:
cur.execute('INSERT INTO Density VALUES (?, ?)', (i[0], i[1]))
cur.commit()
PyCharm Run:
Traceback (most recent call last):
File "/Users/User/Library/Preferences/PyCharmCE2018.1/scratches/scratch_2.py", line 13, in <module>
cur.commit()
AttributeError: 'sqlite3.Cursor' object has no attribute 'commit'
Error in textbook. cur.commit() does not exist.
Thanks unutbu and s3n0

con.commit() and conn.commit() are the same ... they are created object types ... in both cases they are otherwise named ... important is mainly .commit() and not the naming that the programmer has specified
There are object types that use a different name (con and cur - as you asked) to calling the method. You can also use a different name in your code, for example:
db = sqlite3.connect('/tmp/filename.db')
cursor = db.cursor()
cursor.execute("CREATE TABLE ....
.... some DB-API 2.0 commands ....
")
db.commit()
Please check again the webpage https://docs.python.org/3/library/sqlite3.html .
You forgot to copy these two lines from the webpage:
import sqlite3
conn = sqlite3.connect('example.db')
And then continuing the code (just copied it):
c = conn.cursor()
# Create table
c.execute('''CREATE TABLE stocks
(date text, trans text, symbol text, qty real, price real)''')
# Insert a row of data
c.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
# Save (commit) the changes
conn.commit()
# We can also close the connection if we are done with it.
# Just be sure any changes have been committed or they will be lost.
conn.close()

I think if you're using a specified cursor to commit changes, in your case, it should be cur.connection.commit().
You can always use connect to commit in the end of your code, whether it's named db, or con or conn.
But when your code gets complicated, you'll have different function to do certain operation to the database, if you only use connection commit, when there is a bug, you gonna have a hard time to find which function failed. So you create specific cursor for specific operation, when that failed, the traceback message will show you which specific cursor when wrong.

To #s3n0 & #DanielYu's point they can be handled two different ways. I had to list these out to better understand the overlap:
Connection Objects
backup
close
commit
create_aggregate
create_collation
create_function
cursor
enable_load_extension
execute
executemany
executescript
in_transaction
interrupt
isolation_level
iterdump
load_extension
rollback
row_factory
set_authorizer
set_progress_handler
set_trace_callback
text_factory
total_changes
Cursor objects
arraysize
close
connection
description
execute
executemany
executescript
fetchall
fetchmany
fetchone
lastrowid
rowcount
setinputsizes
setoutputsize

Psycopg2 command hangs, possibly because of table name?

The code below hangs (even ctrl-C won't stop it, I have to close the terminal) when it tries to create the second table, and I'm not sure why. The first table is created successfully (I can see it in psql with \dt cyanobacteria.*). A simple solution would be to rename the table, but I'm trying to restore someone else's code to working order and I'd have to go through changing lots of stuff. And he had it working once, so it ought to work for me!
I've created a database called 'genomes', a user called 'genomes_admin' and a schema called 'cyanobacteria'. Then I try to make some tables:
#!/usr/bin/python
import psycopg2
psql = "dbname='genomes' user='genomes_admin'"
schm = 'cyanobacteria'
conn = psycopg2.connect(psql)
cur = conn.cursor()
cur.execute('''SET search_path TO %s''', (schm,))
conn.commit()
cur.execute('''CREATE TABLE IF NOT EXISTS testnm(blah text, length int) ''')
print 'created testnm'
conn.commit()
print 'committed'
cur.execute('''CREATE TABLE IF NOT EXISTS genomes(blah text, length int) ''') # hangs here
print 'created genomes' # this line never executes
conn.commit()
print 'committed'
cur.close()
conn.close()

How do I insert data into table?

I have created table using this create command as:
CREATE TABLE test_table(id INT PRIMARY KEY,name
VARCHAR(50),price INT)
i want to insert into this table wherein values are stored already in variable
bookdb=# name = 'algorithms'
bookdb-# price = 500
bookdb-# INSERT INTO test_table VALUES(1,'name',price);
I get the following error:
ERROR: syntax error at or near "name"
LINE 1: name = 'algorithms'
Can anyone point out the mistake and propose solution for the above?
Thanks in advance
Edit:
import psycopg2
import file_content
try:
conn = psycopg2.connect(database='bookdb',user='v22')
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS book_details")
cur.execute("CREATE TABLE book_details(id INT PRIMARY KEY,name VARCHAR(50),price INT)")
cur.execute("INSERT INTO book_details VALUES(1,'name',price)")
conn.commit()
except:
print "unable to connect to db"
I have used the above code to insert values into table,variables name and price containing the values to be inserted into table are available in file_content python file and i have imported that file.The normal INSERT statement takes values manually but i want my code to take values which are stored in variables.

SQL does not support the concept of variables.
To use variables, you must use a programming language, such as Java, C, Xojo. One such language is PL/pgSQL, which you can think of as a superset of SQL. PL/PgSQL is often bundled as a part of Postgres installers, but not always.
I suggest you read some basic tutorials on SQL.
See this similar question: How do you use script variables in PostgreSQL?

don't have postgres installed here, but you can try this
import psycopg2
import file_content
try:
conn = psycopg2.connect(database='bookdb',user='v22')
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS book_details")
cur.execute("CREATE TABLE book_details(id INT PRIMARY KEY,name VARCHAR(50),price INT)")
cur.execute("INSERT INTO book_details VALUES(1, '%s', %s)" % (name, price))
conn.commit()
except:
print "unable to connect to db"

If you are using PSQL console:
\set name 'algo'
\set price 10
insert into test_table values (1,':name',:price)
\g

pymysql callproc() appears to affect subsequent selects

I'm attempting to transition a code base from using MySQLdb to pymysql. I'm encountering the following problem and wonder if anyone has seen something similar.
In a nutshell, if I call a stored procedure through the pymysql cursor callproc() method a subsequent 'select' call through the execute() method using the same or a different cursor returns incorrect results. I see the same results for Python 2.7.2 and Python 3.2.2
Is the callproc() method locking up the server somehow? Code is shown below:
conn = pymysql.connect(host='localhost', user='me', passwd='pwd',db='mydb')
curr = conn.cursor()
rargs = curr.callproc("getInputVar", (args,))
resultSet = curr.fetchone()
print("Result set : {0}".format(resultSet))
# curr.close()
#
# curr = conn.cursor()
curr.execute('select * from my_table')
resultSet = curr.fetchall()
print("Result set len : {0}".format(len(resultSet)))
curr.close()
conn.close()
I can uncomment the close() and cursor creation calls above but this doesn't change the result. If I comment out the callproc() invocation the select statement works just fine.

I have a similar problem with (committed) INSERT statements not appearing in the database. PyMySQL 0.5 für Python 3.2 and MySQL Community Server 5.5.19.
I found the solution for me: instead of using the execute() method, I used the executemany method, explained in the module reference on
http://code.google.com/p/pymssql/wiki/PymssqlModuleReference
There is also a link to examples.
Update
A little later, today, I found out that this is not yet the full solution.
A too fast exit() at the end of the python script makes the data getting lost in the database.
So, I added a time.sleep() before closing the connection and before exit()ing the script, and finally all the data appeared!
(I also switched to using a myisam table)
import pymysql
conn = pymysql.connect(host='localhost', user='root', passwd='', db='mydb', charset='utf8')
conn.autocommit(True)
cur = conn.cursor()
# CREATE tables (SQL statements generated by MySQL workbench, and exported with Menu -> Database -> Forward Engineer)
cur.execute("""
SET #OLD_UNIQUE_CHECKS=##UNIQUE_CHECKS, UNIQUE_CHECKS=0;
SET #OLD_FOREIGN_KEY_CHECKS=##FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0;
SET #OLD_SQL_MODE=##SQL_MODE, SQL_MODE='TRADITIONAL';
DROP SCHEMA IF EXISTS `mydb` ;
CREATE SCHEMA IF NOT EXISTS `mydb` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci ;
USE `mydb` ;
# […]
SET SQL_MODE=#OLD_SQL_MODE;
SET FOREIGN_KEY_CHECKS=#OLD_FOREIGN_KEY_CHECKS;
SET UNIQUE_CHECKS=#OLD_UNIQUE_CHECKS;
""")
# Fill lookup tables:
cur.executemany("insert into mydb.number(tagname,name,shortform) values (%s, %s, %s)", [('ЕД','singular','sg'), ('МН','plural','p')] )
cur.executemany("insert into mydb.person(tagname,name,shortform) values (%s, %s, %s)", [('1-Л','first','1st'), ('2-Л','second','2nd'), ('3-Л','third','3rd')] )
cur.executemany("insert into mydb.pos(tagname,name,shortform) values (%s, %s, %s)", [('S','noun','s'), ('A','adjective','a'), ('ADV','adverb','adv'), ('NUM','numeral','num'), ('PR','preposition','pr'), ('COM','composite','com'), ('CONJ','conjunction','conj'), ('PART','particle','part'), ('P','word-clause','p'), ('INTJ','interjection','intj'), ('NID','foreign-named-entity','nid'), ('V','verb','v')] )
#[…]
import time
time.sleep(3)
cur.close()
conn.close()
time.sleep(3)
exit()
I suggest the forum/group https://groups.google.com/forum/#!forum/pymysql-users for further discussion with the developer.