I have been trying to create a sqlite database using one python file and access data from it using another, but keep getting an error. I have 2 files, main.py and file2.py
main.py:
import sqlite3, os
conn = sqlite3.connect(':memory:')
queryCurs = conn.cursor()
def createTable():
queryCurs.execute('''CREATE TABLE test(id INTEGER PRIMARY KEY, name TEXT)''')
def addInitial(name):
queryCurs.execute('''INSERT INTO test(name) VALUES (?)''',(name,))
createTable()
addInitial("John")
conn.commit()
os.system('file2.py')
and here is the code in file2.py
import sqlite3, os, time
conn = sqlite3.connect(':memory:')
queryCurs = conn.cursor()
queryCurs.execute('SELECT name FROM test WHERE id=1')
for i in queryCurs:
for j in i:
name = j
print name
conn.commit()
I receive the error: OperationalError: no such table: test
Each connect call creates its own in-memory database.
To share the same in-memory database, create a single connection and share that Python object in both modules.
Related
import sqlite3
conn = sqlite3.connect("test.db")
cursor = conn.cursor()
It should create the database, but it does not. Any help?
This code will create an sqlite db file called "test.db" in the same directory you are running your script from.
For example, if you have your python file in:
/home/user/python_code/mycode.py
And you run it from:
/home/user/
With:
python python_code/mycode.py # or python3
It will create an "empty" sqlite db file at
/home/user/test.db
If you can't find the test.db file, make sure you pass it the full path of where you want it to be located.
i.e.
conn = sqlite3.connect("/full/path/to/location/you/want/test.db")
I had the same problem, my .db file wasn't appearing because I forgot to add test.db at the end of path, see line 2 below
import sqlite3
databaseFile = "/home/user/test.db" #don't forget the test.db
conn = sqlite3.connect(databaseFile)
cursor = conn.cursor()
I suspect the DB will not be created on disk until you create at least one table in it. Just calling conn.cursor() is not sufficient.
Console sqlite3 utility behaves this way, too.
I'm ssh'd into a remote server, and there I created two python files as a test: one to create a database, and one to read data from it.
The script that creates the database
import os
import sqlite3
# set up database
conn = sqlite3.connect('Materials.db')
c = conn.cursor()
def createTable():
c.execute("DROP TABLE IF EXISTS images")
c.execute("CREATE TABLE images(ID TEXT, url TEXT)")
createTable()
path = os.getcwd()
imagepath = "/home/rootadmin/1080_images"
imagedir = os.listdir(imagepath)
for image in range(0,len(imagedir)):
c.execute('INSERT INTO images(ID, url) VALUES(?,?)',(imagedir[image],'www.google.com'))
print(imagedir[image])
Here the print commands the data that is required, e.g. it prins the ID's of the images.
In my script to read the data from the db:
import sqlite3
conn = sqlite3.connect('Materials.db')
c = conn.cursor()
c.execute('SELECT ID FROM images')
objectId = c.fetchall()
print(objectId)
I have a limited knowledge of sqlite3, but I would expect the print command in the second script to print the ID's found in the images table, from the Materials.db, yet it returns an empty array.
You need to commit your transaction when inserting:
for image in range(0,len(imagedir)):
c.execute('INSERT INTO images(ID, url) VALUES(?,?)',(imagedir[image],'www.google.com'))
print(imagedir[image])
conn.commit()
or use the connection as a context manager to auto-commit when the context exits:
with conn:
for image in range(0,len(imagedir)):
c.execute('INSERT INTO images(ID, url) VALUES(?,?)',(imagedir[image],'www.google.com'))
print(imagedir[image])
This also ensures that the transaction is explicitly rolled back if there was an exception.
I wrote a Python script which initializes an empty database if it doesn't exist.
import os
if not os.path.exists('Database'):
os.makedirs('Database')
os.system('sqlite3 Database/testDB.db ";"')
# rest of the script...
Can I do this in a more Pythonic fashion, with a try-except, or is this kind of code acceptable?
I think you can do it like this:
import sqlite3
conn = sqlite3.connect('Database/testDB.db')
This should connect to your database and create it in case that it doesn't exist. I'm not sure this is the most pythonic way, but it does use the sqlite3 module instead of the sqlite3 command.
Making it Pythonic: create a sqlite3 database if it doesn't exist?
The most Pythonic way to do this is to use the context manager:
import sqlite3
# if we error, we rollback automatically, else commit!
with sqlite3.connect('/Temp/testDB.db') as conn:
cursor = conn.cursor()
cursor.execute('SELECT SQLITE_VERSION()')
data = cursor.fetchone()
print('SQLite version:', data)
In a python shell this echoes for me:
<sqlite3.Cursor object at 0x0CCAD4D0>
SQLite version: (u'3.5.9',)
To ensure you have a tempfile path that works across platforms, use tempfile.gettempdir:
import tempfile
with sqlite3.connect(tempfile.gettempdir() + '/testDB.db') as conn:
...
Create directory path, database file and table
Here is a recipe to create the directory path, database file and table
when necessary. If these already exist, the script will overwrite nothing and simply use what is at hand.
import os
import sqlite3
data_path = './really/deep/data/path/'
filename = 'whatever'
os.makedirs(data_path, exist_ok=True)
db = sqlite3.connect(data_path + filename + '.sqlite3')
db.execute('CREATE TABLE IF NOT EXISTS TableName (id INTEGER PRIMARY KEY, quantity INTEGER)')
db.close()
sqlite3.connect will attempt to create a database if it doesn't exist - so the only way to tell if one does exist is to try to open it and catch an IOError. Then to create a blank database, just connect using the sqlite3 module.
import sqlite3
try:
open('idonotexist')
print 'Database already exists!'
except IOError as e:
if e.args == 2: # No such file or directory
blank_db = sqlite3.connect('idontexist')
print 'Blank database created'
else: # permission denied or something else?
print e
Of course, you may still have to do something with os.makedirs depending on if the structure already exists.
I'm new to Python and I'm doing some tests with it.
What I need to know is what is the best way of dealing with configuration variables.
For example, for this code:
import twitter
import random
import sqlite3
import time
import bitly_api #https://github.com/bitly/bitly-api-python
class TwitterC:
def logtodatabase(self, tweet, timestamp):
# Will log to the database
database = sqlite3.connect('database.db') # Create a database file
cursor = database.cursor() # Create a cursor
cursor.execute("CREATE TABLE IF NOT EXISTS twitter(id_tweet INTEGER AUTO_INCREMENT PRIMARY KEY, tweet TEXT, timestamp TEXT);") # Make a table
# Assign the values for the insert into
msg_ins = tweet
timestamp_ins = timestamp
values = [msg_ins, timestamp_ins]
# Insert data into the table
cursor.execute("INSERT INTO twitter(tweet, timestamp) VALUES(?, ?)", values)
database.commit() # Save our changes
database.close() # Close the connection to the database
In this code, how can I replace 'database.db' with a variable outside of the class, for a better configuration. ?
Best Regards,
Or you could use argparse if you want to pass in configuration from the command line.
Then when creating the TwitterC class, you could pass in the configuration options you want.
class TwitterC:
def __init__(self, database):
self.database = database
def logtodatabase(self, tweet, timestamp):
# Will log to the database
database = sqlite3.connect(self.database) # Create a database file
#(...)
You could use ConfigParser out of the Python Standard Library.
You can create a python script that include the configuration variables:
config.py:
dbname = 'database.db'
...
your file:
import config
database = sqlite3.connect(config.dbname)
I have an access table that I am trying to add fields programmatically using Python. It is not a personal geodatabase. Just a standard Access database with some tables in it.
I have been able to access the table and get the list of field names and data types.
How do I add a new field and assign the data type to this Access table using Python.
Thanks!
SRP
Using the pyodbc module:
import pyodbc
MDB = 'c:/path/to/my.mdb'
DRV = '{Microsoft Access Driver (*.mdb)}'
PWD = 'my_password'
conn = pyodbc.connect('DRIVER=%s;DBQ=%s;PWD=%s' % (DRV,MDB,PWD))
c = conn.cursor()
c.execute("ALTER TABLE my_table ADD COLUMN my_column INTEGER;")
conn.commit()
c.close()
conn.close()
Edit:
Using win32com.client...
import win32com.client
conn = win32com.client.Dispatch(r'ADODB.Connection')
DSN = 'PROVIDER=Microsoft.Jet.OLEDB.4.0;DATA SOURCE=c:/path/to/my.mdb;'
conn.Open(DSN)
conn.Execute("ALTER TABLE my_table ADD COLUMN my_column INTEGER;")
conn.Close()