Develop Reference

How do I store file names in a database in python?

I am trying to get a list of files in a user specified directory to be saved to a database. What I have at the moment is :
import os
import sqlite3
def get_list():
folder = input("Directory to scan : ")
results = []
for path in os.listdir(folder):
if os.path.isfile(os.path.join(folder, path)):
results.append(path)
print(results)
return results
def populate(results):
connection = sqlite3.connect("videos.db")
with connection:
connection.execute("CREATE TABLE IF NOT EXISTS files (id INTEGER PRIMARY KEY, file_name TEXT);")
for filename in results:
insert_string = "INSERT INTO files (file_name) VALUES ('"+filename+"');"
connection.execute(insert_string)
filelist = get_list()
populate(filelist)
It runs without a problem and prints out a list of the file names, which is great, but then when it's running the INSERT SQL statement, that seems to have no effect on the database table. I have tried to debug it, and the statement which is saved in the variable looks good, and when executing it manually in the console, it inserts a row in the table, but when running it, nothing changes. Am I missing something really simple here ?

Python's SQLite3 module doesn't auto-commit by default, so you need to call connection.commit() after you've finished executing queries. This is covered in the tutorial.
In addition, use ? placeholders to avoid SQL injection issues:
cur.execute('INSERT INTO files (file_name) VALUES (?)', (filename,))
Once you do that, you can insert all of your filenames at once using executemany:
cur.executemany(
'INSERT INTO files (file_name) VALUES (?)',
[(filename,) for filename in results],
)

Convert Jpeg to Blob images in order to insert them in SQLITE3

I would like to store my pictures into my sqlite database with python. How can I do that?
Here is my code, but it is not working:
import sqlite3
import os
conn = sqlite3.connect('images.db')
cursor = conn.cursor()
sql_bd = 'CREATE TABLE IF NOT EXISTS tabela (foto BLOB);'
for i in os.listdir('\myphotos'):
cursor.execute("INSERT INTO tabela (foto) VALUES (?);", i)
conn.commit()
cursor.close()
conn.close()
Anyone could help me, please?

Using pathlib instead of os.listdir, it can become:
from pathlib import Path
mydir = Path("myphotos")
for image_path in Path("/myphotos").iterdir():
...
if image_path.suffix.lower() in [".jpeg", ".jpg", ...]:
cursor.execute("INSERT INTO tabela (foto) VALUES (?);", image_path.read_bytes()
...
Pathlib's iterdir already yields the full path for each file, not just the filename, and provide the read_bytes method: no need to open the file and call a method on the returned object.

How to update SQLite column with a list of values? [duplicate]

Can anyone tell me where I have done wrong?
my code:
import csv
import sqlite3
import os
import subprocess
import glob
#Connect to database
conn = sqlite3.connect("Mpeg_editor_Final.db")
try:
conn.execute("drop table Mpeg_editor_Final")
conn.execute("drop table edited")
conn.execute("drop table ffmpeg")
except sqlite3.OperationalError, e:
print e.message
#CREATE table in databse
conn.execute("PRAGMA foreign_keys = 1")
conn.execute("CREATE TABLE Mpeg_editor_Final (fileName VARCHAR(120), fileType VARCHAR(120), fileFolder VARCHAR(120))")
conn.execute("CREATE TABLE edited (fileName VARCHAR(120), fileType VARCHAR(120), fileFolder VARCHAR(120))")
conn.execute("CREATE TABLE ffmpeg (fileName VARCHAR(120), fileType VARCHAR(120), fileFolder VARCHAR(120))")
#mpegEditorFinal file location
mpegEditorFinal = 'C:\Mpeg_editor_Final'
#list all folders and file in Mpeg_editor_Final
mpegEditorFinaldirs = os.listdir(mpegEditorFinal)
# tell file's extensions
for i in mpegEditorFinaldirs:
mpegEditorFinalext = os.path.splitext(i)
#find current path
for x in mpegEditorFinaldirs:
mpegEditorFinalpath = os.path.dirname(x)
#To write information into the Mpeg_editor_final table
conn.executemany("INSERT INTO Mpeg_editor_Final (fileName, fileType, fileFolder) VALUES (?,?,?);", [mpegEditorFinaldirs , mpegEditorFinalext , mpegEditorFinalpath,])
conn.commit()
The error message:
Message File Name Line Position
Traceback
<module> C:\Mpeg_editor_Final\database.py 36
ProgrammingError: Incorrect number of bindings supplied. The current statement uses 3, and there are 16 supplied.

You've used executemany(), but you only provide parameters for a single execution. Either nest them further, or use execute() instead.
conn.executemany("INSERT INTO Mpeg_editor_Final (fileName, fileType, fileFolder) VALUES (?,?,?);", [[mpegEditorFinaldirs, mpegEditorFinalext, mpegEditorFinalpath]])
conn.execute("INSERT INTO Mpeg_editor_Final (fileName, fileType, fileFolder) VALUES (?,?,?);", [mpegEditorFinaldirs, mpegEditorFinalext, mpegEditorFinalpath])

Its because the conn.execute requires two parameters ..Here you havnt provided the exact parameters.
See here.It solves your problem .

Python, converting CSV file to SQL table

I have a CSV file without headers and am trying to create a SQL table from certain columns in the file. I tried the solutions given here: Importing a CSV file into a sqlite3 database table using Python,
but keep getting the error that col1 is not defined. I then tried inserting headers in my CSV file and am still getting a KeyError.
Any help is appreciated! (I am not very familiar with SQL at all)

If the .csv file has no headers, you don't want to use DictReader; DictReader assumes line 1 is a set of headers and uses them as keys for every subsequent line. This is probably why you're getting KeyErrors.
A modified version of the example from that link:
import csv, sqlite3
con = sqlite3.connect(":memory:")
cur = con.cursor()
cur.execute("CREATE TABLE t (col1, col2);")
with open('data.csv','rb') as fin:
dr = csv.reader(fin)
dicts = ({'col1': line[0], 'col2': line[1]} for line in dr)
to_db = ((i['col1'], i['col2']) for i in dicts)
cur.executemany("INSERT INTO t (col1, col2) VALUES (?, ?);", to_db)
con.commit()

This below code, will read all the csv files from the path and load all the data into table present in sqllite 3 database.
import sqllite3
import io
import os.path
import glob
cnx = sqlite3.connect(user='user', host='localhost', password='password',
database='dbname')
cursor=cnx.cursor(buffered= True);
path ='path/*/csv'
for files in glob.glob(path + "/*.csv"):
add_csv_file="""LOAD DATA LOCAL INFILE '%s' INTO TABLE tabkename FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 LINES;;;""" %(files)
print ("add_csv_file: %s" % files)
cursor.execute(add_csv_file)
cnx.commit()
cursor.close();
cnx.close();
Let me know if this works.

Python Sqlite3 Get Sqlite Connection path

Given an sqlite3 connection object, how can retrieve the file path to the sqlite3 file?

The Python connection object doesn't store this information.
You could store the path before you open the connection:
path = '/path/to/database/file.db'
conn = sqlite3.connect(path)
or you can ask the database itself what connections it has, using the database_list pragma:
for id_, name, filename in conn.execute('PRAGMA database_list'):
if name == 'main' and filename is not None:
path = filename
break
If you used a connection URI (connecting with the sqlite3.connect() parameter uri=True), the filename will not include the URI parameters or the file:// prefix.

We can use the PRAGMA database_list command.
cur = con.cursor()
cur.execute("PRAGMA database_list")
rows = cur.fetchall()
for row in rows:
print(row[0], row[1], row[2])
The third parameter (row[2]) is the file name of the database.
Note that there could be more databases attached to SQLite engine.
$ ./list_dbs.py
0 main /home/user/dbs/test.db
2 movies /home/user/dbs/movies.db
The above is a sample output of a script that contains the Python code.

Referencing Martijn Pieters, except hardcoding is a must, you should do this:
path = os.path.dirname(os.path.abspath(__file__))
db = os.path.join(path, 'file.db')
conn = sqlite3.connect(db)