I'm trying to perform queries on an SQLite database from Python. Problem is, I do not have write access to the database file, and also not to the directory containing the database file.
When I connect to the database and perform a SELECT query I get an "Unable to open database file" error. I tried following the advice this answer, but it didn't work. I guess SQLite fails when trying to create the lock files.
When I have write access to the directory, but not to the sqlite file, I get another error - a locking error. This is because sqlite creates the shm and wal files with the same permissions as the db file, meaning I get shm and wal files I can't write to, resulting in a locking error.
Other than copying all files to a directory I do have full access to, is there another way around this?
The documentation says:
It is not possible to open read-only WAL databases. The opening process must have write privileges for "-shm" wal-index shared memory file associated with the database, if that file exists, or else write access on the directory containing the database file if the "-shm" file does not exist.
To allow read-only access to that database, some user with write permissions needs to change it to some other journal mode.
You can open a sqlite3 connection in read mode with the following syntax:
con = sqlite3.connect('file:path/to/database.sqlite?mode=ro', uri=True)
Related
I am using flask sqlalchemy to create db which in turns create a app.db files to store tables and data. Now for the backup it should be simple to just take a copy of app.db somewhere in the server. But suppose while the app is writing data to app.db and we make a copy at that time then we might have inconsistent app.db file.
How do we maintain the consistency of the backup. I can implement locks to do so. But I was wondering on standard and good solutions for database backup and how is this implemented in python.
SQLite has the backup API for this, but it is not available in the built-in Python driver.
You could
use the APSW library for the backup; or
execute the .backup command in the sqlite3 command-line shell; or
run BEGIN IMMEDIATE to prevent other connections from writing to the DB, and copy the file(s).
I wanted also create backups of my SQL file, which stores data from my Flask.
Since my Flask web don't have too many users, and the database don't carry more than 4 small-medium size tables I did the following:
#Using Linux cmd
cd pathOfMyDb
nano backupDbRub.sh # backupDBRun is the name of the file that will be called periodically
Nano text editor will open a file and you can write the following
#!/bin/bash
current_time=$(date "+%Y.%m.%d-%H.%M.%S");
sep='"';
file_name = '_site.db"';
functionB = '.backup ';
queryWrite = "$functionB$sep$current_time$file_name";
echo "$queryWrite";
sqlite3 yoursqlitefile.db "$queryWrite"
Basically with this code we create a function ($queryWrite) which will do a copy of the current database with a timestamp. Now in Unix you will have to modify the crontab file.
Write in the Unix terminal
crontab -e
in this new file , as explained in this post you should add the periodicity (in my case , for testing purposed, I wanted to run it every minute to check it is working.
* * * * * /backupDbRub.sh
Think that the copies will appear in the folder you choose in the $querywrite sentence. So you can find them there.
In case you want to test that the script works, you must go to the folder where this script and run
/backupDbRub.sh
This should print the 'echo' sentence called in your file. Sometimes you did not provide permissions in this file, so you should add permissions with 'chmod -x', or whatever other you need to set up.
For backup of SQLite database, I went through https://www.sqlite.org/backup.html
I came to know that there is one python wrapper over these SQLite online backup APIs.So, I went through https://github.com/husio/python-sqlite3-backup
Here are some doubts that I have about sqlitebck(python package)
I checked out the code in tests, it shows db is copied from :memory: to file and vice versa using sqlitebck.copy(:memory:,dbfile), I am confused about ":memory:" and its use.
Instead from memory, can I copy one database file to other database file like sqlitebck(dbfile1, dbfile2). so that dbfile2 will be the backup of dbfile1?
':memory:' as a "filename" is how you tell sqlite to keep a small DB in memory rather than on-disk. But yes, sqlitebck is fine to copy from file to file -- though the arguments it takes are sqlite connections so you'd need to sqlite3.connect to each file first (and usually might as well just copy the file directly w/o involving sqlite -- as the sqlite page you link to implies, suggesting Unix cp or Windows copy... Python has its own standard library module for file copies, https://docs.python.org/2/library/shutil.html ).
In db.py,I can use a function(func insert) insert data into sqlite correctly.
Now I want to insert data into sqlite through python-fastcgi, in
fastcgi (just named post.py ) I can get the request data correctly,but
when I call db.insert,it gives me internal server error.
I already did chmod 777 slqite.db. Anyone know whats problem?
Ffinally I found the answer:
the sqlite3 library needs write permissions also on the directory that contains it, probably because it needs to create a lockfile.
Therefor when I use sql to insert data there is no problem, but when I do it through web cgi,fastcgi etc）to insert data there would be an error.
Just add write permission to the directory.
I need to copy an existing neo4j database in Python. I even do not need it for backup, just to play around with while keeping the original database untouched. However, there is nothing about copy/backup operations in neo4j.py documentation (I am using python embedded binding).
Can I just copy the whole folder with the original neo4j database to a folder with a new name?
Or is there any special method available in neo4j.py?
Yes,
you can copy the whole DB directory when you have cleanly shut down the DB for backup.
This question already has answers here:
Moving back and forth between an on-disk database and a fast in-memory database?
(2 answers)
Closed 7 years ago.
What I need to do is load an sqlite3 database from the disk into memory, work with it, and when the script exits save the in-memory database to the disk. How would I go about doing this? Thanks!
All you need to do is connect to the database - you can do anything you want to with it then.
from sqlite3 import connect
conn = connect("/path/to/your/sqlite.db")
# Do what you need to with the database here
# Changes (inserts, updates, etc) will be persisted
# to disk whenever you commit a transaction.
If you need to be able to run a series of commands and roll them all back if one does not succeed you should use a transaction.
If you need to make a copy of the existing database, make alterations to it and then save it somewhere else only if the alterations succeed then you can do one of the following things:
Create an in-memory database and then use SQLite's ATTATCH DATABASE command to attach the existing database to the new in-memory database. Once you have copied all of the data over from the existing database into the in-memory database and transformed it you can attach a new database and copy everything over again.
Copy the existing database file, attempt to make alterations to it, and delete the new file if the operations did not succeed.
Use sqlite3's Connection.iterdump to get the contents of the tables as a SQL script which you can then pass to sqlite3.Cursor.executescript as this answer suggests (using either an in-memory or an on-disk connection string for your new database).