I'm creating a small python program and i use the ORM peewee to manage my Sqlite database. I want to create a table without primary key using peewee. The problem is that peewee is adding a primary key if i dont specify one. After read the docs, i found the parameter without_rowid supposed to prevent peewee to create this primary key. But it doesnt works.
Here is a small example of code :
#!/usr/bin/python
import peewee
import traceback
db_proxy = peewee.Proxy()
class BaseModel(peewee.Model):
class Meta:
database = db_proxy
class table1(BaseModel):
id = peewee.IntegerField(primary_key=True)
field1 = peewee.CharField()
field2 = peewee.IntegerField()
class table2(BaseModel):
table1_id = peewee.IntegerField()
field1 = peewee.CharField()
class Meta:
without_rowid = True
try:
# create database
db = peewee.SqliteDatabase("test.db")
db_proxy.initialize(db)
db.create_tables([table1, table2])
except Exception as e:
traceback.print_exc(e)
Same with the without_rowid, peewee automatically add an id primary key to my table2 table. Here is the schema of the created database :
CREATE TABLE IF NOT EXISTS "table1"(
"id" INTEGER NOT NULL PRIMARY KEY,
"field1" VARCHAR(255) NOT NULL,
"field2" INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS "table2"(
"id" INTEGER NOT NULL PRIMARY KEY,
"table1_id" INTEGER NOT NULL,
"field1" VARCHAR(255) NOT NULL
) WITHOUT ROWID;
Do you know a way to solve this problem and permit me to create a table without rowid ?
ps : if you're asking why i want to create a table without primary key, it's because i need to insert data from a csv file. (sqlite3 database.db ".mode csv" ".import file.csv table1"). As i have an AUTO INCREMENT PRIAMRY KEY on my table, i need to trick a little bit by importing the csv file in a temporary table without primary key as explained here : http://objectiveme.com/populating-a-sqlite-database-using-csv/
Thx ! :)
Peewee documentation is also VERY CLEAR about how to disable primary key:
http://docs.peewee-orm.com/en/latest/peewee/models.html#models-without-a-primary-key
Please READ THE DOCS.
To create a Peewee model without a primary key (which is distinctly different from "WITHOUT ROWID"), you can specify "primary_key = False":
class NoPKModel(Model):
key = TextField()
data = TextField()
class Meta:
primary_key = False
This will not automatically create an "id" field.
Without ROWID is a SQLite optimization with rather specific use-case. Please read the SQLite documentation and understand the SQLite data-model before using it: https://www.sqlite.org/rowidtable.html
Related
I am importing a CSV file to postgres and there is no unique column in the dataset. I want to add a serial ID field to uniquely identify each record as its inserted into the table.
I have created a sequence and added an ID field to the table structure before triggering the import:
CREATE SEQUENCE IF NOT EXISTS serial;
CREATE TABLE my_tbl (
fname varchar(100),
lname varchar(100),
company varchar(200),
id integer PRIMARY KEY DEFAULT nextval('serial')
);
I run this code to import the CSV which has data for fname, lname and company:
conn = psycopg2.connect(dbname=dbname, host=host, port=port, user=user, password=pwd)
cur = conn.cursor()
cur.copy_expert("copy {} from STDIN CSV HEADER QUOTE '\"'".format(table_name), file)
cur.execute("commit;")
However, I get an error saying I'm missing data for field "id". I assume under the hood psycopg2 is matching the schemas of the CSV and PG table to validate the COPY before it attempts the insert. A regular insert would succeed as the id field would be populated with a value from the SEQ.
How can I add a unique id field to each record that's copied from the CSV to the PG table?
You have two options. You can specify columns of the target table in the COPY command, e.g.:
COPY my_tbl(fname, lname, company) FROM STDIN CSV HEADER QUOTE '"'
Alternatively, create the table without the id primary key, import the csv data and only then add the primary key:
ALTER TABLE my_tbl ADD id serial PRIMARY KEY;
Not related. You do not have to create a sequence for a serial column, let Postgres do this for you:
CREATE TABLE my_tbl (
fname varchar(100),
lname varchar(100),
company varchar(200),
id serial PRIMARY KEY
);
Then the system knows the relationship between the table and the sequence. (Also, serial is not the best name for a sequence, how do you name the next one when you need it?)
Please excuse any terminology typos, don't have a lot of experience with databases other than SQLite. I'm trying to replicate what I would do in SQLite where I could ATTACH a database to a second database and query across all the tables. I wasn't using SQLAlchemy with SQLite
I'm working with SQLAlchemy 1.0.13, Postgres 9.5 and Python 3.5.2 (using Anaconda) on Win7/54. I have connected two databases (on localhost) using postgres_fdw and imported a few of the tables from the secondary database. I can successfully manually query the connected table with SQL in PgAdminIII and from Python using psycopg2. With SQLAlchemy I've tried:
# Same connection string info that psycopg2 used
engine = create_engine(conn_str, echo=True)
class TestTable(Base):
__table__ = Table('test_table', Base.metadata,
autoload=True, autoload_with=engine)
# Added this when I got the error the first time
# test_id is a primary key in the secondary table
Column('test_id', Integer, primary_key=True)
and get the error:
sqlalchemy.exc.ArgumentError: Mapper Mapper|TestTable|test_table could not
assemble any primary key columns for mapped table 'test_table'
Then I tried:
insp = reflection.Inspector.from_engine(engine)
print(insp.get_table_names())
and the attached tables aren't listed (the tables from the primary database do show up). Is there a way to do what I am trying to accomplish?
In order to map a table SQLAlchemy needs there to be at least one column denoted as a primary key column. This does not mean that the column need actually be a primary key column in the eyes of the database, though it is a good idea. Depending on how you've imported the table from your foreign schema it may not have a representation of a primary key constraint, or any other constraints for that matter. You can work around this by either overriding the reflected primary key column in the Table instance (not in the mapped classes body), or better yet tell the mapper what columns comprise the candidate key:
engine = create_engine(conn_str, echo=True)
test_table = Table('test_table', Base.metadata,
autoload=True, autoload_with=engine)
class TestTable(Base):
__table__ = test_table
__mapper_args__ = {
'primary_key': (test_table.c.test_id, ) # candidate key columns
}
To inspect foreign table names use the PGInspector.get_foreign_table_names() method:
print(insp.get_foreign_table_names())
Building on sibling answer by #ilja.
When using the SQLAlchemy automap feature to automatically generate mapped classes and relationships from an existing database schema, I found that the __mapper_args__ solution didn't create the model.
This alternative method where you manually define the private key will correctly enable automap to create your model.
from sqlalchemy import Column, create_engine, Text
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.schema import Table
Base = automap_base()
engine = create_engine(conn_str, convert_unicode=True)
pk = Column('uid', Text, primary_key=True)
test_table = Table(
'test_table', Base.metadata, pk, autoload=True, autoload_with=engine
)
# Inspect postgres schema
Base.prepare(engine, reflect=True)
print(dict(Base.classes))
print(test_table)
I'm writing a program that creates a sqlite3 database through python. I have one table of Authors (AuthorID, Name) and a second table of books (BookID, Title, AuthorID) I've created these as shown below:
Authors = sqlite3.connect('Authors.db')
Authors.execute('''CREATE TABLE Authors
(AuthorID INT PRIMARY KEY,
Name TEXT);''')
Authors.close()
Books = sqlite3.connect('Books.db')
Books.execute('''CREATE TABLE Books
(BookID INT PRIMARY KEY,
Title TEXT,
AuthorID INT,
FOREIGN KEY(AuthorID) REFERENCES Authors(AuthorID));''')
Books.close()
I then go to add a record to each of the tables as shown below:
Authors = sqlite3.connect('Authors.db')
Authors.execute("INSERT INTO Authors (AuthorID, Name) \
VALUES (1, 'Jane Austin')");
Authors.commit()
Authors.close()
Books = sqlite3.connect('Books.db')
Books.execute("INSERT INTO Books (BookID, Title, AuthorID) \
VALUES (1, 'Emma', 1)");
Books.commit()
Books.close()
The database is correctly updated but I don't think the foreign keys are working correctly because it allows me to remove the Author 'Jane Austin', when there are books associated with it.
I've seen some tutorials use this line:
Books.execute("PRAGMA foreign_keys = 1")
Is this the answer to the problem and if so where do I put this line?
The PRAGMA foreign_keys setting applies to a connection, so you should execute it immediately after calling sqlite3.connect().
Please note that foreign key constraints work only inside the same database; you should put both tables into the same file.
So to do what you want to do you need to create one database file with 2 tables.
Example:
conn=sqlite3.connect("clientdatabase.db")
conn.execute("PRAGMA foreign_keys = 1")
cur=conn.cursor()
# Create 2 tables if they don't exist: Clients and Work_Done
cur.execute('''CREATE TABLE IF NOT EXISTS Clients
(CID INTEGER PRIMARY KEY,
First_Name TEXT NOT NULL,
Last_Name TEXT,
Business_Name TEXT,
Phone TEXT,
Address TEXT,
City TEXT,
Notes TEXT,
Active_Status TEXT NOT NULL)''')
cur.execute('''CREATE TABLE IF NOT EXISTS Work_Done
(ID INTEGER PRIMARY KEY,
Date TEXT NOT NULL,
Onsite_Contact TEXT,
Work_Done TEXT NOT NULL,
Parts_Installed TEXT,
Next_Steps TEXT,
CID INT,
FOREIGN KEY (CID) REFERENCES CLIENTS (CID))''')
conn.commit()
Note that both tables are in the same database and you add the line after connection and before the cursor object.
Hope this helps.
Also note that if there is an active transaction, the PRAGMA foreign_keys does not work. There is no error message if you try to do so but foreign keys will still be turned off.
If you have problems with foreign keys even after using the pragma, it may be worth an attempt to execute COMMIT once before using it.
FYI, according to the recent official document, you can use PRAGMA foreign_keys = ON.
connection = sqlite3.connect(DB_FILE)
connection.execute('PRAGMA foreign_keys = ON')
cursor = connection.cursor()
(...)
I have created 3 tables like the following
----------------Database name is chiledata.sqlite---------------
CREATE TABLE child (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT
);
----------------Database name is dogdata.sqlite---------------
CREATE TABLE dog (
id INTEGER PRIMARY KEY AUTOINCREMENT,
dog TEXT
);
----------------Database name is dogChilddata.sqlite---------------
CREATE TABLE child_dog {
child_id INTEGER,
dog_id INTEGER,
FOREIGN KEY(child_id) REFERENCES child(id),
FOREIGN KEY(dog_id) REFERENCES dog(id)
};
If I use python for make relationship between these tables and execute "SELECT" query how can I connect these 3 tables for this task?
ex:
#Import the sqlite3 module
import sqlite3
# Create a connection and cursor to your database
conn = sqlite3.connect('chiledata.sqlite')
c = conn.cursor()
So I can connect chiledata.sqlite but how can I connect other two database tables (dogdata.sqlite, dogChilddata.sqlite) to execute SELECT query?
Don't use separate database files if your tables are supposed to be related.
Store all your tables in one database file; connect to it, create the tables with that one connection only.
I'm trying to set up a database ORM with peewee and am not clear on the use of foreign key relationships.
from peewee import *
db = SqliteDatabase('datab.db')
class datab(Model):
class Meta:
database = db
class Collection(datab):
identifier = CharField()
title = CharField()
class File(datab):
identifier = ForeignKeyField(Collection, related_name='files')
name = CharField()
Later, I do an import of "Collections"
for value in collection:
Collection(**value).save()
Finally, where I am having trouble is adding the Files to the collections
for value in collectionFiles:
File(**value).save()
Within the value dict, there is a keyword pair with key of "identifier" and a value that should associate with the Collection identifier keyword.
However I get an error message:
ValueError: invalid literal for int() with base 10: 'somevalue'
If I change the File(datab): identifier Type to VarChar, it saves the data.
I'm realizing I'm doing it wrong. My assumption was that the unique identifier value in each table would apply the foreign key. After reading the documentation, it looks like the foreign key setup is a bit different. Do I need to do something like
Collections.File.files(**values).save() ? In other words, instead of doing a data import, loading the collection object and then adding the file associated fields through peewee?
Values that make up class File
{'crc32': '63bee49d',
'format': 'Metadata',
'identifier': u'somevalue',
'md5': '34104ffce9e4084fd3641d0decad910a',
'mtime': '1368328224',
'name': 'lupi.jpg_meta.txt',
'sha1': '1448ed1159a5d770da76067dd1c53e94d5a01535',
'size': '1244'}
I think the naming of your fields might be part of the confusion. Rather than calling the foreign key from File -> Collection "identifier", you might call it "collection" instead.
class File(datab):
collection = ForeignKeyField(Collection, related_name='files')
name = CharField()
Peewee prefers that, when setting the value of a Foreign Key, it be a model instance. For example, rather than doing:
File.create(collection='foobar', name='/secret/password')
It is preferable to do something like this:
collection = Collection.get(Collection.identifier == 'foobar')
File.create(collection=collection, name='/secret/password')
As a final note, if the Collection "identifier" is the unique primary key, you can set it up thus:
class Collection(datab):
identifier = CharField(primary_key=True)
title = CharField()
(I'm not familiar with peewee, but if it's like Django then this should work.)
class File has a ForeignKeyField and a CharField, so you can't simply save a pair of strings with File(**value). You need to convert a string to a key first, like this:
for value in collectionFiles:
identifier = value['identifier']
name = value['name']
collection_entity = Collection.objects.filter(identifier=identifier).get()
File(identifier=collection_entity, name=name).save()