Develop Reference

Why does Sqlite create a new table that I have not specified within my create_tables.py file

So I have a file called create_tables.py with the following code:
import sqlite3
connection = sqlite3.connect("mydata.db")
cursor = connection.cursor()
create_table = "CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, username text, password text)"
cursor.execute(create_table)
create_table = "CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name text, price real)"
cursor.execute(create_table)
connection.commit()
connection.close()
When trying to make a GET request via postman I am provided this error:
sqlite3.OperationalError: no such table: items
But there is a table that has been created called item_model.
I didn't specify the creation of this table. I am wondering why my items table is not being created but this item_model table was.

Create a table without rowid with peewee

I'm creating a small python program and i use the ORM peewee to manage my Sqlite database. I want to create a table without primary key using peewee. The problem is that peewee is adding a primary key if i dont specify one. After read the docs, i found the parameter without_rowid supposed to prevent peewee to create this primary key. But it doesnt works.
Here is a small example of code :
#!/usr/bin/python
import peewee
import traceback
db_proxy = peewee.Proxy()
class BaseModel(peewee.Model):
class Meta:
database = db_proxy
class table1(BaseModel):
id = peewee.IntegerField(primary_key=True)
field1 = peewee.CharField()
field2 = peewee.IntegerField()
class table2(BaseModel):
table1_id = peewee.IntegerField()
field1 = peewee.CharField()
class Meta:
without_rowid = True
try:
# create database
db = peewee.SqliteDatabase("test.db")
db_proxy.initialize(db)
db.create_tables([table1, table2])
except Exception as e:
traceback.print_exc(e)
Same with the without_rowid, peewee automatically add an id primary key to my table2 table. Here is the schema of the created database :
CREATE TABLE IF NOT EXISTS "table1"(
"id" INTEGER NOT NULL PRIMARY KEY,
"field1" VARCHAR(255) NOT NULL,
"field2" INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS "table2"(
"id" INTEGER NOT NULL PRIMARY KEY,
"table1_id" INTEGER NOT NULL,
"field1" VARCHAR(255) NOT NULL
) WITHOUT ROWID;
Do you know a way to solve this problem and permit me to create a table without rowid ?
ps : if you're asking why i want to create a table without primary key, it's because i need to insert data from a csv file. (sqlite3 database.db ".mode csv" ".import file.csv table1"). As i have an AUTO INCREMENT PRIAMRY KEY on my table, i need to trick a little bit by importing the csv file in a temporary table without primary key as explained here : http://objectiveme.com/populating-a-sqlite-database-using-csv/
Thx ! :)

Peewee documentation is also VERY CLEAR about how to disable primary key:
http://docs.peewee-orm.com/en/latest/peewee/models.html#models-without-a-primary-key
Please READ THE DOCS.
To create a Peewee model without a primary key (which is distinctly different from "WITHOUT ROWID"), you can specify "primary_key = False":
class NoPKModel(Model):
key = TextField()
data = TextField()
class Meta:
primary_key = False
This will not automatically create an "id" field.
Without ROWID is a SQLite optimization with rather specific use-case. Please read the SQLite documentation and understand the SQLite data-model before using it: https://www.sqlite.org/rowidtable.html

psycopg2 - Error missing data for field "id"?

I am importing a CSV file to postgres and there is no unique column in the dataset. I want to add a serial ID field to uniquely identify each record as its inserted into the table.
I have created a sequence and added an ID field to the table structure before triggering the import:
CREATE SEQUENCE IF NOT EXISTS serial;
CREATE TABLE my_tbl (
fname varchar(100),
lname varchar(100),
company varchar(200),
id integer PRIMARY KEY DEFAULT nextval('serial')
);
I run this code to import the CSV which has data for fname, lname and company:
conn = psycopg2.connect(dbname=dbname, host=host, port=port, user=user, password=pwd)
cur = conn.cursor()
cur.copy_expert("copy {} from STDIN CSV HEADER QUOTE '\"'".format(table_name), file)
cur.execute("commit;")
However, I get an error saying I'm missing data for field "id". I assume under the hood psycopg2 is matching the schemas of the CSV and PG table to validate the COPY before it attempts the insert. A regular insert would succeed as the id field would be populated with a value from the SEQ.
How can I add a unique id field to each record that's copied from the CSV to the PG table?

You have two options. You can specify columns of the target table in the COPY command, e.g.:
COPY my_tbl(fname, lname, company) FROM STDIN CSV HEADER QUOTE '"'
Alternatively, create the table without the id primary key, import the csv data and only then add the primary key:
ALTER TABLE my_tbl ADD id serial PRIMARY KEY;
Not related. You do not have to create a sequence for a serial column, let Postgres do this for you:
CREATE TABLE my_tbl (
fname varchar(100),
lname varchar(100),
company varchar(200),
id serial PRIMARY KEY
);
Then the system knows the relationship between the table and the sequence. (Also, serial is not the best name for a sequence, how do you name the next one when you need it?)

SQLAlchemy not finding Postgres table connected with postgres_fdw

Please excuse any terminology typos, don't have a lot of experience with databases other than SQLite. I'm trying to replicate what I would do in SQLite where I could ATTACH a database to a second database and query across all the tables. I wasn't using SQLAlchemy with SQLite
I'm working with SQLAlchemy 1.0.13, Postgres 9.5 and Python 3.5.2 (using Anaconda) on Win7/54. I have connected two databases (on localhost) using postgres_fdw and imported a few of the tables from the secondary database. I can successfully manually query the connected table with SQL in PgAdminIII and from Python using psycopg2. With SQLAlchemy I've tried:
# Same connection string info that psycopg2 used
engine = create_engine(conn_str, echo=True)
class TestTable(Base):
__table__ = Table('test_table', Base.metadata,
autoload=True, autoload_with=engine)
# Added this when I got the error the first time
# test_id is a primary key in the secondary table
Column('test_id', Integer, primary_key=True)
and get the error:
sqlalchemy.exc.ArgumentError: Mapper Mapper|TestTable|test_table could not
assemble any primary key columns for mapped table 'test_table'
Then I tried:
insp = reflection.Inspector.from_engine(engine)
print(insp.get_table_names())
and the attached tables aren't listed (the tables from the primary database do show up). Is there a way to do what I am trying to accomplish?

In order to map a table SQLAlchemy needs there to be at least one column denoted as a primary key column. This does not mean that the column need actually be a primary key column in the eyes of the database, though it is a good idea. Depending on how you've imported the table from your foreign schema it may not have a representation of a primary key constraint, or any other constraints for that matter. You can work around this by either overriding the reflected primary key column in the Table instance (not in the mapped classes body), or better yet tell the mapper what columns comprise the candidate key:
engine = create_engine(conn_str, echo=True)
test_table = Table('test_table', Base.metadata,
autoload=True, autoload_with=engine)
class TestTable(Base):
__table__ = test_table
__mapper_args__ = {
'primary_key': (test_table.c.test_id, ) # candidate key columns
}
To inspect foreign table names use the PGInspector.get_foreign_table_names() method:
print(insp.get_foreign_table_names())

Building on sibling answer by #ilja.
When using the SQLAlchemy automap feature to automatically generate mapped classes and relationships from an existing database schema, I found that the __mapper_args__ solution didn't create the model.
This alternative method where you manually define the private key will correctly enable automap to create your model.
from sqlalchemy import Column, create_engine, Text
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.schema import Table
Base = automap_base()
engine = create_engine(conn_str, convert_unicode=True)
pk = Column('uid', Text, primary_key=True)
test_table = Table(
'test_table', Base.metadata, pk, autoload=True, autoload_with=engine
)
# Inspect postgres schema
Base.prepare(engine, reflect=True)
print(dict(Base.classes))
print(test_table)

Using foreign keys in sqlite3 for Python

I'm writing a program that creates a sqlite3 database through python. I have one table of Authors (AuthorID, Name) and a second table of books (BookID, Title, AuthorID) I've created these as shown below:
Authors = sqlite3.connect('Authors.db')
Authors.execute('''CREATE TABLE Authors
(AuthorID INT PRIMARY KEY,
Name TEXT);''')
Authors.close()
Books = sqlite3.connect('Books.db')
Books.execute('''CREATE TABLE Books
(BookID INT PRIMARY KEY,
Title TEXT,
AuthorID INT,
FOREIGN KEY(AuthorID) REFERENCES Authors(AuthorID));''')
Books.close()
I then go to add a record to each of the tables as shown below:
Authors = sqlite3.connect('Authors.db')
Authors.execute("INSERT INTO Authors (AuthorID, Name) \
VALUES (1, 'Jane Austin')");
Authors.commit()
Authors.close()
Books = sqlite3.connect('Books.db')
Books.execute("INSERT INTO Books (BookID, Title, AuthorID) \
VALUES (1, 'Emma', 1)");
Books.commit()
Books.close()
The database is correctly updated but I don't think the foreign keys are working correctly because it allows me to remove the Author 'Jane Austin', when there are books associated with it.
I've seen some tutorials use this line:
Books.execute("PRAGMA foreign_keys = 1")
Is this the answer to the problem and if so where do I put this line?

The PRAGMA foreign_keys setting applies to a connection, so you should execute it immediately after calling sqlite3.connect().
Please note that foreign key constraints work only inside the same database; you should put both tables into the same file.

So to do what you want to do you need to create one database file with 2 tables.
Example:
conn=sqlite3.connect("clientdatabase.db")
conn.execute("PRAGMA foreign_keys = 1")
cur=conn.cursor()
# Create 2 tables if they don't exist: Clients and Work_Done
cur.execute('''CREATE TABLE IF NOT EXISTS Clients
(CID INTEGER PRIMARY KEY,
First_Name TEXT NOT NULL,
Last_Name TEXT,
Business_Name TEXT,
Phone TEXT,
Address TEXT,
City TEXT,
Notes TEXT,
Active_Status TEXT NOT NULL)''')
cur.execute('''CREATE TABLE IF NOT EXISTS Work_Done
(ID INTEGER PRIMARY KEY,
Date TEXT NOT NULL,
Onsite_Contact TEXT,
Work_Done TEXT NOT NULL,
Parts_Installed TEXT,
Next_Steps TEXT,
CID INT,
FOREIGN KEY (CID) REFERENCES CLIENTS (CID))''')
conn.commit()
Note that both tables are in the same database and you add the line after connection and before the cursor object.
Hope this helps.

Also note that if there is an active transaction, the PRAGMA foreign_keys does not work. There is no error message if you try to do so but foreign keys will still be turned off.
If you have problems with foreign keys even after using the pragma, it may be worth an attempt to execute COMMIT once before using it.

FYI, according to the recent official document, you can use PRAGMA foreign_keys = ON.
connection = sqlite3.connect(DB_FILE)
connection.execute('PRAGMA foreign_keys = ON')
cursor = connection.cursor()
(...)