I'm having a little trouble querying my sqlite DB. Has anyone seen this before? It may very well be that I am doing something silly
Here we have the schema:
BEGIN;
CREATE TABLE "homepage_user" (
"username" varchar(20) NOT NULL PRIMARY KEY,
"first_name" varchar(20) NOT NULL,
"last_name" varchar(20) NOT NULL
)
;
CREATE TABLE "homepage_oncall" (
"id" integer NOT NULL PRIMARY KEY,
"user_id" varchar(20) NOT NULL REFERENCES "homepage_user" ("username"),
"start_date" datetime,
"end_date" datetime
)
;
COMMIT;
And then when I query that data from django's interactive shell:
...
>>> cursor.execute('SELECT user_id from homepage_oncall').fetchall()
[(u'amencke',), (u'jdoe',), (u'jbloggs',)]
However - any attempt to query 'homepage_oncall.user_id' will be met with a stern rebuke:
>>> query = cursor.execute('SELECT first_name, last_name FROM homepage_user WHERE homepage_user.username = homepage_oncall.user_id AND NOW() BETWEEN homepage_oncall.start_date AND homepage_oncall.end_date')
...
DatabaseError: no such column: homepage_oncall.user_id
I had a look around on here and it seems this issue surfaces after editing a schema after it was created - but (after failing to solve the problem with 'manage.py migrate' I actually deleted and recreated the entire database to rule that out - there was nothing in it anyway
Thanks,
Arthur
Your problem is in your SQL. You SELECT from one table and try to apply a WHERE condition on another table without performing an implicit or explicit JOIN.
Also Django provides a pretty nice ORM, so I would recommend sticking to it instead of running raw queries against the database especially if you are not well versed with writing SQL queries
Related
I'm basically building a secured online diary application with Flask. However my Python source code returns a syntax error when I try to test the app. I can't detect what's wrong with the syntax. Your help will be appreciated.
I'm attaching a screenshot of the error. And here's my SQL database's schema:
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
username TEXT NOT NULL,
hash TEXT NOT NULL
);
CREATE TABLE sqlite_sequence(name,seq);
CREATE UNIQUE INDEX username ON users (username);
CREATE TABLE diaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
name TEXT NOT NULL,
time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
title TEXT NOT NULL,
description TEXT NOT NULL,
img_url TEXT,
FOREIGN KEY(user_id) REFERENCES users(id)
);
New error: unsupported value
It is INSERT statement that causes error.
Well, not the insert itself but the way you're using it.
Values should be passed as a tuple (values between "(" and ")")
So, you need to update db.execute line with something like that
db.execute("insert into table_name(col1, col2) values(?, ?)", (col1_val, col2_val))
UPD. regarding the error on second screenshot.
db.execute("Select...) does not return a value but a set of values.
So, you might wanted to use fetchone() as in docs
res = cur.execute('SELECT count(rowid) FROM stocks') # gets you set records
print(res.fetchone()) # get first record
Anyway, check the docs I provided you link to with.
I am trying to create a database using python to execute the SQL commands (for CS50x problem set 7).
I have created a table with an id field set to AUTO_INCREMENT, but the field in the database is populated only by NULL values. I just want it to have an incrementing id starting at 1.
I've tried searching online to see if I'm using the right syntax and can't find anything obvious, nor can I find someone else with a similar problem, so any help would be much appreciated.
Here is the SQL command I am running:
# For creating the table
db.execute("""
CREATE TABLE students (
id INTEGER AUTO_INCREMENT PRIMARY KEY,
first_name VARCHAR(255) NOT NULL,
middle_name VARCHAR(255) DEFAULT (NULL),
last_name VARCHAR(255) NOT NULL,
house VARCHAR(10),
birth INTEGER
);
""")
# An example insert statement
db.execute("""
INSERT INTO students (
first_name,
middle_name,
last_name,
house,
birth
)
VALUES (
?, ?, ?, ?, ?
);
""", "Harry", "James", "Potter", "Gryffindor", 1980)
Here is a screenshot of the database schema shown in phpliteadmin :
And here is a screenshot of the resulting database:
My guess is that you are using SQLite with phpliteadmin and not MySql, in which case this:
id INTEGER AUTO_INCREMENT PRIMARY KEY
is not the correct definition of the auto increment primary key.
In fact, the data type of this column is set to INTEGER AUTO_INCREMENT, as you can see in phpliteadmin, which according to 3.1. Determination Of Column Affinity, has INTEGER affinity.
Nevertheless it is the PRIMARY KEY of the table but this allows NULL values.
The correct syntax to have an integer primary key is this:
id INTEGER PRIMARY KEY AUTOINCREMENT
This cannot happen, if your statements are executed correctly.
I notice that you are not checking for errors in your code. You should be doing that!
My guess is that the table is already created without the auto_increment attribute. The create table is generating an error and you are inserting into the older version.
You can fix this by dropping the table before you create it. You should also modify the code to check for errors.
I run into a problem w. selecting from a encrypted column from a table using psycopg2.
After having created a testtable using
create table users (
id BIGSERIAL NOT NULL PRIMARY KEY,
first_name VARCHAR(100) NOT NULL,
last_name VARCHAR(100) NOT NULL,
secret_val_1 BYTEA,
secret_val_2 BYTEA
);
I was able to insert encrypted values into it.
Now I am trying to query values from the table using psycopg2 with:
cur.execute("""
SELECT PGP_SYM_DECRYPT(%s::BYTEA, 'compress-algo=1, cipher-algo=aes256')
FROM users;
""",
('secret_val_1',))
Now this raises an error:
ExternalRoutineInvocationException: Wrong key or corrupt data
Interestingly, when passing the values like so, it works:
def query_users_decrypt(col):
cur.execute("""
SELECT PGP_SYM_DECRYPT({}::BYTEA, 'compress-algo=1, cipher-
algo=aes256') FROM users;
""".format(col),
(col,))
But this is not secure for sql-injection attacks right?
Does anyone know how how to do this right? Thanks!
The format() works because when you pass the secret_val_1 in, it ends up looking like:
SELECT PGP_SYM_DECRYPT(secret_val_1::BYTEA, 'compress-algo=1, cipher-algo=aes256')
FROM users;
What you are looking for is just the straight query:
select pgp_sym_decrypt(secret_val_1, 'compress-algo=1, cipher-algo=aes256')
from users;
The parameter binding is meant for when you want to pass in a value to be used by your query. The secret_val_1 is not a value as it is the name of a column.
Use parameter binding for something like this:
cur.execute("""select pgp_sym_decrypt(secret_val_1, 'compress-algo=1, cipher-algo=aes256'
from users
where username = %s""", ('joeuser',))
We're trying to migrate a large table (bigger than local memory space) from a postgresql database to a local sqlite3 database using sqlalchemy, but we're getting a syntax error when trying to create the table in the sqlite3 database. Reading through the other posts, the recommended fixes don't seem to apply to this error. Is there some usage detail we're missing here?
# reflect table from remote postgresql database
read_engine = create_engine('postgresql://dbusername:dbuserpassword#server/dbname')
meta = MetaData()
raw = Table('db_table', meta, autoload=True, autoload_with=read_engine)
# write table to local sqlite3 database
write_engine = create_engine('sqlite:///db_local.sqlite3')
table.create(write_engine)
This is the error we get:
OperationalError: (sqlite3.OperationalError) near "(": syntax error
[SQL:
CREATE TABLE db_table (
id INTEGER DEFAULT nextval('db_table_id_seq'::regclass) NOT NULL,
created_at TIMESTAMP,
updated_at TIMESTAMP,
data TEXT,
device_key TEXT,
format TEXT,
uuid TEXT,
CONSTRAINT db_table_pkey PRIMARY KEY (id)
)
]
I think the issue is that SQLite doesn't support DEFAULT nextval(...) -- nextval() is a Postgres-specific function that isn't part of the SQLite database.
You'll need to re-write the CREATE TABLE statement to use AUTOINCREMENT, I believe.
Disclosure: I am an EnterpriseDB (EDB) employee
I am trying to add an 'id' primary key column to an already existing MySQL table using alembic. I tried the following...
op.add_column('mytable', sa.Column('id', sa.Integer(), nullable=False))
op.alter_column('mytable', 'id', autoincrement=True, existing_type=sa.Integer(), existing_server_default=False, existing_nullable=False)
but got the following error
sqlalchemy.exc.OperationalError: (OperationalError) (1075, 'Incorrect table definition; there can be only one auto column and it must be defined as a key') 'ALTER TABLE mytable CHANGE id id INTEGER NOT NULL AUTO_INCREMENT' ()
looks like the sql statement generated by alembic did not add PRIMARY KEY at the end of the alter statement. Could I have missed some settings?
Thanks in advance!
I spent some time digging through the alembic source code, and this doesn't seem to be supported. You can specify primary keys when creating a table, but not when adding columns. In fact, it specifically checks and won't let you (link to source):
# from alembic.operations.toimpl.add_column, line 132
for constraint in t.constraints:
if not isinstance(constraint, sa_schema.PrimaryKeyConstraint):
operations.impl.add_constraint(constraint)
I looked around, and adding a primary key to an existing table may result in unspecified behavior - primary keys aren't supposed to be null, so your engine may or may not create primary keys for existing rows. See this SO discussion for more info: Insert auto increment primary key to existing table
I'd just run the alter query directly, and create primary keys if you need to.
op.execute("ALTER TABLE mytable ADD id INT PRIMARY KEY AUTO_INCREMENT;")
If you really need cross-engine compatibility, the big hammer would be to (1) create a new table identical to the old one with a primary key, (2) migrate all your data, (3)delete the old table and (4) rename the new table.
Hope that helps.
You have to remove the primary key that is in the table and then create a new one that includes all columns that you want as the primary key.
eg. In psql use \d <table name> to define the schema, then check the primary key constraint.
Indexes:
"enrollments_pkey" PRIMARY KEY, btree (se_crs_id, se_std_id)
then use this information in alembic
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('enrollments', sa.Column(
'se_semester', sa.String(length=30), nullable=False))
op.drop_constraint('enrollments_pkey', 'enrollments', type_='primary')
op.create_primary_key('enrollments_pkey', 'enrollments', [
'se_std_id', 'se_crs_id', 'se_semester'])
The results after running \d enrollments should be updated to
Indexes:
"enrollments_pkey" PRIMARY KEY, btree (se_std_id, se_crs_id, se_semester)
This solution worked fine for me.