I run into a problem w. selecting from a encrypted column from a table using psycopg2.
After having created a testtable using
create table users (
id BIGSERIAL NOT NULL PRIMARY KEY,
first_name VARCHAR(100) NOT NULL,
last_name VARCHAR(100) NOT NULL,
secret_val_1 BYTEA,
secret_val_2 BYTEA
);
I was able to insert encrypted values into it.
Now I am trying to query values from the table using psycopg2 with:
cur.execute("""
SELECT PGP_SYM_DECRYPT(%s::BYTEA, 'compress-algo=1, cipher-algo=aes256')
FROM users;
""",
('secret_val_1',))
Now this raises an error:
ExternalRoutineInvocationException: Wrong key or corrupt data
Interestingly, when passing the values like so, it works:
def query_users_decrypt(col):
cur.execute("""
SELECT PGP_SYM_DECRYPT({}::BYTEA, 'compress-algo=1, cipher-
algo=aes256') FROM users;
""".format(col),
(col,))
But this is not secure for sql-injection attacks right?
Does anyone know how how to do this right? Thanks!
The format() works because when you pass the secret_val_1 in, it ends up looking like:
SELECT PGP_SYM_DECRYPT(secret_val_1::BYTEA, 'compress-algo=1, cipher-algo=aes256')
FROM users;
What you are looking for is just the straight query:
select pgp_sym_decrypt(secret_val_1, 'compress-algo=1, cipher-algo=aes256')
from users;
The parameter binding is meant for when you want to pass in a value to be used by your query. The secret_val_1 is not a value as it is the name of a column.
Use parameter binding for something like this:
cur.execute("""select pgp_sym_decrypt(secret_val_1, 'compress-algo=1, cipher-algo=aes256'
from users
where username = %s""", ('joeuser',))
Related
I'm basically building a secured online diary application with Flask. However my Python source code returns a syntax error when I try to test the app. I can't detect what's wrong with the syntax. Your help will be appreciated.
I'm attaching a screenshot of the error. And here's my SQL database's schema:
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
username TEXT NOT NULL,
hash TEXT NOT NULL
);
CREATE TABLE sqlite_sequence(name,seq);
CREATE UNIQUE INDEX username ON users (username);
CREATE TABLE diaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
name TEXT NOT NULL,
time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
title TEXT NOT NULL,
description TEXT NOT NULL,
img_url TEXT,
FOREIGN KEY(user_id) REFERENCES users(id)
);
New error: unsupported value
It is INSERT statement that causes error.
Well, not the insert itself but the way you're using it.
Values should be passed as a tuple (values between "(" and ")")
So, you need to update db.execute line with something like that
db.execute("insert into table_name(col1, col2) values(?, ?)", (col1_val, col2_val))
UPD. regarding the error on second screenshot.
db.execute("Select...) does not return a value but a set of values.
So, you might wanted to use fetchone() as in docs
res = cur.execute('SELECT count(rowid) FROM stocks') # gets you set records
print(res.fetchone()) # get first record
Anyway, check the docs I provided you link to with.
I am trying to create a database using python to execute the SQL commands (for CS50x problem set 7).
I have created a table with an id field set to AUTO_INCREMENT, but the field in the database is populated only by NULL values. I just want it to have an incrementing id starting at 1.
I've tried searching online to see if I'm using the right syntax and can't find anything obvious, nor can I find someone else with a similar problem, so any help would be much appreciated.
Here is the SQL command I am running:
# For creating the table
db.execute("""
CREATE TABLE students (
id INTEGER AUTO_INCREMENT PRIMARY KEY,
first_name VARCHAR(255) NOT NULL,
middle_name VARCHAR(255) DEFAULT (NULL),
last_name VARCHAR(255) NOT NULL,
house VARCHAR(10),
birth INTEGER
);
""")
# An example insert statement
db.execute("""
INSERT INTO students (
first_name,
middle_name,
last_name,
house,
birth
)
VALUES (
?, ?, ?, ?, ?
);
""", "Harry", "James", "Potter", "Gryffindor", 1980)
Here is a screenshot of the database schema shown in phpliteadmin :
And here is a screenshot of the resulting database:
My guess is that you are using SQLite with phpliteadmin and not MySql, in which case this:
id INTEGER AUTO_INCREMENT PRIMARY KEY
is not the correct definition of the auto increment primary key.
In fact, the data type of this column is set to INTEGER AUTO_INCREMENT, as you can see in phpliteadmin, which according to 3.1. Determination Of Column Affinity, has INTEGER affinity.
Nevertheless it is the PRIMARY KEY of the table but this allows NULL values.
The correct syntax to have an integer primary key is this:
id INTEGER PRIMARY KEY AUTOINCREMENT
This cannot happen, if your statements are executed correctly.
I notice that you are not checking for errors in your code. You should be doing that!
My guess is that the table is already created without the auto_increment attribute. The create table is generating an error and you are inserting into the older version.
You can fix this by dropping the table before you create it. You should also modify the code to check for errors.
I'm having a little trouble querying my sqlite DB. Has anyone seen this before? It may very well be that I am doing something silly
Here we have the schema:
BEGIN;
CREATE TABLE "homepage_user" (
"username" varchar(20) NOT NULL PRIMARY KEY,
"first_name" varchar(20) NOT NULL,
"last_name" varchar(20) NOT NULL
)
;
CREATE TABLE "homepage_oncall" (
"id" integer NOT NULL PRIMARY KEY,
"user_id" varchar(20) NOT NULL REFERENCES "homepage_user" ("username"),
"start_date" datetime,
"end_date" datetime
)
;
COMMIT;
And then when I query that data from django's interactive shell:
...
>>> cursor.execute('SELECT user_id from homepage_oncall').fetchall()
[(u'amencke',), (u'jdoe',), (u'jbloggs',)]
However - any attempt to query 'homepage_oncall.user_id' will be met with a stern rebuke:
>>> query = cursor.execute('SELECT first_name, last_name FROM homepage_user WHERE homepage_user.username = homepage_oncall.user_id AND NOW() BETWEEN homepage_oncall.start_date AND homepage_oncall.end_date')
...
DatabaseError: no such column: homepage_oncall.user_id
I had a look around on here and it seems this issue surfaces after editing a schema after it was created - but (after failing to solve the problem with 'manage.py migrate' I actually deleted and recreated the entire database to rule that out - there was nothing in it anyway
Thanks,
Arthur
Your problem is in your SQL. You SELECT from one table and try to apply a WHERE condition on another table without performing an implicit or explicit JOIN.
Also Django provides a pretty nice ORM, so I would recommend sticking to it instead of running raw queries against the database especially if you are not well versed with writing SQL queries
Here is my Mysql table schema
Table: booking
Columns:
id int(11) PK AI
apt_id varchar(200)
checkin_date date
checkout_date date
price decimal(10,0)
deposit decimal(10,0)
adults int(11)
source_id int(11)
confirmationCode varchar(100)
client_id int(11)
booking_date datetime
note mediumtext
Related Tables:property (apt_id → apt_id)
booking_source (source_id → id)
I am trying to insert the value using python .so Here what I have done
sql = "INSERT INTO `nycaptBS`.`booking` (`apt_id`, `checkin_date`, `checkout_date`, `price`,`deposite` `adults`, `source_id`, `confirmationCode`, `client_id`, `booking_date`) VALUES ('%s','%s','%s','%s','%s','%d','%d','%s','%d','%s' )" % (self.apt_id,self.start_at,self.end_at,self.final_price,self.deposit,self.adults,self.source_id,self.notes,self.client_id,self.booking_date,self.notes)
x.execute(sql)
But while executing the above script I am getting the error .
sql = "INSERT INTO `nycaptBS`.`booking` (`apt_id`, `checkin_date`, `checkout_date`, `price`,`deposite` `adults`, `source_id`, `confirmationCode`, `client_id`, `booking_date`) VALUES ('%s','%s','%s','%s','%s','%d','%d','%s','%d','%s' )" % (self.apt_id,self.start_at,self.end_at,self.final_price,self.deposit,self.adults,self.source_id,self.notes,self.client_id,self.booking_date,self.notes)
TypeError: %d format: a number is required, not NoneType
I think my strings formatter are not correct Please help me out .
it looks like either booking_date, notes, source_id, (also you are inserting notes value 2x?)
is None. You could check/validate each value before inserting.
Also please use parameterized queries, NOT string formatting
Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack (see http://xkcd.com/327/ for
humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method.
something like:
x.execute("INSERT INTO thing (test_one, test_two) VALUES (?, ?)", (python_var_one, python_var_two,))
I have a simple table in mysql with the following fields:
id -- Primary key, int, autoincrement
name -- varchar(50)
description -- varchar(256)
Using MySQLdb, a python module, I want to insert a name and description into the table, and get back the id.
In pseudocode:
db = MySQLdb.connection(...)
queryString = "INSERT into tablename (name, description) VALUES" % (a_name, a_desc);"
db.execute(queryString);
newID = ???
I think it might be
newID = db.insert_id()
Edit by Original Poster
Turns out, in the version of MySQLdb that I am using (1.2.2)
You would do the following:
conn = MySQLdb(host...)
c = conn.cursor()
c.execute("INSERT INTO...")
newID = c.lastrowid
I am leaving this as the correct answer, since it got me pointed in the right direction.
I don't know if there's a MySQLdb specific API for this, but in general you can obtain the last inserted id by SELECTing LAST_INSERT_ID()
It is on a per-connection basis, so you don't risk race conditions if some other client performs an insert as well.
You could also do a
conn.insert_id
The easiest way of all is to wrap your insert with a select count query into a single stored procedure and call that in your code. You would pass in the parameters needed to the stored procedure and it would then select your row count.