Develop Reference

Query executes on phpmyadmin, but not in python script

The statement is set-up so that when a record already exists, it doesn't add a record, else, it does.
I've tried changing the query, even though I don't see anything wrong with it.
I've let the script run on python, and print the query it executed. Then I pasted that query in phpmyadmin, where it executed succesfully.
I have also double checked all parameters.
Query (blank params):
INSERT INTO users (uname,pass) SELECT * FROM (SELECT '{}','{}') AS tmp WHERE NOT EXISTS(SELECT uname FROM users WHERE uname = '{}') LIMIT 1;
Query (filled in parameters):
INSERT INTO users (uname,pass) SELECT * FROM (SELECT 'john_doe','password') AS tmp WHERE NOT EXISTS(SELECT uname FROM users WHERE uname = 'john_doe') LIMIT 1;
Python script (the important part)
if action == "add_user":
username = form.getvalue('username')
password = form.getvalue('password')
query = """
INSERT INTO users (uname,pass) SELECT * FROM
(SELECT '{}','{}') AS tmp WHERE NOT EXISTS(SELECT uname FROM users WHERE uname = '{}') LIMIT 1;
""".format(username, password, username)
mycursor.execute(query)
I know a couple of things.
There is nothing wrong with the database connection.
The parameters are not empty (ex. username="john_doe" & password="secret")
The query actually executes in that specific table.
The query seems to add a record and delete it directly afterwards (as AUTO_INCREMENT increases each time, even when the python script executes and doesn't add anything)
A try except doesn't do anything, as mysql.connector.Error doesn't report any error (which is obvious, since the query actually executes succesfully)
phpMyAdmin practical example:
(Removed INSERT INTO part in order to be able to show the resulting tables)
The first time you enter the query (above query as example), it will result in a table with both values as both column names as column values.
Screenshot of table output: http://prntscr.com/nkgaka
Once that result is entered once, next time you will try to insert it, it will simply result in only column names, no values. This means it will insert nothing, as there is nothing to insert as there are no actual values.
Screenshot of table output: http://prntscr.com/nkgbp3
Help is greatly appreciated.

If you want to ensure any field is unique in a table, make that field a PRIMARY KEY or UNIQUE KEY. In this case you want name to be unique.
CREATE UNIQUE INDEX name ON users(name)
With this in place you only need to INSERT. If a duplicate key error occurs the name already exists.
To avoid SQL injection, don't use """SELECT {}""".format. It will be SQL injection vulnerable.
Also don't store plain text passwords, ever. Salted hashes at least. There's plenty of frameworks that do this well already so you don't need to invent your own.

INSERT INTO...NO DUPLICATE KEY clause in python to check new rows added

I try to use INSERT INTO...NO DUPLICATE KEY UPDATE clause in python to update mysql records where name is the primary key. If the name exist, update record's age column, otherwise insert it:
sql = """INSERT INTO mytable(name, age) \
VALUES ('Tim',30),('Sam',21),('John','35') \
ON DUPLICATE KEY UPDATE age=VALUES(age)"""
with db.connection() as conn:
with conn.cursor as cursor:
cursor.execute(sql)
if cursor.rowcount == 0：
result = 'UPDATE'
else:
result = 'INSERT'
I want to find out whether this execution has add one or more new rows or not. But the cursor.rowcount is not correct for each insert and update. Any comments about that?

I ran into this problem before, where I wanted to know if my insert was successful or not. My short-term solution was to call a count(*) on the table before and after the insert and and compare the numbers.
I never found a way to determine which action you have used for both INSERT IGNORE and INSERT ... ON DUPLICATE KEY.

Just to add more clarification to the previous answer.
With a cursor.rowcount is particularly hard to achieve your goal if inserting multiple rows.
The reason is that rowcount returns the number of affected rows.
Here is how it is defined:
The affected-rows value per row is 1 if the row is inserted as a new row, 2 if an existing row is updated, and 0 if an existing row is set to its current values. (https://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html)
So, to solve your problem you will need to do count(*) before insert and after the insert.

Effective batch "update-or-insert" in SqlAlchemy

There exists a table Users and in my code I have a big list of User objects. To insert them I can use :
session.add_all(user_list)
session.commit()
The problem is that there can be several duplicates which I want to update but the database wont allow to insert duplicate entries. For sure, I can iterate over user_list and try to insert user in the database and if it fails - update it :
for u in users:
q = session.query(T).filter(T.fullname==u.fullname).first()
if q:
session.query(T).filter_by(index=q.index).update({column: getattr(u,column) for column in Users.__table__.columns.keys() if column!='id'})
session.commit()
else:
session.add(u)
session.commit()
but I find this solution quiet ineffective : first, I am making several requests to retrieve object q, and instead of batch inserting of new items I insert them one per one. I wonder if there exists a better solution for this task.
UPD better version:
for u in users:
q = session.query(T).filter(Users.fullname==u.fullname).first()
if q:
for column in Users.__table__.columns.keys():
if not column=='index':
setattr(q,column,getattr(u,column))
session.add(q)
else:
session.add(u)
session.commit()

a better solution would be to use
INSERT ... ON DUPLICATE KEY UPDATE ...
bulk MySQL construct (I assume you're using MySQL because your post is tagged with 'mysql'). This way you're both inserting new entries and updating existing ones in one statement / transaction, see http://dev.mysql.com/doc/refman/5.6/en/insert-on-duplicate.html
It's not ideal if you have multiple unique indexes and, depending on your schema, you'll have to fill in all NOT NULL values (hence issuing one bulk SELECT before calling it), but it's definitely the most efficient option and we use it a lot. The bulk version will look something like (let's assume name is a unique key):
INSERT INTO User (name, phone, ...) VALUES
('ksmith', '111-11-11', ...),
('jford', '222-22,22', ...),
...,
ON DUPLICATE KEY UPDATE
phone = VALUES(phone),
... ;
Unfortunately, INSERT ... ON DUPLICATE KEY UPDATE ... is not supported natively by SQLa so you'll have to implement a little helper function which will build the query for you.

How to get the last row id on update?

I am a beginner in mysql and may be its my fault somewhere, and not able to understand how this can be resolved.
This is structure of my table:-
CREATE TABLE `nearest_product_type` (
`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`created` datetime NOT NULL,
`modified` datetime NOT NULL,
`name` varchar(15) NOT NULL UNIQUE
)
;
And this is the code i am trying:-
base = MySQLdb.connect (host="localhost", user = "root", passwd = "sheeshmohsin", db="points")
basecursor = base.cursor()
queryone = """INSERT INTO nearest_product_type (name,created,modified) VALUES (%s,%s,%s) ON DUPLICATE KEY UPDATE name=name """
category = "Indica"
valueone = (category,datetime.datetime.now(),datetime.datetime.now())
basecursor.execute(queryone, valueone)
product_id = basecursor.lastrowid
basecursor.close()
base.commit()
base.close()
print product_id
On running this python script, first time when category is not unique, it works fine, but on running again with the same category as first time, last row id returns 0. but i need the id of the last row which is updated.
And when i checked the rows in table, the auto-increment is also working, suppose if i run the script four times, in first time when category is unique the id is 1 and suppose another unique category comes in fourth time, then the id assigned to this row is 4, but it should be 2, because its second row. how can i solve this?

The ON DUPLICATE KEY UPDATE part here will not work as the key is the auto-increment column, which will never get duplicates.
It is almost certainly this clause that is causing the unexpected counts, particularly given the UNIQUE setting on name.
You can try using something like SELECT MAX(id) FROM nearest_product_type to get the last id added.

Something is wrong in the way you access the database. When you try to insert an new row in your database with a name that already exists, as the column name is declared to be unique, the insert will fail.
If you want to modify an existing row , you must use an UPDATE statement not an INSERT one. And there's nothing in SQL to do an insert or update.
And nothing in autoincrement guarantees that id are consecutive. All you know is that the database will allow a different id for each inserted row, but insertion failure can (and do in you case) result is holes is the id sequence.
Furthermore, some drivers may allow for pre-reservation of ids, particurarly with network connections to allow a client connection to get a bunch of ids in case it would insert more than one row. It that case, if another client asks for ids, and both clients insert rows alternatively, the id will not follow the insertion time.

Stop values from being entered into database python sqlite

How would I stop sqlite3 from adding the same exact values into a table if it is the exact same but otherwise add it? I'm totaly new to sqlite and don't know how to do this.

When you create the table, specify a unique constraint:
create table foo ( name varchar, id integer, unique ( name, id) );

You should define your table as #Robᵩ answered.
If you don't want, however, change an existing table definition - in SQLite you are very limited in ALTER TABLE, you can create a unique index:
CREATE UNIQUE INDEX foo_idx ON foo (name, id);
Note you are not allowed to create this index until you remove all duplicates.