Develop Reference

Python MySQL Connector not inserting

So here is my code. I don't know why insert isn't working. A select statement works. It doesn't fail the try catch either leading me to believe the query is executing. Also entering the insert query manually into MySQL Workbench seems to work fine.
def runQuery(query):
try:
conn = mysql.connector.connect(host='localhost',
database='optionsdata',
user='python',
passwd='python')
cursor = conn.cursor()
cursor.execute(query)
conn.close()
cursor.close()
print(query)
except Error as e:
print("Error", e)
def convertDate(date_str):
date_object = datetime.datetime.strptime(date_str, '%m/%d/%Y').date()
return date_object
ticker = "MSFT"
html = urlopen("https://api.nasdaq.com/api/quote/" + ticker + "/option-chain?assetclass=stocks&todate=2020-05-08&fromdate=2020-04-07&limit=0").read().decode('utf-8')
optionsData = json.loads(html)
rows = optionsData["data"]["optionChainList"]["rows"]
for row in rows:
call = row["call"]
expiryDate = convertDate(call["expiryDate"])
query = "INSERT INTO `optionsdata`.`call` (`ticker`, `symbol`, `last`, `change`, `bid`, `ask`, `volume`, `openinterest`, `strike`, `expiryDate`, `grabTime`) VALUES ('{0}', '{1}', '{2}', '{3}', '{4}', '{5}', '{6}', '{7}', '{8}', '{9}', '{10}');".format(ticker, call["symbol"], call["last"], call["change"], call["bid"], call["ask"], call["volume"], call["openinterest"], call["strike"], expiryDate, datetime.datetime.now())
runQuery(query)
A sample of what an insert query looks like
INSERT INTO `optionsdata`.`call` (`ticker`, `symbol`, `last`, `change`, `bid`, `ask`, `volume`, `openinterest`, `strike`, `expiryDate`, `grabTime`) VALUES ('MSFT', '#MSFT 200508C00175000', '3.21', '-0.29', '2.80', '4.25', '54', '228', '175.00', '2020-05-08', '2020-04-09 19:39:22.554538');

This is a great question! I spent hours trying to figure this out a few weeks ago. It's tricky because after executing the query, you have to call
conn.commit()
to actually update the data. So change your runQuery function like this:
def runQuery(query):
try:
conn = mysql.connector.connect(host='localhost',
database='optionsdata',
user='python',
passwd='python')
cursor = conn.cursor()
cursor.execute(query)
conn.commit() # Added commit line
conn.close()
cursor.close()
print(query)
except Error as e:
print("Error", e)
See this doc page for more info.

sqlite3 python negative row count [duplicate]

I'm trying to insert new records into SQLite database from Python code.
con = sqlite.connect(connectionString)
cur = con.cursor()
countOfNewItems = 0
for ...
try:
con.execute("insert or ignore into items ...")
countOfNewItems += cur.rowcount
except:
cur.close()
con.close()
print "Error when inserting item '%s' to database." % item
exit(1)
cur.close()
con.commit()
con.close()
print "%d new items have been inserted." % countOfNewItems
My code reports negative number of inserted records (-5141).
Because my database was empty, I could find out how many records were inserted via command line
select count(*) from items;
4866
Could you advise me what's wrong. Why the two values don't match and why it's negative?

http://docs.python.org/library/sqlite3.html#sqlite3.Cursor.rowcount
Although the Cursor class of the sqlite3 module implements this attribute, the database engine’s own support for the determination of “rows affected”/”rows selected” is quirky.
and
As required by the Python DB API Spec, the rowcount attribute “is -1 in case no executeXX() has been performed on the cursor or the rowcount of the last operation is not determinable by the interface”.

Try cur.execute instead of con.execute. cur.rowcount then returns 1 for me for a simple insert.

Use a python dictionary to insert into mysql

I am trying to take the data from a dictionary (the example is simplified for readability) and insert it into a mysql database.
I have the following piece of code.
import pymysql
conn = pymysql.connect(server, user , password, "db")
cur = conn.cursor()
ORFs={'E7': '562', 'E6': '83', 'E1': '865', 'E2': '2756 '}
table="genome"
cols = ORFs.keys()
vals = ORFs.values()
sql = "INSERT INTO %s (%s) VALUES(%s)" % (
table, ",".join(cols), ",".join(vals))
print sql
print ORFs.values()
cur.execute(sql, ORFs.values())
cur.close()
conn.close()
the print sql statement returns
INSERT INTO genome (E7,E6,E1,E2) VALUES(562,83,865,2756 )
when I type this directly into the mysql command line, the mysql command works. But when I run the python script I get an error:
<type 'exceptions.TypeError'>: not all arguments converted during string formatting
args = ('not all arguments converted during string formatting',)
message = 'not all arguments converted during string formatting'
As always, any suggestions would be highly appreciated.

The previous answer doesn't work for non string dictionary value. This one is a revised version.
format_string = ','.join(['%s'] * len(dict))
self.db.set("""INSERT IGNORE INTO listings ({0}) VALUES ({1})""".format(", ".join(dict.keys()),format_string),
(dict.values()))

sql = "INSERT INTO %s (%s) VALUES(%s)" % (
table, ",".join(cols), ",".join(vals))
This SQL includes values and cur.execute(sql, ORFs.values()) has values, too.
So, it should be cur.execute(sql).

In my case, I will skip null columns.
data = {'k': 'v'}
fs = ','.join(list(map(lambda x: '`' + x + '`', [*data.keys()])))
vs = ','.join(list(map(lambda x: '%(' + x + ')s', [*data.keys()])))
sql = "INSERT INTO `%s` (%s) VALUES (%s)" % (table, fs, vs)
count = cursor.execute(sql, data)

Python: tuple indices must be integers, not str when selecting from mysql table

I have following method that I select all the ids from table and append them to a list and return that list. But when execute this code I end up getting tuple indicies must be integers... error. I have attached the error and the print out along with my method:
def questionIds(con):
print 'getting all the question ids'
cur = con.cursor()
qIds = []
getQuestionId = "SELECT question_id from questions_new"
try:
cur.execute(getQuestionId)
for row in cur.fetchall():
print 'printing row'
print row
qIds.append(str(row['question_id']))
except Exception, e:
traceback.print_exc()
return qIds
Printing what my method does:
Database version : 5.5.10
getting all the question ids
printing row
(u'20090225230048AAnhStI',)
Traceback (most recent call last):
File "YahooAnswerScraper.py", line 76, in questionIds
qIds.append(str(row['question_id'][0]))
TypeError: tuple indices must be integers, not str

The python standard mysql library returns tuples from cursor.execute. To get at the question_id field you'd use row[0], not row['question_id']. The fields come out in the same order that they appear in the select statement.
A decent way to extract multiple fields is something like
for row in cursor.execute("select question_id, foo, bar from questions"):
question_id, foo, bar = row

There are multiple cursor types in the MySQLdb module. The default cursor returns the data in a tuple of tuples. When we use a dictionary cursor, the data is sent in a form of Python dictionaries. This way we can refer to the data by their column names. Source
#!/usr/bin/python
# -*- coding: utf-8 -*-
import MySQLdb as mdb
con = mdb.connect('localhost', 'testuser', 'test623', 'testdb')
with con:
cur = con.cursor(mdb.cursors.DictCursor)
cur.execute("SELECT * FROM Writers LIMIT 4")
rows = cur.fetchall()
for row in rows:
print row["Id"], row["Name"]

I know the question is old, but I found another way to do it that I think it is better than the accepted solution. So I'll just leave it here in case anyone needs it.
When creating the cursor you can use
cur = connection.cursor(dictionary=True);
which will allow you to do exactly what you want without any additional modifications.
rows = cur.fetchall()
for row in rows:
print "%s %s %s" % (row["Id"], row["Name"], row["Price"])

you can see here: enter link description here ,I think its your want
#!/usr/bin/python
# -*- coding: utf-8 -*-
import sqlite3 as lite
con = lite.connect('test.db')
with con:
con.row_factory = lite.Row # its key
cur = con.cursor()
cur.execute("SELECT * FROM Cars")
rows = cur.fetchall()
for row in rows:
print "%s %s %s" % (row["Id"], row["Name"], row["Price"])

To retrieve data from database use dictionary cursor
import psycopg2
import psycopg2.extras
con = psycopg2.connect(database="test", user="test", password="test", host="localhost", port="5432")
if con != None:
print "Connection Established..!\n"
else:
print "Database Connection Failed..!\n"
cur = con.cursor(cursor_factory=psycopg2.extras.DictCursor)
cur.execute("SELECT * FROM emp")
rows = cur.fetchall()
for row in rows:
print "%s %s %s" % (row["id"],row["name"],row["address"])
print "\nRecords Display Successfully"
con.commit()
con.close()

Integer indices are not allowed. To get it working you can declare the DICT as specified below:
VarName = {}
Hope this works for you.

row is a tuple. When you do row['question_id'], you are trying to access a tuple using a string index which gives you an error.

TypeError: 'NoneType' object is not iterable

I need to process mysql data one row at a time and i have selected all rows put them in a tuple but i get the error above.
what does this mean and how do I go about it?

Provide some code.
You probably call some function that should update database, but the function does not return any data (like cursor.execute()). And code:
data = cursor.execute()
Makes data a None object (of NoneType). But without code it's hard to point you to the exact cause of your error.

It means that the object you are trying to iterate is actually None; maybe the query produced no results?
Could you please post a code sample?

The function you used to select all rows returned None. This "probably" (because you did not provide code, I am only assuming) means that the SQL query did not return any values.

Try using the cursor.rowcount variable after you call cursor.execute(). (this code will not work because I don't know what module you are using).
db = mysqlmodule.connect("a connection string")
curs = dbo.cursor()
curs.execute("select top 10 * from tablename where fieldA > 100")
for i in range(curs.rowcount):
row = curs.fetchone()
print row
Alternatively, you can do this (if you know you want ever result returned):
db = mysqlmodule.connect("a connection string")
curs = dbo.cursor()
curs.execute("select top 10 * from tablename where fieldA > 100")
results = curs.fetchall()
if results:
for r in results:
print r

This error means that you are attempting to loop over a None object. This is like trying to loop over a Null array in C/C++. As Abgan, orsogufo, Dan mentioned, this is probably because the query did not return anything. I suggest that you check your query/databse connection.
A simple code fragment to reproduce this error is:
x = None
for each i in x:
#Do Something
pass

This may occur when I try to let 'usrsor.fetchone' execute twice. Like this:
import sqlite3
db_filename = 'test.db'
with sqlite3.connect(db_filename) as conn:
cursor = conn.cursor()
cursor.execute("""
insert into test_table (id, username, password)
values ('user_id', 'myname', 'passwd')
""")
cursor.execute("""
select username, password from test_table where id = 'user_id'
""")
if cursor.fetchone() is not None:
username, password = cursor.fetchone()
print username, password
I don't know much about the reason. But I modified it with try and except, like this:
import sqlite3
db_filename = 'test.db'
with sqlite3.connect(db_filename) as conn:
cursor = conn.cursor()
cursor.execute("""
insert into test_table (id, username, password)
values ('user_id', 'myname', 'passwd')
""")
cursor.execute("""
select username, password from test_table where id = 'user_id'
""")
try:
username, password = cursor.fetchone()
print username, password
except:
pass
I guess the cursor.fetchone() can't execute twice, because the cursor will be None when execute it first time.

I know it's an old question but I thought I'd add one more possibility. I was getting this error when calling a stored procedure, and adding SET NOCOUNT ON at the top of the stored procedure solved it. The issue is that earlier selects that are not the final select for the procedure make it look like you've got empty row sets.

Try to append you query result to a list, and than you can access it. Something like this:
try:
cursor = con.cursor()
getDataQuery = 'SELECT * FROM everything'
cursor.execute(getDataQuery)
result = cursor.fetchall()
except Exception as e:
print "There was an error while getting the values: %s" % e
raise
resultList = []
for r in result:
resultList.append(r)
Now you have a list that is iterable.