I wrote some code to learn SQL databases. My code works fine like I want it to. But I get this error and want to learn what is that.
import sqlite3
con = sqlite3.connect("items.db")
cursor = con.cursor()
cursor.execute("Create table if not exists weapons (name TEXT,ilvl TEXT,source TEXT)")
weapons_txt = open("C:\\Users\\kaytu\\Desktop\\Python\\Exercises\\weapons.txt","r")
for i in weapons_txt:
cursor.execute("Insert into weapons values(?,?,?)",(i.split(";")[0],i.split(";")[1],i.split(";")[2],))
con.commit()
weapons_txt.close()
con.close()
Traceback (most recent call last):
File "c:\Users\kaytu\Desktop\Python\Exercises\Testing.py", line 9, in <module>
cursor.execute("Insert into weapons values(?,?,?)",(i.split(";")[0],i.split(";")[1],i.split(";")[2],))
IndexError: list index out of range
And why do i get the "..." string after every source text? printscreen
this means that at least one of the lines in the txt file has less than two semicolons, check all the lines of the file again
Related
import psycopg2
def creat_tabel():
conn=psycopg2.connect("dbname='database1' user='postgres' password='postgres123' host='localhost' port='5432' ")
cur=conn.cursor()
cur.execute("CREATE TABLE İF NOT EXISTS store (item TEXT , quantity INTEGER , price REAL)")
conn.commit()
conn.close()
creat_tabel()
This is my code and this is my error. How can I fix it? Please help.
C:\Users\sinan urgun\Desktop\python\db>script2_postgresql.py
Traceback (most recent call last):
File "C:\Users\sinan urgun\Desktop\python\db\script2_postgresql.py", line 10, in <module>
creat_tabel()
File "C:\Users\sinan urgun\Desktop\python\db\script2_postgresql.py", line 4, in creat_tabel
conn=psycopg2.connect("dbname='database1' user='postgres' password='postgres123' host='localhost' port='5432' ")
File "C:\Users\sinan urgun\AppData\Local\Programs\Python\Python39\lib\site-packages\psycopg2\__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError
You have a typo in your SQL. You have written "İF", where that first character is U+0130 : LATIN CAPITAL LETTER I WITH DOT ABOVE. You want to write "IF" instead.
You can see the dot above the I in your question; you should also be able to see this in your local editor. If this is a regular problem you may want to experiment with different fonts that make the issue more obvious.
I'm working on an application and currently focusing on the database area.
I have successfully setup functions that create databases and write to them, however right now I am trying to retrieve specific rows from the database however NO MATTER WHAT I TRY I always get a syntax error DESPITE my SQL query running successfully in in sqlite browser.
This is my trouble code:
def findapple():
conn = sqlite3.connect('working.db')
c = conn.cursor()
for row in c.execute('SELECT * FROM "working" WHERE "symbol" = 'aapl';')
print row
If I understand the code I've written, it should connect to "working.db" create a cursor for the database then for each row that match my query it should be printed to the console.
Can anyone please help shed some light on this?
Thank you!
Current Code Per comments below:
def findapple():
conn = sqlite3.connect('working.db')
c = conn.cursor()
for row in c.execute('SELECT * FROM working WHERE symbol = 'aapl'')
print row
and the error output:
Traceback (most recent call last):
File "main.py", line 1, in <module>
from stock import *
File "/Users/ME/Documents/Code/Stocks/stock/database.py", line 61
for row in c.execute('SELECT * FROM working WHERE symbol = 'aapl'')
Your string syntax is wrong:
c.execute('SELECT * FROM working WHERE symbol = "aapl"')
I am trying to take a data from a log file in cvs format, open the log file and inserting row by row into mysql. I am getting an error like this:
ERROR Traceback (most recent call last): File "/Users/alex/PycharmProjects/PA_REPORTING/padb_populate.py", line 26, in VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)', row) File "/Users/alex/anaconda/lib/python2.7/site-packages/MySQLdb/cursors.py", line 187, in execute query = query % tuple([db.literal(item) for item in args]) TypeError: not all arguments converted during string formatting.
import csv
import MySQLdb
mydb = MySQLdb.connect(host='192.168.56.103',
user='user',
passwd='pass',
db='palogdb')
cursor = mydb.cursor()
csv_data = csv.reader(file('/tmp/PALOG_DEMODATA-100.csv'))
for row in csv_data:
cursor.execute('INSERT INTO palogdb(RECEIVE_TIME,SERIAL,TYPE,SUBTYPE,COL1,TIME_GENERATED,SRC,DST,NATSRC,NATDST,RULE,\
SRCUSR,DSTUSR,APP,VSYS1,FROM,TO,INBOUND_IF,OUTBOUND_IF,LOGSET,COL2,SESSIONID,COL3,REPEATCNT,SOURCEPORT,NATSPORT,NATDPORT, \
FLAGS,PROTO,ACTION,BYTES,BYTES_SENT,BYTES_RECEIVED,PACKETS,START,ELAPSED,CATEGORY,COL4,SEQNO,ACTIONFLAGS,SRCLOC,DSTLOC,NONE, \
PKTS_SENT,PKTS_RECEIVED,SESSION_END_REASON) \
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)', row)
#close the connection to the database.
mydb.commit()
cursor.close()
Is it possible, that you don't have enough data in row for all your %s's? Maybe your row is interpreted as one value, and thus only the first %s is expanded? Try *row to expand the vector to values.
To debug, you could try to build the string passed to execute by some other method, e.g.
sql_string = 'INSERT ... VALUES ({}, {}, {})'.format(*row)
and print it. If you get such an error, you can check, whether the generated string looks reasonable...
I get the following error code while executing my Code. The error does not occur immediately - it occurs randomly after 2-7 hours. Until the error occurs there is no problem to stream the online feeds and write them in a DB.
Error message:
Traceback (most recent call last):
File "C:\Python27\MySQL_finalversion\RSS_common_FV.py", line 78, in <module>
main()
File "C:\Python27\MySQL_finalversion\RSS_common_FV.py", line 63, in main
feed_iii = feed_load_iii(feed_url_iii)
File "C:\Python27\MySQL_finalversion\RSS_common_FV.py", line 44, in feed_load_iii
in feedparser.parse(feed_iii).entries]
IndexError: list index out of range
Here you can find my Code:
import feedparser
import MySQLdb
import time
from cookielib import CookieJar
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="root", # your username - SELECT * FROM mysql.user
passwd="****", # your password
db="sentimentanalysis_unicode",
charset="utf8") # name of the data base
cur = db.cursor()
cur.execute("SET NAMES utf8")
cur.execute("SET CHARACTER SET utf8")
cur.execute("SET character_set_connection=utf8")
cur.execute("DROP TABLE IF EXISTS feeddata_iii")
sql_iii = """CREATE TABLE feeddata_iii(III_ID INT NOT NULL AUTO_INCREMENT, PRIMARY KEY(III_ID),III_UnixTimesstamp integer,III_Timestamp varchar(255),III_Source varchar(255),III_Title varchar(255),III_Text TEXT,III_Link varchar(255),III_Epic varchar(255),III_CommentNr integer,III_Author varchar(255))"""
cur.execute(sql_iii)
def feed_load_iii(feed_iii):
return [(time.time(),
entry.published,
'iii',
entry.title,
entry.summary,
entry.link,
(entry.link.split('=cotn:')[1]).split('.L&id=')[0],
(entry.link.split('.L&id=')[1]).split('&display=')[0],
entry.author)
for entry
in feedparser.parse(feed_iii).entries]
def main():
feed_url_iii = "http://www.iii.co.uk/site_wide_discussions/site_wide_rss2.epl"
feed_iii = feed_load_iii(feed_url_iii)
print feed_iii[1][1]
for item in feed_iii:
cur.execute("""INSERT INTO feeddata_iii(III_UnixTimesstamp, III_Timestamp, III_Source, III_Title, III_Text, III_Link, III_Epic, III_CommentNr, III_Author) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)""",item)
db.commit()
if __name__ == "__main__":
while True:
main()
time.sleep(240)
If you need further information - please feel free to ask. I need your help!
Thanks and Regards from London!
In essence, your program is insufficiently resilient to poorly-formatted data.
Your code makes very explicit assumptions about the structure of the data, and is unable to cope if the data is not so structured. You need to detect the cases where the data is incorrectly formatted and take some other action then.
A rather sloppy way to do this would simply trap the exception that's currently being raised which you could do with (something like)
try:
feed_iii = feed_load_iii(feed_url_iii)
except IndexError:
# do something to report or handle the data format problem
I am working with Python 3.3, pypyodbc 1.2.1, and a Quickbooks Enterprise 12 company file being access over Flexquarters QODBC version 14. I'm new to programming and python, so still learning :) I can run a query using the pypyodbc examples just fine, and produces expected results.
Notice the hardcoded email address in the execute. This works as expected:
def get_customer_id(search_col,search_str):
'''(str,str) --> str
>>>get_customer_id(email, foo#foo.com)
80000001-1385782702
'''
cur.execute("SELECT listid FROM CUSTOMER WHERE email='foo#foo.com'")
for row in cur.fetchall():
for field in row:
return field
If I try to do the same thing using the parameters that I am reading from the pypyodbc documentation, I throw an error. I'm having problems with the quotes, and parameter markers I think.
def get_customer_id(search_col,search_str):
'''(str,str) --> str
>>>get_customer_id(email, foo#foo.com)
80000001-1385782702
'''
cur.execute("SELECT listid FROM CUSTOMER WHERE email=?",(search_str,))
for row in cur.fetchall():
for field in row:
return field
Trying to be more pythonistic? I really want to reuse the function to search different columns. Something like:
cur.execute("SELECT listid FROM CUSTOMER WHERE search_str=search_col")
I have looked at a few other threads, and most of them seem to just be dealing with the parameter, and not the column to search. Can anyone help me learn this?
PS forgot to include the traceback:
Traceback (most recent call last):
File "C:\Users\Mike\Documents\Projects\qb_sync\quickbooks.py", line 32, in <module>
print(get_customer_id('email','foo#foo.com'))
File "C:\Users\Mike\Documents\Projects\qb_sync\quickbooks.py", line 27, in get_customer_id
cur.execute("SELECT listid FROM CUSTOMER WHERE email=?",[search_str,])
File "C:\Python\lib\site-packages\pypyodbc.py", line 1457, in execute
self._BindParams(param_types)
File "C:\Python\lib\site-packages\pypyodbc.py", line 1420, in _BindParams
check_success(self, ret)
File "C:\Python\lib\site-packages\pypyodbc.py", line 982, in check_success
ctrl_err(SQL_HANDLE_STMT, ODBC_obj.stmt_h, ret, ODBC_obj.ansi)
File "C:\Python\lib\site-packages\pypyodbc.py", line 960, in ctrl_err
raise Error(state,err_text)
pypyodbc.Error: ('HY004', '[HY004] [Microsoft][ODBC Driver Manager] SQL data type out of range')
[Finished in 1.7s]
I think the use of
cur.execute("""SELECT listid FROM CUSTOMER WHERE ?=?""",[column, email])
can not be accepted by database engine rather than pypyodbc or any other odbc interface. It's the database engine refuse to accept the query for the use of parameter on column names.
Probably you would have to try this instead to reuse the function:
# First construct your dynamic query for the targeted column
sql = """SELECT listid FROM CUSTOMER WHERE %s=?""" %(column)
# Then provide the dynamic value for the dynamic query string
cur.execute(sql, (value,))
Python 3 also has the str.format() method which will do string replacement on {index} items within your string. This is useful if you have many values to inject into your strings like:
myStr = "I like {0} and {1}, but I don't like {2}.".format("apples","bananas","spinach")
myStr
"I like apples and bananas, but I don't like spinach."
# First construct your dynamic query for the targeted column
sql = """SELECT listid FROM CUSTOMER WHERE {0}=?""".format(column)
# Then provide the dynamic value for the dynamic query string
cur.execute(sql, (value,))
It's worth noting that this method of replacing values in a string query can be subject to sql injection.
The safer way to do this would be with parameterized stored procs.
I got 1/2 the answer so far. This works for one parameter, IF I format the string before calling the function;
print(get_custid_email(b'foo#foo.org'))
cur.execute("""SELECT listid FROM CUSTOMER WHERE email=?""",[email])
I still can't get it to do the same thing with column name though.
print(get_custid_email(b'email',b'foo#foo.org'))
cur.execute("""SELECT listid FROM CUSTOMER WHERE ?=?""",[column, email])
That throws a differnt error:
Traceback (most recent call last):
File "C:\Users\Mike\Documents\Projects\qb_sync\quickbooks.py", line 34, in <module>
print(get_custid_email(b'wendy.lindsay#gmail.com'))
File "C:\Users\Mike\Documents\Projects\qb_sync\quickbooks.py", line 29, in get_custid_email
cur.execute("""SELECT listid FROM CUSTOMER WHERE ?=?""",['email',email])
pyodbc.ProgrammingError: ('42S00', '[42S00] [QODBC] Data type of parameter cannot be determined (11023) (SQLPrepare)')