Have a terrible time trying to get this working. Spent way too many hours doing searches only to come up with the same examples which are not helping.
Made myself a GUI that starts a Thread to read the serial ports and parse some GPS data into NMEA sentences.
I take the data and want to write it to a database instead of a text file just to make it cleaner. However, when I write to the DB it throws an error saying that:
'type' object has no attribute '__getitem__'
As I have zero experience dealing with databases and python I'm at an impass as to what is wrong.
I know its the cur.execute INSERT line tossing the exception, I just don't know how to fix it.
Anyone care to walk me through it? I'm guessing it might be syntax? I'm using the pynmea2 module for GPS parsing and the pypyodbc module to make the MDB file.
If I write hard-coded-data it works fine. It's when I use variables that it craps out.
try:
print ("Attempt to Execute...")
Ndx=ID;Tod=str(gps_msg.timestamp);Lat=str(gps_msg.latitude);Lon=str(gps_msg.longitude);TmpLat='0';TmpLon='0';Alt=str(gps_msg.altitude);Flags='0';SystemID=str(gps_msg.ref_station_id);NumSat=str(gps_msg.num_sats);dop='0';Ch1_RSSI=str(rssi_dB)
cur.execute("INSERT INTO LogFile(Ndx,Tod,Lat,Lon,TmpLat,TmpLon,Alt,Flags,SystemID,NumSat,dop,Ch1_RSSI) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",([Ndx],[Tod],[Lat],[Lon],[TmpLat],[TmpLon],[Alt],[Flags],[SystemID],[NumSat],[dop],[Ch1_RSSI]))
print ("Attempting to commit...")
conn.commit()
print ("Commit succeeded..")
ID = ID + 1
except Exception as e:
errorLog.writelines('File Write Failure: {0}'.format(e)+"\n")
You are trying to pass the parameter values as a tuple whose elements are single-item lists. That won't work. The parameters must be passed as a list (or tuple) of individual values or objects.
So, instead of passing
([Ndx],[Tod],[Lat], ... )
you should be passing
[Ndx,Tod,Lat, ... ]
as the second argument to the .execute() method.
Related
I have a Python script that takes a list of names from the user, creates an SQL query string, and queries the database and puts the data into a dataframe using pandas.read_sql() method. However, I have noticed that if the list of people I want to query is very very long, my code never reaches the line coming after pandas.read_sql(), and for some reason, no exception is caught.
My query string looks like the following:
SELECT * FROM table1 WHERE
table1.name = 'Alice' OR
table1.name = 'Bob' OR
table1.name = 'Charlie' OR ...
where there could be hundreds of names included.
The Python code I am using looks like the following:
query_string = construct_string_above(names_list)
pymongoConnection = ...
try:
print("Trying")
df = pandas.read_sql(query_string, pymongoConnection)
except Exception as ex:
print(ex)
traceback.print_exc()
sys.exit(0)
print("Finishing")
My code always executes up until the try block. The program prints "Trying" but never prints "Finishing" nor does it return any errors. I checked, and the pymongoConnection object is correct. I also tried executing the query_string manually in MySQL and it works.
The issue must be at pandas.read_sql() and even here, the issue only arises when the size of the query string exceeds a certain threshold, otherwise the code finishes correctly. Are there any limitations to the use of pandas.read_sql(), such as maximum size of input string or maximum WHERE clauses or maximum amount of data returned to the dataframe? Because I cannot think of anything else that might cause the problem.
I am writing a simple python program to retrieve information from a localhost database i set up.
I am encountering the following error:
('24000', '[24000] [Microsoft][ODBC Driver 17 for SQL Server]Invalid cursor state')
My code:
try:
cursor = conn.cursor()
cursor.execute(getSQLStatement('SelectRecipe'),[item])
recipe = cursor.fetchone()[0]
cursor.close()
except Exception as e:
print(e)
return
The query i am running works fine in the database client, the query isnt the issue, it is a simple SELECT statement:
SELECT
Recipe
FROM recipes
WHERE Name = ? --(item)
My desired result is just the most basic statement just so i can get it running, and then be able to expand the complexity later, but something is wrong at this level. I am just trying to get the value of one record and one column i believe with this code. When i searched other similar issues people had with this the answer was always related to multiple result sets being attempted to be returned? My query is just one result set and i am only running it once? I am kind of scratching my head here to find what i am missing?
The error seems to be related to the:
recipe = cursor.fetchone()[0]
...line as it runs fine without throwing the error without this line, however then i am not sure how to get the data as a variable if i dont do it that way.
Wondering if anyone could help me out here, if there is a better way to get my data or if there is something wrong with my implementation of this way. Thanks.
So I just started working with sqlite in Python (and I am a beginner with Python as well), and I'm struggling with using the data from a Table.
Basically, I created my database but now I would like to do something with the data. I have tried just simply printing it using this function:
def Select():
surname = input("Whose birthdate would you like to know? (surname)\n")
print(c.execute("SELECT Birthday FROM People WHERE Surname=?", (surname,)))
but this is what gets printed when I call on the function.
<sqlite3.Cursor object at 0x0000020825545F10>
Anybody know what the fix for this is?
Thanks in advance!
You have to use fetchone function of the cursor to get one record or you can use fetchall to get all the row of the query.
Try using the method fetchone or fetch many.
print(c.fetchone())
or
print(c.fetchmany())
you can see the documentation here.
I'm having this function that communicates via pymysql to an SQL database stored to my localhost. I know there are similar posts about formatting an SQL section especially this one but could anyone suggest a solution?
Always getting TypeError: can't concat tuple to bytes. I suppose it's sth with the WHERE clause.
def likeMovement(pID):
print("Give a rating for the movement with #id:%s" %pID)
rate=input("Give from 0-5: ")
userID=str(1)
print(rate,type(rate))
print(pID,type(pID))
print(userID,type(userID))
cursor=con.cursor()
sqlquery='''UDPATE likesartmovement SET likesartmovement.rating=%s WHERE
likesartmovement.artisticID=? AND likesartmovement.userID=?''' % (rate,),
(pID,userID)
cursor.execute(sqlquery)
TypeError: not all arguments converted during string formatting
Thanks in advance!
The problem is that you're storing (pID,userID) as part of a tuple stored in sqlquery, instead of passing them as the arguments to execute:
sqlquery='''UDPATE likesartmovement SET likesartmovement.rating=%s WHERE
likesartmovement.artisticID=? AND likesartmovement.userID=?''' % (rate,)
cursor.execute(sqlquery, (pID,userID))
It may be clearer to see why these are different if you take a simpler example:
s = 'abc'
spam(s, 2)
s = 'abc', 2
spam(s)
Obviously those two don't do the same thing.
While we're at it:
You have to spell UPDATE right.
You usually want to use query parameters for SET clauses for exactly the same reasons you want to for WHERE clauses.
You don't need to include the table name in single-table operations, and you're not allowed to include the table name in SET clauses in single-table updates.
So:
sqlquery='''UDPATE likesartmovement SET rating=? WHERE
artisticID=? AND userID=?'''
cursor.execute(sqlquery, (rating, pID, userID))
I have created a database to store NGS sequencing results. It consists of 17 tables to store all of the information. The results are stored in spreadsheets which I parse data from and store in variables using python (2.7), and then use the python package mysqldb to insert data into the database. I mainly use functions to obtain the information i need in variables, then write a loop in which I call this function followed by a 'try:' statement to insert. Here is a simple example:
def sample_processer(file):
my_file = open(file, 'r+')
samples = []
for line in my_file:
...get info...
samples.append(line[0])
return(samples)
samples = sample_processor('path/to/file')
for sample in samples:
try:
sql = "samsql = "INSERT IGNORE INTO sample(sample_id, diagnosis, screening) VALUES ("
samsql = samsql + "'"+sample+"'," +sam_screen_dict.get(sample)+"')"
except e:
db.rollback()
print("Something went wrong inserting data into the sample table: %s" %(e))
*sam_screen_dict is a dictionary i made from another function.
This is a simple table that I upload into but many of them call of different dictionaries to make sure the correct results are uploaded. However I was wondering whether there would be a more robust way in which to do this using a class.
For example, my sample_id has an associated screening attribute in the sample table, so this is easy to do with one dictionary. I have more complex junction tables, such as the table in which the sample_id, experiment_id and found mutation are stored, alongside other data, would it be a good idea to create a class for this table, calling on a simple 'sample' class to inherit from? That way I would always know that the results being inserted will be for the correct sample/experiment etc.
Also, using classes could I write rules for each attribute so that if the source spreadsheet is for some reason incorrect, it will not insert into the database?
I.e: sample_id is in the format A123/16. Therefore using a class it will check that the first character is 'A' and that sample_id[-3] should always == '/'. I know I could write these into functions, but I feel like it would take up so much space and time writing so many 'if' statements, that if it is stored once in a class then this would be alot better.
Has anybody done anything similar using classes to pass through their variables to test that they are correct before it gets to the insert stage and an error is created?
I am new to python classes and understand the basics, still trying to get to grips with them so a point in the right direction would be great - as would any help on how to go about actually writing the code for a python class that would be used to make a more robust database insertion program.
17tables it means you may use about 17 classes.
Use a simple script. webpy.db
https://github.com/webpy/webpy/blob/master/web/db.py just modify few code.
Then you can use webpy api: http://webpy.org/docs/0.3/api#web.db to finish your job.
Hope it's useful for you