Develop Reference

Python pyodbc loop - remove brackets, hyphens and commas from list

I'm using the below Python script to loop around a SQL Server table and save the values to a list.
However, I don't want the brackets \ commas etc. When I run the below the records in the list look like this:
('TN12345', )
I just want the below - how it exists in the SQL table:
TN12345
This is my script:
import pyodbc
connstr = 'DRIVER={SQL Server};SERVER=XXXXXXXXXXX;DATABASE=XXXXXXXXXXX;Trusted_Connection=yes;'
conn = pyodbc.connect(connstr)
cursor = conn.cursor()
cursor.execute("""SELECT TN_Number FROM Table""")
records = cursor.fetchall()
insertObject = []
columnNames = [column[0] for column in cursor.description]
for record in records:
insertObject.append( dict( zip( columnNames , record ) ) )
for i in range(len(records)):
print(records[i])

This
('TN12345', )
is tuple with one element, representing row in database, if you can guarantee that your query will always return exactly 1 column then you might use [0] to access single element in said tuple, in your case replace
for i in range(len(records)):
print(records[i])
using
for i in range(len(records)):
print(records[i][0])
alternatively you might use for loop directly (without caring about index) in following way
for record in records:
print(record[0])

If you want to get all elements one by one in your tuples:
for i in range(len(records)):
for j in records[i]:
print(j)
Advantage is it will return every entry in your tuples no matter the length of the tuple or the length of the list of tuples.

Insert record from list if not exists in table

cHandler = myDB.cursor()
cHandler.execute('select UserId,C1,LogDate from DeviceLogs_12_2019') // data from remote sql server database
curs = connection.cursor()
curs.execute("""select * from biometric""") //data from my database table
lst = []
result= cHandler.fetchall()
for row in result:
lst.append(row)
lst2 = []
result2= curs.fetchall()
for row in result2:
lst2.append(row)
t = []
r = [elem for elem in lst if not elem in lst2]
for i in r:
print(i)
t.append(i)
for i in t:
frappe.db.sql("""Insert into biometric(UserId,C1,LogDate) select '%s','%s','%s' where not exists(select * from biometric where UserID='%s' and LogDate='%s')""",(i[0],i[1],i[2],i[0],i[2]),as_dict=1)
I am trying above code to insert data into my table if record not exists but getting error :
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '1111'',''in'',''2019-12-03 06:37:15'' where not exists(select * from biometric ' at line 1")
Is there anything I am doing wrong or any other way to achieve this?

It appears you have potentially four problems:
There is a from clause missing between select and where not exists.
When using a prepared statement you do not enclose your placeholder arguments, %s, within quotes. Your SQL should be:
Your loop:
Loop:
t = []
r = [elem for elem in lst if not elem in lst2]
for i in r:
print(i)
t.append(i)
If you are trying to only include rows from the remote site that will not be duplicates, then you should explicitly check the two fields that matter, i.e. UserId and LogDate. But what is the point since your SQL is taking care of making sure that you are excluding these duplicate rows? Also, what is the point of copying everything form r to t?
SQL:
Insert into biometric(UserId,C1,LogDate) select %s,%s,%s from DUAL where not exists(select * from biometric where UserID=%s and LogDate=%s
But here is the problem even with the above SQL:
If the not exists clause is false, then the select %s,%s,%s from DUAL ... returns no columns and the column count will not match the number of columns you are trying to insert, namely three.
If your concern is getting an error due to duplicate keys because (UserId, LogDate) is either a UNIQUE or PRIMARY KEY, then add the IGNORE keyword on the INSERT statement and then if a row with the key already exists, the insertion will be ignored. But there is no way of knowing since you have not provided this information:
for i in t:
frappe.db.sql("Insert IGNORE into biometric(UserId,C1,LogDate) values(%s,%s,%s)",(i[0],i[1],i[2]))
If you do not want multiple rows with the same (UserId, LogDate) combination, then you should define a UNIQUE KEY on these two columns and then the above SQL should be sufficient. There is also an ON DUPLICATE KEY SET ... variation of the INSERT statement where if the key exists you can do an update instead (look this up).
If you don't have a UNIQUE KEY defined on these two columns or you need to print out those rows which are being updated, then you do need to test for the presence of the existing keys. But this would be the way to do it:
cHandler = myDB.cursor()
cHandler.execute('select UserId,C1,LogDate from DeviceLogs_12_2019') // data from remote sql server database
rows = cHandler.fetchall()
curs = connection.cursor()
for row in rows:
curs.execute("select UserId from biometric where UserId=%s and LogDate=%s", (ros[0], row[2])) # row already in biometric table?
biometric_row = curs.fetchone()
if biometric_row is None: # no, it is not
print(row)
frappe.db.sql("Insert into biometric(UserId,C1,LogDate) values(%s, %s, %s)", (row[0],row[1],row[2]))

How to use psycopg2 to retrieve a certain key's value from a postgres table which has key-value pairs

New to python, trying to use psycopg2 to read Postgres
I am reading from a database table called deployment and trying to handle a Value from a table with three fields id, Key and Value
import psycopg2
conn = psycopg2.connect(host="localhost",database=database, user=user, password=password)
cur = conn.cursor()
cur.execute("SELECT \"Value\" FROM deployment WHERE (\"Key\" = 'DUMPLOCATION')")
records = cur.fetchall()
print(json.dumps(records))
[["newdrive"]]
I want this to be just "newdrive" so that I can do a string comparison in the next line to check if its "newdrive" or not
I tried json.loads on the json.dumps output, didn't work
>>> a=json.loads(json.dumps(records))
>>> print(a)
[['newdrive']]
I also tried to print just the records without json.dump
>>> print(records)
[('newdrive',)]

The result of fetchall() is a sequence of tuples. You can loop over the sequence and print the first (index 0) element of each tuple:
cur.execute("SELECT \"Value\" FROM deployment WHERE (\"Key\" = 'DUMPLOCATION')")
records = cur.fetchall()
for record in records:
print(record[0])
Or simpler, if you are sure the query returns no more than one row, use fetchone() which gives a single tuple representing returned row, e.g.:
cur.execute("SELECT \"Value\" FROM deployment WHERE (\"Key\" = 'DUMPLOCATION')")
row = cur.fetchone()
if row: # check whether the query returned a row
print(row[0])

Fetching a variable

I got this code:
cursor.execute('SELECT nom FROM productes WHERE listacompra = 1')
producteslc = cursor.fetchone()
The problem is that when I do print producteslc, it returns (u'Natillas',), when the value on the SQL Database is just Natillas.
What could I do to have a variable with value = Natillas? I'm trying to do some stuff with split but I'm not able to do it at my own.
Thank you

The result of fetchone is a tuple of the values of one row.
Since you only fetch a single column, the result is a tuple singleton: (u'Natillas',)
To get the string:
producteslc = cursor.fetchone()[0]
See: Tuples and Sequences in the doc
EDIT
To fetch several rows, you can use fetchall() function:
rows = cursor.fetchall()
for row in rows:
print(row[0])
To print each name.

Comparing tuple elements against integers with Python

I am having a hard time converting data. I select the data from my database, which is returned in tuple format. I try to convert them using list(), but all I get is a list of tuples. I am trying to compare them to integers which i receive from parsing my JSON. What would be the easiest way to convert and compare these two?
from DBConnection import db
import pymssql
from data import JsonParse
db.execute('select id from party where partyid = 1')
parse = JsonParse.Parse()
for row in cursor:
curList = list(cursor)
i = 0
for testData in parse:
print curList[i], testData['data']
i += 1
Output:
(6042,) 6042
(6043,) 6043
(6044,) 6044
(6045,) 6045

SQL results always come as rows, which are sequences of columns; this is true even if there is just one column in each row.
Next, you are executing the query on the db object (whatever that may be), but are iterating over the cursor; if this works at all is more down to luck. You'd normally execute a query on the cursor object.
If you expect just one row to be returned, you can use cursor.fetchone() to retrieve that one row. Your for row in cursor loop is actually skipping the first row.
You could use:
cursor = connection.cursor()
cursor.execute('select id from party where partyid = 1')
result = cursor.fetchone()[0]
to retrieve the first column of the first row, or you could use tuple assignment:
cursor = connection.cursor()
cursor.execute('select id from party where partyid = 1')
result, = cursor.fetchone()
If you do need to match against multiple rows, you could use a list comprehension to extract all those id columns:
cursor = connection.cursor()
cursor.execute('select id from party where partyid = 1')
result = [row[0] for row in cursor]
Now you have a list of id values.

Quick and dirty:
print curList[i][0], testData['data']
Or how about:
for db_tuple, json_int in zip(cursor, parse):
print db_tuple[0], json_int