Develop Reference

SQLITE3 + Python (I need to ask bank 1 table if its data exists in bank 2 table)

I have a doubt about python and sqlite3.
import sqlite3
conna= sqlite3.connect('db_a')
a = conna.cursor()
connb= sqlite3.connect('db_b')
b = conna.cursor()
I don't know how to ask the relational question between banks, can someone instruct me?
I don't want to use DEF, just the SELECT code for a variable to assume
query = """SELECT COL1 FROM TABLE1.DB_A WHERE NOT EXISTS (SELECT COL1 FROM TABLE2.DB_B WHERE COL1.TABLE2.DE_B = COL1.TABLE1.DE_A)"""
cursor.execute(query)
records = cursor.fetchall()
for row in records:
print(row[0])
Can someone help me?

If the tables exist in different databases you need the ATTACH DATABASE statement to use the 2nd database with the connection object that you connect to the 1st database:
import sqlite3
conn = sqlite3.connect('db_a')
cursor = conn.cursor()
attach = "ATTACH DATABASE 'db_b' AS db_b;"
cursor.execute(attach)
query = """
SELECT t1.COL1
FROM TABLE1 AS t1
WHERE NOT EXISTS (
SELECT t2.COL1
FROM db_b.TABLE2 AS t2
WHERE t2.COL1 = t1.COL1
)
"""
cursor.execute(query)
records = cursor.fetchall()
for row in records:
print(row[0])
detach = "DETACH DATABASE db_b;"
cursor.execute(detach)
Also, instead of EXISTS you could use EXCEPT with the difference being that EXCEPT returns only distinct results:
query = """
SELECT COL1 FROM TABLE1
EXCEPT
SELECT COL1 FROM db_b.TABLE2
"""

psycopg2 select result gets nested with additional array

I am trying to get JSON rows from postgres using "psycopg2". value of records is like [ [{....},{...},{...}] ]. To get the correct JSON result ie. [{....},{...},{...}] i have to go for records1. Not sure why this is happening.
import psycopg2
import sys
import json
conn_string = "'host='localhost' dbname='postgres' user='postgres' password='password123'"
con=psycopg2.connect(conn_string)
cur = con.cursor()
cur.execute("select json_agg(art) from (select a.*, (select json_agg(b) from (select * from pfstklist where pfportfolioid = a.pfportfolioid ) as b) as pfstklist, (select json_agg(c) from (select * from pfmflist where pfportfolioid = a.pfportfolioid ) as c) as pfmflist from pfmaindetail as a) art")
records = cur.fetchall()
print(records) #This gives result [[{....},{...},{...}]]
records1=records[0]
print(records1) #This gives expected result [{....},{...},{...}]

cur.fetchall() gives you a list of records. You probably want cur.fetchone().

select multiple columns using SQLite3 in Python

I have a list that contains the name of columns I want to retrieve from a table in the database.
My question is how to make the cursor select columns specified in the list. Do I have to convert nameList to a string variable before include it in the select statement? Thanks
nameList = ['A','B','C','D',...]
with sqlite3.connect(db_fileName) as conn:
cursor = conn.cursor()
cursor.execute("""
select * from table
""")

As long as you can be sure your input is sanitized -- to avoid SQL injection attack -- you can do:
...
qry = "select {} from table;"
qry.format( ','.join(nameList) )
cursor.execute(qry)
If you're on a really old version of Python do instead:
...
qry = "select %s from table;"
qry % ','.join(nameList)
cursor.execute(qry)

nameList = ["'A(pct)'",'B','C','D',...]
with sqlite3.connect(db_fileName) as conn:
cursor = conn.cursor()
cursor.execute("""
select {} from table
""".format(", ".join(nameList)))

Replace L, in SQL results in python

I'm running pyodbc connected to my db and when i run a simply query I get a load of results back such as
(7L, )(12L,) etc.
How do I replace the the 'L, ' with '' so I can pass the ids into another query
Thanks
Here's my code
import pyodbc
cnxn = pyodbc.connect('DSN=...;UID=...;PWD=...', ansi=True)
cursor = cnxn.cursor()
rows = cursor.execute("select id from orders")
for row in rows:
test = cursor.execute("select name from customer where order_id = %(id)s" %{'id':row})
print test

Use parameters:
...
test = cursor.execute("select name from customer where order_id = ?", row.id)
...
The L after the number indicates that the value is a long type.

Using dict_cursor in django

To get a cursor in django I do:
from django.db import connection
cursor = connection.cursor()
How would I get a dict cursor in django, the equivalent of -
import MySQLdb
connection = (establish connection)
dict_cursor = connection.cursor(MySQLdb.cursors.DictCursor)
Is there a way to do this in django? When I tried cursor = connection.cursor(MySQLdb.cursors.DictCursor) I got a Exception Value: cursor() takes exactly 1 argument (2 given). Or do I need to connect directly with the python-mysql driver?
The django docs suggest using dictfetchall:
def dictfetchall(cursor):
"Returns all rows from a cursor as a dict"
desc = cursor.description
return [
dict(zip([col[0] for col in desc], row))
for row in cursor.fetchall()
]
Is there a performance difference between using this and creating a dict_cursor?

No there is no such support for DictCursor in Django.
But you can write a small function to that for you.
See docs: Executing custom SQL directly:
def dictfetchall(cursor):
"Returns all rows from a cursor as a dict"
desc = cursor.description
return [
dict(zip([col[0] for col in desc], row))
for row in cursor.fetchall()
]
>>> cursor.execute("SELECT id, parent_id from test LIMIT 2");
>>> dictfetchall(cursor)
[{'parent_id': None, 'id': 54360982L}, {'parent_id': None, 'id': 54360880L}]

Easily done with Postgres at least, i'm sure mysql has similar ( Django 1.11)
from django.db import connections
from psycopg2.extras import NamedTupleCursor
def scan_tables(app):
conn = connections['default']
conn.ensure_connection()
with conn.connection.cursor(cursor_factory=NamedTupleCursor) as cursor:
cursor.execute("SELECT table_name, column_name "
"FROM information_schema.columns AS c "
"WHERE table_name LIKE '{}_%'".format(app))
columns = cursor.fetchall()
for column in columns:
print(column.table_name, column.column_name)
scan_tables('django')
Obviously feel free to use DictCursor, RealDictCursor, LoggingCursor etc

The following code converts the result set into a dictionary.
from django.db import connections
cursor = connections['default'].cursor()
columns = (x.name for x in cursor.description)
result = cursor.fetchone()
result = dict(zip(columns, result))
If the result set has multiple rows, iterate over the cursor instead.
columns = [x.name for x in cursor.description]
for row in cursor:
row = dict(zip(columns, row))

The main Purpose of using RealDictCursor is to get data in list of dictionary format.
And the apt solution is this and that too without using django ORM
def fun(request):
from django.db import connections
import json
from psycopg2.extras import RealDictCursor
con = connections['default']
con.ensure_connection()
cursor= con.connection.cursor(cursor_factory=RealDictCursor)
cursor.execute("select * from Customer")
columns=cursor.fetchall()
columns=json.dumps(columns)
output:
[{...},{...},{......}]