Develop Reference

AttributeError: 'Engine' object has no attribute 'conn'

So I am trying to create an auto update to SQL from another excel file, by unique value, as to know what is the new data to add to the database..
There's different in columns names between the database and the excel file as in the database and names without spaces...
I tried to do it with pandas it gave me the same error
So here's my simple code tried with xlrd
import xlrd
from sqlalchemy import create_engine
def insert():
book = xlrd.open_workbook(r"MNM_Rotterdam_5_Daily_Details-20191216081027 - Copy (2).xlsx")
sheet = book.sheet_by_name("GSM Details")
database = create_engine(
'mssql+pyodbc://WWX542337CDCD\SMARTRNO_EXPRESS/myDB?driver=SQL+Server+Native+Client+11.0') # name of database
cnxn = database.raw_connection
cursor = cnxn.cursor()
query = """Insert INTO [myDB].[dbo].[mnm_rotterdam_5_daily_details-20191216081027] (Date, SiteName, CellCI, CellLAC, CellName, CellIndex) values (?,?,?,?,?,?)"""
for r in range(1, sheet.nrows):
date = sheet.cell(r,0).value
site_name = sheet.cell(r,3).value
cell_ci = sheet.cell(r,4).value
cell_lac = sheet.cell(r,5).value
cell_name = sheet.cell(r,6).value
cell_index = sheet.cell(r,7).value
values = (date, site_name, cell_ci, cell_lac, cell_name, cell_index)
cursor.execute(query, values)
cnxn.commit()
# Close the cursor
cursor.close()
# Commit the transaction
database.commit()
# Close the database connection
database.close()
# Print results
print ("")
print ("")
columns = str(sheet.ncols)
rows = str(sheet.nrows)
print ("Imported", columns,"columns and", rows, "rows. All Done!")
insert()
and this is the error:
I tried to change the range I found another error:
Traceback (most recent call last):
File "D:/Tooling/20200207/uniquebcon.py", line 48, in <module>
insert()
File "D:/Tooling/20200207/uniquebcon.py", line 37, in insert
database.commit()
AttributeError: 'Engine' object has no attribute 'commit'
I think this is related to SQL-Alchemy in the connection

Instead of creating the cursor directly with
cursor = database.raw_connection().cursor()
you can create a connection object, then create the cursor from that, and then call .commit() on the connection:
cnxn = database.raw_connection()
crsr = cnxn.cursor()
# do stuff with crsr ...
cnxn.commit()

cursor.execute returns 1 int

When running a query to a mysql database using MySqlHook, cursor.execute(query) returns int 1
My code is
import logging
from airflow.hooks.mysql_hook import MySqlHook
query = "SELECT col1, col2 FROM myschema.mytable LIMIT 1"
mysql = MySqlHook(mysql_conn_id=conn_id)
conn = mysql.get_conn()
cursor = conn.cursor()
result_cursor = cursor.execute(query)
logging.info(result_cursor) # this prints out "INFO - 1" in the log
df = pd.DataFrame(result_cursor.fetchall(), columns=result_cursor.keys()) # this triggers error "ERROR - 'int' object has no attribute 'fetchall'"
I would have expected result_cursor to return a "fetchable" result, since the query is working fine.

Cursor.execute() return value is not defined by the db-api spec, but for most implementations it returns the number of rows affected by the query.
To retrieve data, you have to either iterate over the cursor or call .fetchall().

It seems I cannot save cursor.execute(query) into variable result_cursor.
To make the code work, I simply needed to define the data for the data-frame as cursor.fetchall()
cursor.execute(query)
df = pd.DataFrame(list(cursor.fetchall()), column=[col[0] for col in cursor.description])

Get list without JSON-like style mysql python

I am building a POS (point of sale) in python with MySQL!
I want to list all articles, but I get the JSON-like output.
My code:
import mysql.connector
print('eXperience POS\nArtikli:')
posdb = mysql.connector.connect\
(
user='root',
password='',
host='127.0.0.1',
database='experiencepos'
)
try:
cursor = posdb.cursor()
cursor.execute('SELECT * FROM artikli')
sviartikli = cursor.fetchall()
print(sviartikli)
finally:
posdb.close()

I think you're confusing JSON output with it printing a dict. Each item in the cursor returns a dict containing the item's fields. Since you did not show the example output I'll just use a name and description field for example. This is what I think you're looking for though. Also you may want to just loop through the cursor as printing the entire cursor would just print a list of dicts with all the items.
import mysql.connector
print('eXperience POS\nArtikli:')
posdb = mysql.connector.connect\
(
user='root',
password='',
host='127.0.0.1',
database='experiencepos'
)
try:
cursor = posdb.cursor()
cursor.execute('SELECT * FROM artikli')
for sviartikli in cursor:
name = sviartikli['name'] #example field
description = sviartikli['description'] #another example field
print("Name: {} Description: {}")
finally:
posdb.close()

Retrieve data from sql server database using Python

I am trying to execute the following script. but I don't get neither the desired results nor a error message ,and I can't figure out where I'm doing wrong.
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=mySRVERNAME;"
"Database=MYDB;"
"uid=sa;pwd=MYPWD;"
"Trusted_Connection=yes;")
cursor = cnxn.cursor()
cursor.execute('select DISTINCT firstname,lastname,coalesce(middlename,\' \') as middlename from Person.Person')
for row in cursor:
print('row = %r' % (row,))
any ideas ? any help is appreciated :)

You have to use a fetch method along with cursor. For Example
for row in cursor.fetchall():
print('row = %r' % (row,))
EDIT :
The fetchall function returns all remaining rows in a list.
If there are no rows, an empty list is returned.
If there are a lot of rows, *this will use a lot of memory.*
Unread rows are stored by the database driver in a compact format and are often sent in batches from the database server.
Reading in only the rows you need at one time will save a lot of memory.
If we are going to process the rows one at a time, we can use the cursor itself as an interator
Moreover we can simplify it since cursor.execute() always returns a cursor :
for row in cursor.execute("select bla, anotherbla from blabla"):
print row.bla, row.anotherbla
Documentation

I found this information useful to retrieve data from SQL database to python as a data frame.
import pandas as pd
import pymssql
con = pymssql.connect(server='use-et-aiml-cloudforte-aiops- db.database.windows.net',user='login_username',password='login_password',database='database_name')
cursor = con.cursor()
query = "SELECT * FROM <TABLE_NAME>"
cursor.execute(query)
df = pd.read_sql(query, con)
con.close()
df

import mysql.connector as mc
connection creation
conn = mc.connect(host='localhost', user='root', passwd='password')
print(conn)
#create cursor object
cur = conn.cursor()
print(cur)
cur.execute('show databases')
for i in cur:
print(i)
query = "Select * from employee_performance.employ_mod_recent"
emp_data = pd.read_sql(query, conn)
emp_data

Rows as dictionaries - pymssql

I want to receive rows as dictionaries in pymssql. in python-idle i ran:
>>> conn = pymssql.connect(host='192.168.1.3', user='majid', password='123456789', database='GeneralTrafficMonitor', as_dict=True)
>>> cur = conn.cursor()
>>> cur.execute('SELECT TOP 10 * FROM dbo.tblTrafficCounterData')
>>> cur.as_dict
True
>>> for row in cur:
print row['ID']
But it gives:
Traceback (most recent call last):
File "<pyshell#83>", line 2, in <module>
print row['ID']
TypeError: tuple indices must be integers, not str
Could someone help?

You need to add the parameter as_dict=True like so:
cursor = conn.cursor(as_dict=True)
You will then be able to access row['id'] if it is a field name in the result set.
As per documentation below:
https://pythonhosted.org/pymssql/pymssql_examples.html#rows-as-dictionaries

Look at the version of pymssql that you are using. Only since 1.0.2 does it return a dict, earlier versions seem to return tuple.

It is possible to set as_dict=True while creating the connection itself.
pymssql.connect(server='.', user='', password='', database='', as_dict=True,)
https://pythonhosted.org/pymssql/ref/pymssql.html?highlight=connect#pymssql.connect

Specifying results as dictionaries can be done on a cursor by cursor basis:
import pymysql
from pymysql.cursors import DictCursor
# create database connection
# connect to database
mydb = pymysql.connect( ... )
mycursor = mydb.cursor(cursor=DictCursor)