I have some Python code the selects data from Oracle spatial and inserts into Spatialite. My problem is that the cursor contains the geometry in binary and I can’t figure out how to read the binary into the Spatialite insert statement. Just to added this all works if I use WKT but some of the geometries are too long hence the reason for the binary format.
Can anyone help please?
# Import system modules
import cx_Oracle
from pyspatialite import dbapi2 as sl_db
def db_connect():
# Build connect from TNS names
o_db = cx_Oracle.connect("xxxxx", "xxxxx", "xxxxx_gl_dev")
cursor = o_db.cursor()
return cursor
def db_lookup(cursor):
# Select records
sql = "SELECT sdo_util.to_wkbgeometry(a.shape), a.objectid FROM span a WHERE a.objectid = 1382372"
cursor.execute(sql)
row = cursor.fetchall()
return row
def db_insert(row):
# Insert Rows in new spatailite table
database_name = 'C:\\Temp\\MYDATABASE.sqlite'
db_connection = sl_db.connect(database_name)
db_cursor = db_connection.cursor()
sql = 'INSERT INTO "SPAN_OFL" ("geometry", "OBJECTID") Values GeomFromWKB(?,27700),?);'
db_cursor.executemany(sql, row)
db_connection.commit()
db_connection.close()
# main code
cursor = db_connect()
row = db_lookup(cursor)
db_insert(row)
Related
I have a problem getting the query results from my Python-Code. The connection to the database seems to work, but i always get the error:
"InterfaceError: No result set to fetch from."
Can somebody help me with my problem? Thank you!!!
cnx = mysql.connector.connect(
host="127.0.0.1" ,
user="root" ,
passwd="*****",
db="testdb"
)
cursor = cnx.cursor()
query = ("Select * from employee ;")
cursor.execute(query)
row = cursor.fetchall()
If your problem is still not solved, you can consider replacing the python mysql driver package and use pymysql.
You can write code like this
#!/usr/bin/python
import pymysql
db = pymysql.connect(host="localhost", # your host, usually localhost
user="test", # your username
passwd="test", # your password
db="test") # name of the data base
# you must create a Cursor object. It will let
# you execute all the queries you need
cur = db.cursor()
query = ("SELECT * FROM employee")
# Use all the SQL you like
cur.execute(query)
# print all the first cell of all the rows
for row in cur.fetchall():
print(row[0])
db.close()
This should be able to find the result you want
add this to your code
for i in row:
print(i)
you did not print anything which is why that's not working
this will print each row in separate line
first try to print(row),if it fails try to execute using the for the loop,remove the semicolon in the select query statement
cursor = connection.cursor()
rows = cursor.execute('SELECT * FROM [DBname].[dbo].TableName where update_status is null ').fetchall()
for row in rows:
ds = row[0]
state = row[1]
here row[0] represent the first columnname in the database
& row[1] represent the second columnname in the database & so on
I am using this sample sqlite database and my code is
import sqlite3
conn = sqlite3.connect('chinook.db')
conn.execute("SELECT * FROM tracks")
rows = conn.cursor().fetchall()
It should have worked, but rows is empty?
What am I doing wrong here?
The Connection.execute shortcut returns a cursor instance, which you need to use with fetchall. In your code, you're creating a new, independent cursor.
Thus:
import sqlite3
conn = sqlite3.connect('chinook.db')
cursor = conn.execute("SELECT * FROM tracks")
rows = cursor.fetchall()
or even shorter (not recommended, but for those who like obscured one-liners):
rows = sqlite3.connect('chinook.db').execute("SELECT * FROM tracks").fetchall()
Or don't use Connection.execute shortcut, to avoid confusion:
import sqlite3
conn = sqlite3.connect('chinook.db')
cursor = conn.cursor()
cursor.execute("SELECT * FROM tracks")
rows = cursor.fetchall()
I am trying to execute the following script. but I don't get neither the desired results nor a error message ,and I can't figure out where I'm doing wrong.
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=mySRVERNAME;"
"Database=MYDB;"
"uid=sa;pwd=MYPWD;"
"Trusted_Connection=yes;")
cursor = cnxn.cursor()
cursor.execute('select DISTINCT firstname,lastname,coalesce(middlename,\' \') as middlename from Person.Person')
for row in cursor:
print('row = %r' % (row,))
any ideas ? any help is appreciated :)
You have to use a fetch method along with cursor. For Example
for row in cursor.fetchall():
print('row = %r' % (row,))
EDIT :
The fetchall function returns all remaining rows in a list.
If there are no rows, an empty list is returned.
If there are a lot of rows, *this will use a lot of memory.*
Unread rows are stored by the database driver in a compact format and are often sent in batches from the database server.
Reading in only the rows you need at one time will save a lot of memory.
If we are going to process the rows one at a time, we can use the cursor itself as an interator
Moreover we can simplify it since cursor.execute() always returns a cursor :
for row in cursor.execute("select bla, anotherbla from blabla"):
print row.bla, row.anotherbla
Documentation
I found this information useful to retrieve data from SQL database to python as a data frame.
import pandas as pd
import pymssql
con = pymssql.connect(server='use-et-aiml-cloudforte-aiops- db.database.windows.net',user='login_username',password='login_password',database='database_name')
cursor = con.cursor()
query = "SELECT * FROM <TABLE_NAME>"
cursor.execute(query)
df = pd.read_sql(query, con)
con.close()
df
import mysql.connector as mc
connection creation
conn = mc.connect(host='localhost', user='root', passwd='password')
print(conn)
#create cursor object
cur = conn.cursor()
print(cur)
cur.execute('show databases')
for i in cur:
print(i)
query = "Select * from employee_performance.employ_mod_recent"
emp_data = pd.read_sql(query, conn)
emp_data
I am creating two dataframes from MySQL database query results, later I join them on a column and get the final result.
Below is the code I currently have
import MySQLdb
from sqlalchemy import create_engine
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="root", # your username
passwd="M#nUtd10", # your password
db="Mediator") # name of the data base
cur = db.cursor()
from pandas import DataFrame
cur.execute("SELECT invoice_ref_2 AS PPS_Id ,cast(total_amount AS char(10)) AS Amount FROM core_invoices WHERE client_ref_id like 'payment_gc_sale%' AND partition_key = 20161129")
invoices_res = DataFrame(cur.fetchall())
invoices_res.columns = cur.keys()
cur.execute("SELECT invoice_ref_2 AS PPS_Id ,cast(total_amount AS char(10)) AS Amount FROM core_invoices WHERE client_ref_id like 'payment_gc_sale%' AND partition_key = 20161129")
accrual_res = DataFrame(cur.fetchall())
accrual_res.columns = cur.keys()
final_res = invoices_res.merge(accrual_res, on='PPS_Id')
db.close()
When I execute the code I get,
"PandasError: DataFrame constructor not properly called!"
Please assist.
I am trying to retrieve data from an SQL server using pyodbc and print it in a table using Python. However, I can only seem to retrieve the column name and the data type and stuff like that, not the actual data values in each row of the column.
Basically I am trying to replicate an Excel sheet that retrieves server data and displays it in a table. I am not having any trouble connecting to the server, just that I can't seem to find the actual data that goes into the table.
Here is an example of my code:
import pyodbc
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=SQLSRV01;DATABASE=DATABASE;UID=USER;PWD=PASSWORD')
cursor = cnxn.cursor()
cursor.execute("SELECT * FROM sys.tables")
tables = cursor.fetchall()
#cursor.execute("SELECT WORK_ORDER.TYPE,WORK_ORDER.STATUS, WORK_ORDER.BASE_ID, WORK_ORDER.LOT_ID FROM WORK_ORDER")
for row in cursor.columns(table='WORK_ORDER'):
print row.column_name
for field in row:
print field
However the result of this just gives me things like the table name, the column names, and some integers and 'None's and things like that that aren't of interest to me:
STATUS_EFF_DATE
DATABASE
dbo
WORK_ORDER
STATUS_EFF_DATE
93
datetime
23
16
3
None
0
None
None
9
3
None
80
NO
61
So I'm not really sure where I can get the values to fill up my table. Would it should be in table='WORK_ORDER', but could it be under a different table name? Is there a way of printing the data that I am just missing?
Any advice or suggestions would be greatly appreciated.
You are so close!
import pyodbc
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=SQLSRV01;DATABASE=DATABASE;UID=USER;PWD=PASSWORD')
cursor = cnxn.cursor()
cursor.execute("SELECT WORK_ORDER.TYPE,WORK_ORDER.STATUS, WORK_ORDER.BASE_ID, WORK_ORDER.LOT_ID FROM WORK_ORDER")
for row in cursor.fetchall():
print row
(the "columns()" function collects meta-data about the columns in the named table, as opposed to the actual data).
you could try using Pandas to retrieve information and get it as dataframe
import pyodbc as cnn
import pandas as pd
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=SQLSRV01;DATABASE=DATABASE;UID=USER;PWD=PASSWORD')
# Copy to Clipboard for paste in Excel sheet
def copia (argumento):
df=pd.DataFrame(argumento)
df.to_clipboard(index=False,header=True)
tableResult = pd.read_sql("SELECT * FROM YOURTABLE", cnxn)
# Copy to Clipboard
copia(tableResult)
# Or create a Excel file with the results
df=pd.DataFrame(tableResult)
df.to_excel("FileExample.xlsx",sheet_name='Results')
I hope this helps!
Cheers!
In order to receive actual data stored in the table, you should use one of fetch...() functions or use the cursor as an iterator (i.e. "for row in cursor"...). This is described in the documentation:
cursor.execute("select user_id, user_name from users where user_id < 100")
rows = cursor.fetchall()
for row in rows:
print row.user_id, row.user_name
Just do this:
import pandas as pd
import pyodbc
cnxn = pyodbc.connect("Driver={SQL Server}\
;Server=SERVER_NAME\
;Database=DATABASE_NAME\
;Trusted_Connection=yes")
df = pd.read_sql("SELECT * FROM myTableName", cnxn)
df.head()
Instead of using the pyodbc library, use the pypyodbc library... This worked for me.
import pypyodbc
conn = pypyodbc.connect("DRIVER={SQL Server};"
"SERVER=server;"
"DATABASE=database;"
"Trusted_Connection=yes;")
cursor = conn.cursor()
cursor.execute('SELECT * FROM [table]')
for row in cursor:
print('row = %r' % (row,))
import pyodbc
conn = pyodbc.connect('Driver={SQL Server};'
'Server=db-server;'
'Database=db;'
'Trusted_Connection=yes;')
sql = "SELECT * FROM [mytable] "
cursor.execute(sql)
for r in cursor:
print(r)
Why pyodbc you can try with pymssql. For more information follow this link: https://stackoverflow.com/a/70445445/8614314.
import pandas as pd
import pymssql
con = pymssql.connect(<conncetion to the server and db>)
cursor = con.cursor()
query = "<Your query>"
cursor.execute(query)
df = pd.read_sql(query, con)
con.close()
Upvoted answer din't work for me, It was fixed by editing connection line as follows(replace semicolons with coma and also remove those quotes):
import pyodbc
cnxn = pyodbc.connect(DRIVER='{SQL Server}',SERVER=SQLSRV01,DATABASE=DATABASE,UID=USER,PWD=PASSWORD)
cursor = cnxn.cursor()
cursor.execute("SELECT WORK_ORDER.TYPE,WORK_ORDER.STATUS, WORK_ORDER.BASE_ID, WORK_ORDER.LOT_ID FROM WORK_ORDER")
for row in cursor.fetchall():
print row