I am writing python script to fetch mysql db rows and copy to excel file
The Code I am trying on 2.7.6 :
from __future__ import print_function
import MySQLdb as mdb
conn = mdb.connect("localhost","username","passwd","db")
table = open("table.csv" , 'w')
with conn:
query = "SELECT NAME FROM mytable WHERE H_ID= 5 ORDER BY NAME;"
cursor = conn.cursor()
cursor.execute(query)
for item in range(cursor.rowcount):
row = cursor.fetchone()
print(row[0],file=table)
If I execute that query in mysql means I will get 446 rows and while printing on python shell [ idle ] also I will get proper result , But when I try to copy the output to file I am able to see only 341 rows. What may be the problem.?
I Don't know what exactly the problem, But It got resolved when I remove with keyword and closing the file manually ,
Below code works properly.
from __future__ import print_function
import MySQLdb as mdb
conn = mdb.connect("localhost","username","passwd","db")
table = open("table.csv" , 'w')
query = "SELECT NAME FROM mytable WHERE H_ID= 5 ORDER BY NAME;"
cursor = conn.cursor()
cursor.execute(query)
for item in range(cursor.rowcount):
row = cursor.fetchone()
print(row[0],file=table)
table.close()
From the doc of Python connector for cursor.rowcount:
For nonbuffered cursors, the row count cannot be known before the rows have been fetched. In this case, the number of rows is -1 immediately after query > execution and is incremented as rows are fetched.
Therefore, the block for for item in range(cursor.rowcount): will not be executed.
Try:
for row in cursor:
print(row[0], file=table)
Related
I have the following python code, it reads through a text file line by line and takes characters x to y of each line as the variable "Contract".
import os
import pyodbc
cnxn = pyodbc.connect(r'DRIVER={SQL Server};CENSORED;Trusted_Connection=yes;')
cursor = cnxn.cursor()
claimsfile = open('claims.txt','r')
for line in claimsfile:
#ldata = claimsfile.readline()
contract = line[18:26]
print(contract)
cursor.execute("USE calms SELECT XREF_PLAN_CODE FROM calms_schema.APP_QUOTE WHERE APPLICATION_ID = "+str(contract))
print(cursor.fetchall())
When including the line cursor.fetchall(), the following error is returned:
Programming Error: Previous SQL was not a query.
The query runs in SSMS and replace str(contract) with the actual value of the variable results will be returned as expected.
Based on the data, the query will return one value as a result formatted as NVARCHAR(4).
Most other examples have variables declared prior to the loop and the proposed solution is to set NO COUNT on, this does not apply to my problem so I am slightly lost.
P.S. I have also put the query in its own standalone file without the loop to iterate through the file in case this was causing the problem without success.
In your SQL query, you are actually making two commands: USE and SELECT and the cursor is not set up with multiple statements. Plus, with database connections, you should be selecting the database schema in the connection string (i.e., DATABASE argument), so TSQL's USE is not needed.
Consider the following adjustment with parameterization where APPLICATION_ID is assumed to be integer type. Add credentials as needed:
constr = 'DRIVER={SQL Server};SERVER=CENSORED;Trusted_Connection=yes;' \
'DATABASE=calms;UID=username;PWD=password'
cnxn = pyodbc.connect(constr)
cur = cnxn.cursor()
with open('claims.txt','r') as f:
for line in f:
contract = line[18:26]
print(contract)
# EXECUTE QUERY
cur.execute("SELECT XREF_PLAN_CODE FROM APP_QUOTE WHERE APPLICATION_ID = ?",
[int(contract)])
# FETCH ROWS ITERATIVELY
for row in cur.fetchall():
print(row)
cur.close()
cnxn.close()
I am connecting to mysql database via mysql connector and running a simple query to pull a list of IDs. I need to loop over that list and pass them into some other code. For some reason I am getting a list of tuples. Is this expected behavior? If not, what am I doing wrong?
Here is the snippet of my code:
import mysql.connector
conn = mysql.connector.connect(host='127.0.0.1', database='t', user='r', password='pwd')
cursor = conn.cursor()
query = ( "select id from T where updated < '%s'" % (run_date) )
cursor.execute(query)
for row in cursor:
print (row)
cursor.close()
I am getting the following back (from an INT field in d/b):
(Decimal('991837'),)
(Decimal('991838'),)
(Decimal('991839'),)
(Decimal('991871'),)
(Decimal('991879'),)
(Decimal('991899'),)
(Decimal('992051'),)
(Decimal('992299'),)
(Decimal('992309'),)
if you want to access just the data in the row you need to go into the dictionary
first you must make it true in the cursor
cur = db.cursor( buffered=True , dictionary=True)
then the result will be like this :
{'Decimal' : '991837'}
i'm sure the Decimal is your row name
so when you need to access to the value do this
import mysql.connector
conn = mysql.connector.connect(host='127.0.0.1', database='t', user='r', password='pwd')
cursor = conn.cursor()
query = ( "select id from T where updated < '%s'" % (run_date) )
cursor.execute(query)
for row in cursor:
print (row['Decimal'])
cursor.close()
i hope it works for i was looking for this solution for the past 2 days and no answers
the only way i debugged i opened the debugger and print out all the variables
have fun with Python :)
Yes, this is expected behavior. Using the cursor as an iterable is basically equivalent to looping over it using the fetchone() method. From the documentation for fetchone() (emphasis mine):
This method retrieves the next row of a query result set and returns a
single sequence, or None if no more rows are available. By default,
the returned tuple consists of data returned by the MySQL server,
converted to Python objects. If the cursor is a raw cursor, no such
conversion occurs;
I have a database named "sina2013",and the columus is Title,Content
Now I want to use pymssql module to get the data.At the same time ,using the Title as the filename of a txt file,the Content as the content of the txt file.
The strange thing is the number of files is less than the items in database.
where is the error?
the code i have tried is:
import pymssql
conn = pymssql.connect(...)
cur = conn.cursor()
cur.execute('SELECT Title,Content FROM sina2013')
count=len(cur.fetchall()) #Will return the right number :5913
for Title,Content in cur:
filename=file(str(Title)+r'.txt',r'w')
filename.write(Content )
filename.close()
cur.close()
The number of txt file is less than it should be.
what is the reason?
Perhaps changing your for loop into this:
# cursor fetchall() method returns all rows from a query
for Title,Content in cur.fetchall():
... will fix the issue?
I am currently connecting to a Sybase 15.7 server using sybpydb. It seems to connect fine:
import sys
sys.path.append('/dba/sybase/ase/15.7/OCS-15_0/python/python26_64r/lib')
sys.path.append('/dba/sybase/ase/15.7/OCS-15_0/lib')
import sybpydb
conn = sybpydb.connect(user='usr', password='pass', servername='serv')
is working fine. Changing any of my connection details results in a connection error.
I then select a database:
curr = conn.cursor()
curr.execute('use db_1')
however, now when I try to run queries, it always returns None
print curr.execute('select * from table_1')
I have tried running the use and select queries in the same execute, I have tried including go commands after each, I have tried using curr.connection.commit() after each, all with no success. I have confirmed, using dbartisan and isql, that the same queries I am using return entries.
Why am I not getting results from my queries in python?
EDIT:
Just some additional info. In order to get the sybpydb import to work, I had to change two environment variables. I added the lib paths (the same ones that I added to sys.path) to $LD_LIBRARY_PATH, i.e.:
setenv LD_LIBRARY_PATH "$LD_LIBRARY_PATH":dba/sybase/ase/15.7/OCS-15_0/python/python26_64r/lib:/dba/sybase/ase/15.7/OCS-15_0/lib
and I had to change the SYBASE path from 12.5 to 15.7. All this was done in csh.
If I print conn.error(), after every curr.execute(), I get:
("Server message: number(5701) severity(10) state(2) line(0)\n\tChanged database context to 'master'.\n\n", 5701)
I completely understand where you might be confused by the documentation. Its doesn't seem to be on par with other db extensions (e.g. psycopg2).
When connecting with most standard db extensions you can specify a database. Then, when you want to get the data back from a SELECT query, you either use fetch (an ok way to do it) or the iterator (the more pythonic way to do it).
import sybpydb as sybase
conn = sybase.connect(user='usr', password='pass', servername='serv')
cur = conn.cursor()
cur.execute("use db_1")
cur.execute("SELECT * FROM table_1")
print "Query Returned %d row(s)" % cur.rowcount
for row in cur:
print row
# Alternate less-pythonic way to read query results
# for row in cur.fetchall():
# print row
Give that a try and let us know if it works.
Python 3.x working solution:
import sybpydb
try:
conn = sybpydb.connect(dsn="Servername=serv;Username=usr;Password=pass")
cur = conn.cursor()
cur.execute('select * from db_1..table_1')
# table header
header = tuple(col[0] for col in cur.description)
print('\t'.join(header))
print('-' * 60)
res = cur.fetchall()
for row in res:
line = '\t'.join(str(col) for col in row)
print(line)
cur.close()
conn.close()
except sybpydb.Error:
for err in cur.connection.messages:
print(f'Error {err[0]}, Value {err[1]}')
So I have a mysql table and I am trying to take each element from one of the fields of the table. The field and table are both called "keywords". In the field there are many different random words and I am trying to take all of those and save them to a text file. Any help on how to implement this would be great, here is what I have so far.
#!/usr/bin/env python
import MySQLdb
db = MySQLdb.connect(host="", user="", passwd="", db="")
cursor = db.cursor()
sql = """SELECT DISTINCT keywords FROM keywords"""
cursor.execute(sql)
cursor.fetchall()
db.close()
for s in sql:
tweets = open("keywords.txt", "w")
What I was thinking is to turn what sql fetches into a list if possible and write that to the file. But I am open to any suggestions, thanks.
Something like this should work:
import MySQLdb
db = MySQLdb.connect(host="", user="", passwd="", db="")
cursor = db.cursor()
sql = """SELECT DISTINCT keywords FROM keywords"""
tweets = open("keywords.txt", "w")
cursor.execute(sql)
for row in cursor:
print>>tweets, row[0]
tweets.close()
db.close()