SQLite output from query into Python script - python

I have this Python script:
s = stdscr.getstr(0,0, 20) #input length last number
c = db.execute("""SELECT "debit" FROM "members" WHERE "barcode" = '%s' LIMIT 1""" % (s,))
for row in c:
print row
if row == '(0,)':
#display cross
print 'Tick'
else:
#display tick
print 'Cross'
Where it is asking for a barcode input, and matching the debit field in the database.
The "print row" command returns "(0,)" but when I try to match it, I always get "Cross" as the output, which is not the intended result. Is there a semantic I'm obviously not observing?
Many thanks!

The variable row is a tuple, and '(0,)' is its string representation. Your are comparing a variable with its string representation, which cannot work.
You need to compare it to the tuple value
if row == (0,):
Simply remove the quote marks.
Alternatively, you can write
if row[0] == 0:
which will avoid the creation of a tuple just for the comparison. As noted by #CL., row will never be an empty tuple so extracting row[0] is safe.

Related

replace for loop to parallel process in pyspark

I am using for loop in my script to call a function for each element of size_DF(data frame) but it is taking lot of time. I tried by removing the for loop by map but i am not getting any output.
size_DF is list of around 300 element which i am fetching from a table.
Using For:
import call_functions
newObject = call_functions.call_functions_class()
size_RDD = sc.parallelize(size_DF)
if len(size_DF) == 0:
print "No record present in the truncated list"
else:
for row in size_DF:
length = row[0]
print "length: ", length
insertDF = newObject.full_item(sc, dataBase, length, end_date)
Using Map
if len(size_DF) == 0:
print "No record present in the list"
else:
size_RDD.mapPartition(lambda l: newObject.full_item(sc, dataBase, len(l[0]), end_date))
newObject.full_item(sc, dataBase, len(l[0]), end_date)
In full_item() -- I am doing some select ope and joining 2 tables and inserting the data into a table.
Please help me and let me know what i am doing wrong.
pyspark.rdd.RDD.mapPartition method is lazily evaluated.
Usually to force an evaluation, you can a method that returns a value on the lazy RDD instance that is returned.
There are higher-level functions that take care of forcing an evaluation of the RDD values. e.g. pyspark.rdd.RDD.foreach
Since you don't really care about the results of the operation you can use pyspark.rdd.RDD.foreach instead of pyspark.rdd.RDD.mapPartition.
def first_of(it):
for first in it:
return first
return []
def insert_first(it):
first = first_of(it)
item_count = len(first)
newObject.full_item(sc, dataBase, item_count, end_date)
if len(size_DF) == 0:
print('No record present in the truncated list')
else:
size_DF.forEach(insert_first)

Retrieve data from sql database using python

I'm a newbie for python and SQL.
My database contain a field which stored path to images.
I want to assign each path to a variable in python.
I can only store only the path of first row.
This is my code
import pymysql
from arcpy import env
env.workspace = "D:/year 4 semester 1/Python/Data/"
conn= pymysql.connect(host='localhost',user='root',password='',db='research')
cursor = conn.cursor ()
cursor.execute ("select Path from sourcedata")
data = cursor.fetchmany()
for row in data :
a= row[0]
print a
But when I try following way including all other relevant text
for row in data :
a= row[0]
b = row[1]
c = row[2]
print a
Following error appear
IndexError: tuple index out of range
you need to indent the print a statement to make it part of the loop. Python is very sensitive about indentation. In other languages you use braces or BEGIN END blocks, but in python you use indentation.
The row[0], row[1] stuff refers to the elements within the retrieved row, not the different rows.
indent the print line and it will print the first returned field, Path, for each record.
The tuple index is out of range because there is only one field (element zero) returned.
Your current code will iterate through every row returned and set a to each one in turn, but will only print the last value for a after it comes out of the loop.

Need to display a text list of records not on my list

I need to search a feature class for multiple text entries and display all the values and their object ids that aren't on the list, i.e. find all the mistakes. (Basically, want to search for text entries such as AVE, TRL, ST, and display entries that aren't formatted like that). I want to write it in python.
Can I use the searchCursor to do this, or is it something more complicated.
Any help would be appreciated, Thanks! I think this is the solution, but it is still printing AVE. Any idea as to why?
import arcpy
fc = "Z:\Street_Centerlines"
field = "StSuffix"
field1 = "OBJECTID"
cursor = arcpy.SearchCursor(fc)
for row in cursor:
if field == "AVE":
pass
else:
print(row.getValue(field1)); print(row.getValue(field))
The field variable is equal to "StSuffix", so field == "AVE" is always false. I think you want this:
valid_values = 'AVE', 'TRL', 'ST'
for row in cursor:
value = row.getValue(field)
if value in valid_values:
continue
print("Invalid value: OBJECTID={}, StSuffix={}".format(
row.getValue(field1),
value
))

Print different value that is stored in MySQL DB

In MySQL table with myISAM I have a integer value ex.011. When I query in Python it prints me value 11 removing 0 before number. It should print the exact value that is stored in DB ex. 011 instead of 11. Any help ?
Your column is an int, so MySQLdb gives you an integer value back in the query result. However, I think you should be able to write a mySQLdb result set wrapper (or maybe find one someone else already wrote) that inspects the flags set on the columns of the result set and casts to a string appropriately.
Look at cursor.description and cursor.description_flags as well as PEP-249. I think (ie I have not actually tried it) something along the lines of the following should get you started:
def get_result_set_with_db_specified_formatting(cursor):
integer_field_types = (MySQLdb.constants.FIELD_TYPE.TINY,
MySQLdb.constants.FIELD_TYPE.SHORT,
MySQLdb.constants.FIELD_TYPE.LONG,
MySQLdb.constants.FIELD_TYPE.LONGLONG,
MySQLdb.constants.FIELD_TYPE.INT24)
rows = cursor.fetchall()
for row in rows:
for index, value in enumerate(row):
value = str(value)
if (cursor.description[index][1] in integer_field_types
and cursor.description_flags[index] & MySQLdb.constants.FLAG.ZEROFILL):
if len(value) < cursor.description[index][2]:
value = ('0' * (cursor.description[index][2] - len(value))) + value
row[index] = value
return rows
May be, simple zero filling is OK in this case?
>>> print str(11).zfill(3)
011
As I understood, it's additional part of number. If its length is not constant, you need to change data type in DB to VARCHAR.

type of the data from mysql

Here is my code in Python:
queryuniq = "SELECT COUNT(distinct src_ip), COUNT(distinct video_id)FROM video"
cur.execute(queryuniq)
uniq = []
uniq = cur.fetchall()
print uniq
ip = str(uniq[0])
video = str(uniq[1])
fd2.write("There are %d ip addresses and %d video in total" %(int(ip), int(video)))
This is the value of "uniq" variable I got:
((2052L, 163581L),)
And this error message:
fd2.write("There are %d ip addresses in total" %(int(ip)))
ValueError: invalid literal for int() with base 10: '((2052L,),)'
video = str(uniq[1])
IndexError: tuple index out of range
I just simply want to count the distinct items in a column in the database, and print the INT value in a file.
Can anyone explain why the SELECT command return a weird data format like ((2052L, 163581L),) ? Don't understand why there is a "L"after the number..
How can I solve this problem? Many thanks!
uniq is a tuple of tuples (each entry at the outer level represents a database row, within which there is a tuple of column values).
You query always returns one row. Therefore the outer tuple always contains one element, and you could fix your code by replacing:
uniq = cur.fetchall()
with
uniq = cur.fetchall()[0]
Also, the conversions from int to string and then back to int are unnecessary.
To summarize, the following is a tidied up version of your code:
queryuniq = "SELECT COUNT(distinct src_ip), COUNT(distinct video_id)FROM video"
cur.execute(queryuniq)
uniq = cur.fetchall()[0]
ip, video = uniq
fd2.write("There are %d ip addresses and %d video in total" %(ip, video))
There several things wrong with your code.
Firstly, cur.fetchall() - as the name implies - fetches all the results from the query. Since Python does not know that your query only returns a single row, it still returns a tuple of all rows. So uniq[0] does not refer to the first field in the row, it refers to the first row in the result.
Since you know you only want one row, you could use cur.fetchone().
Secondly, why are you converting the results to strings then converting them back to ints? That seems pointless. They are in the correct format already - L just means they are 'long ints'.

Categories

Resources