Beginner here.
I have the following circumstances.
A text file with each line containing a name.
A cassandra 3.5 database
A python script
The intention is to have the script read from the file one line (one name) at a time, and query Cassandra with that name.
FYI, everything works fine except for when I try to pass the value of the list to the query.
I current have something like:
#... driver import, datetime imports done above
#...
with open(fname) as f:
content = f.readlines()
# Loop for each line from the number of lines in the name list file
# num_of_lines is already set
for x in range(num_of_lines):
tagname = str(content[x])
rows = session.execute("""SELECT * FROM tablename where name = %s and date = %s order by time desc limit 1""", (tagname, startDay))
for row in rows:
print row.name + ", " + str(row.date)
Everything works fine if I remove the tagname list component and edit the query itself with a name value.
What am I doing wrong here?
Simply building on the answer from #Vinny above, format simply replaces literal value. You need to put quotes around it.
for x in content:
rows = session.execute("SELECT * FROM tablename where name ='{}' and date ='{}' order by time desc limit 1".format(x, startDay))
for row in rows:
print row.name + ", " + str(row.date)
You can simply iterate over content:
for x in content:
rows = session.execute("SELECT * FROM tablename where name = {} and date = {} order by time desc limit 1".format(x, startDay))
for row in rows:
print row.name + ", " + str(row.date)
....
Also, you don't need to have 3 quotes for the string. Single quotes is good enough (3 quotes is used for documentation / multiple line comments in python)
Note that this might end in a different error; but you will be iterating on the lines instead of iterating over an index and reading lines.
Related
'Code which i have written'
'''
row = crsr.execute(select hostname, employee_add1, employee_add2 from table)
records = crsr.fetchmany(size)
with open(r"samplefile.txt", "w") as f:
for row in records:
print (str(row))
'''
'from this select statement i want to get the value of hostname and append this value to the filename.txt. Filename should be something like this'
'''samplefile-datetime-hostname'''
'How to sort this out?'
Maybe something like:
crsr.execute("select hostname, employee_add1, employee_add2 from dual")
records = crsr.fetchmany(size)
for h,a1,a2 in records:
print(h+"-samplefile.txt")
It wasn't clear whether you wanted to open one file total, or one file per table row, but you can easily add an open() outside or inside the loop.
You can also import datetime and get a date to concatenate into the string.
You may want to strip leading and trailing whitespace from the hostname column value.
I am trying to write user data from a file into a series of insert statements. I feel I am close but just missing one or two things. I am attempting to run a .format, but all I end up with are ?'s
import time, json, sqlite3
def insertsfromfile(file):
results = open(file).readlines()
output = open('UserINSERTFile.txt', 'w')
for rows in results:
jsonobject = json.loads(rows)
userid = jsonobject['user']['id']
name = jsonobject['user']['name']
screenname = jsonobject['user']['screen_name']
description = jsonobject['user']['description']
friendscount = jsonobject['user']['friends_count']
insert = ('INSERT INTO Users VALUES (?,?,?,?,?'.format(userid, name, screenname,description, friendscount)
insert = insert[:-1] + ''
output.write(insert)
output.close()
Thanks
I figured it out after reviewing it. Essentially I was missing that I had to combine the attributes together with my Insert string with the '+'. Also had to convert the variables to str() in case they were int.
I have a script that's parsing through text within each record of a table/field, and inserting those values into field/records of another table. Here's an example of the text being parsed (from "Field1"):
Name Multiple Words (0/24.56) Name Multiple Words2* (96/24.56) Name Multiple Words3* (0) Name Multiple Words4* (96/12.58) Name Multiple Words5* (96/78.12) Name Multiple Words6* (0/98.32) Name Multiple Words7* (96/0) Name Multiple Words8* (0) Name Multiple Words9**
Here's the script:
import re, arcpy, sys
# Local variables:
Table2 = "D:\Source_Data_Convert.gdb\Table2"
RAW_Data = "D:\Source_Data_Convert.gdb\RAW_Data_Table2"
#Create Cursors and Insert Rows
insertcursor = arcpy.da.InsertCursor(Table2, ["Other_Stuff", "That_Stuff", "Some_More", "From", "To", "Num_One", "Num_Two", "Nums"])
with arcpy.da.SearchCursor(RAW_Data, ["Field1", "Other_Stuff", "That_Stuff", "Some_More"]) as searchcursor:
######## Start the SearchCursor Loop ###################
for row in searchcursor:
try:
Other_Stuff = row[1]
That_Stuff = row[2]
Some_More = row[3]
listFrom = re.split(r'\*\s*\(.*?\)\s*', row[0])
print listFrom
Nums = re.findall(r'\(([^)]+)\)', row[0])
for match in re.finditer(r'\(([^)]+)\)', row[0]):
parts = match.group(1).split('/')
print parts
First_Num = parts[0]
try:
Second_Num = parts[1]
except IndexError:
#Second_Num = None
Second_Num = 0
print "First_Num, Second_Num: ", First_Num, Second_Num
print "Parsing Successful"
############ Start the Insertcursor Loop ############
for n,Value in enumerate(match): #enumerate is essentially doing a count
insertcursor.insertRow((Other_Stuff, That_Stuff, Some_More, listFrom[n], listFrom[n+1], First_Num[n], Second_Num[n+1], Nums[n]))
print "Data Inserted"
except:
pass
else:
break
del insertcursor
del searchcursor
del row
I can get the values to print correctly, but I can't seem to get the (2) values from the nested loop to insert with each (1) value from the outer loop. Hopefully my question makes sense. Basically I'm trying to populate (3) fields. One field gets the values from the outer loop, the other two fields get the values (resulting from the split) from the inner loop. Any help would much appreciated. Thanks.
Compile everything that you want to print or handle into a list of dictionary items. This way you can group up data that you want to print more easily.
So you can have something like this:
field_dict['firstField'] = someValue
field_dict['secondField'] = someList
field_dict['thirdField'] = someValue
dict_list.append(field_dict)
Then at the end you can just loop through the list and print however you want without being bound by your nested loops.
I have a text file that contains many different entries. What I'd like to do is take the first column, use each unique value as a key, and then store the second column as values. I actually have this working, sort of, but I'm looking for a better way to do this. Here is my example file:
account_check:"login/auth/broken"
adobe_air_installed:kb_base+"/"+app_name+"/Path"
adobe_air_installed:kb_base+"/"+app_name+"/Version"
adobe_audition_installed:'SMB/Adobe_Audition/'+version+'/Path'
adobe_audition_installed:'SMB/Adobe_Audition/'+version+'/ExePath'
Here is the code I'm using to parse my text file:
val_dict = {}
for row in creader:
try:
value = val_dict[row[0]]
value += row[1] + ", "
except KeyError:
value = row[1] + ", "
val_dict[row[0]] = value
for row in val_dict.items():
values = row[1][:-1],row[0]
cursor.execute("UPDATE 'plugins' SET 'sets_kb_item'= ? WHERE filename= ?", values)
And here is the code I use to query + format the data currently:
def kb_item(query):
db = get_db()
cur = db.execute("select * from plugins where sets_kb_item like ?", (query,))
plugins = cur.fetchall()
for item in plugins:
for i in item['sets_kb_item'].split(','):
print i.strip()
Here is the output:
kb_base+"/Installed"
kb_base+"/Path"
kb_base+"/Version"
It took me many tries but I finally got the output the way I wanted it, however I'm looking for critique. Is there a better way to do this? Could my entire for item in plugins.... print i.strip() be done in one line and saved as a variable? I am very new to working with databases, and my python skills could also use refreshing.
NOTE I'm using csvreader in this code because I originally had a .csv file - however I found it was just as easy to use the .txt file I was provided.
I am trying to read in multiple files to populate multiple tables in sqlite using python. I am trying to do something like this.
def connect_db():
return sqlite3.connect(app.config['DATABASE']) # Flask stuff
# List of tables and corresponding filenames
file_list = ['A','B','C','D']
for file in file_list:
with closing(connect_db()) as db:
cursor = db.execute('select * from ' + file)
# Get column names for the current table
names = tuple(map(lambda x: x[0], cursor.description))
filename = file + '.txt'
with open(filename, 'r') as f:
data = [row.split('\t') for row in f.readlines()]
cur.executemany('INSERT INTO' + file + ' ' + str(names) + ' VALUES ' + ???)
Now I realized that in the executemany() statement above, I need to put the syntax (?,?,?) where ? is the number of columns. I can generate a tuple of ? but then they will have quotes around them. ('?', '?', '?'). Is there a way around this? Thanks.
You can implement a query "maker", that loop over a dict or an object properties and build VALUES string with it.
I did something like that on some project :
d = {
"id":0,
"name":"Foo",
"bar":"foobar"
}
sql = ""
for key, foo in d.iteritems():
# First, copy the current value
bar = copy.copy( foo )
# Test if number
try:
# try to increase the value of 1
bar += 1
except TypeError:
# if we can't, add double quote
foo = '"%s"' % foo
# concat with the previous sequences
sql = "%s%s=%s," % ( sql, key, foo )
# remove the latest comma ...
sql = sql[:-1]
So the value of "sql" at the end of the loop should be something like this
bar="foobar",id=0,name="Foo"