I'm still trying to get the hang of python, so bear with me. please. I have this bit of code that I'm using from a book. The book does not properly show the white space in the code, so the spacing is my best guess. This code is supposed to break the results of a MySQL query into a more readable format.
if form is True:
columns_query = """DESCRIBE %s""" % (table)
print columns_query
columns_command = cursor.execute(columns_query)
headers = cursor.fetchall()
column_list = []
for record in headers:
column_list.append(record[0])
output=""
for record in results:
output = output + "===================================\n\n"
for field_no in xrange(0, len(column_list)):
output = output + column_list[field_no] + ": " + str(record[field_no]) + "\n"
output = output + "\n"
When I try to run it, I get the following:
Traceback (most recent call last):
File "odata_search.py", line 46, in <module>
output = output + column_list[field_no] + ": " + str(record[field_no]) + "\n"
IndexError: tuple index out of range
It has something to do with the str(record[field_no]) portion of the code, but that's what it looks like in the book, so I'm not sure what else to try.
Clearly len(record) != len(column_list). (specifically, column_list is longer than record). Is there a reason that you expect them to be the same length?
One "fix" would be something like:
for col,rec in zip(column_list,record):
output += col + ": " + str(rec) + "\n"
instead of:
for field_no in xrange(0, len(column_list)):
output = output + column_list[field_no] + ": " + str(record[field_no]) + "\n"
This will truncate the output at the shorter of column_list and record.
I would recommend using zip instead of range(0,len(...)) in any event. It's much more idiomatic.
The problem it turns out was both the white space and, more importantly, the MySQL query itself. I was pulling a list that were rows in a column instead of pulling all columns of a row, which is what the loops were written to concatenate. The number of records I would get back in the result of the wrong query was not equal to the number of results in the list that contained all the columns. The code was also intended to be a loop within a loop, so the spacing I have at the top is wrong. It should look like it does below. I added a couple of lines before it to show the query I had to modify:
Old statement looped like this:
statement = """select %s from %s where %s like '%s' limit 10""" % (column, table, column, term)
Should look like this:
statement = """select * from %s where %s like '%s' limit 10""" % (table, column, term)
command = cursor.execute(statement)
results = cursor.fetchall()
column_list = []
for record in results:
column_list.append(record[0])
Loop within a loop:
if form is True:
columns_query = """DESCRIBE %s""" % (table)
columns_command = cursor.execute(columns_query)
headers = cursor.fetchall()
column_list = []
for record in headers:
column_list.append(record[0])
output=""
for record in results:
output = output + "===================================\n\n"
for field_no in xrange(0, len(column_list)):
output = output + column_list[field_no] + ": " + str(record[field_no]) + "\n"
output = output + "\n"
Related
I use Select query in mysql and I would like to get records using python.
client_start is SELECT query
client_start = con.execute("SELECT DISTINCT `clients_agreements`.`date_start` , `buildings`.`id` , `buildings`.`street` , `buildings`.`street_nr` , `clients`.`building_id` , `clients_agreements`.`user_id` FROM `clients_agreements` LEFT JOIN `buildings` On `clients_agreements`.`user_id` = `buildings`.`id` LEFT JOIN `clients` ON `clients`.`building_id` = `buildings`.`id` WHERE `date_start` = (CURRENT_DATE)") #SELECT ID CLIENT QUERY
for row in client_start:
streets = row[2]
house_nr = row[3]
message = str(streets) + " / " + str(house_nr)
msg = 'Subject: {0}\n\n{1}'.format(subject,message).encode('utf-8').strip()
When i use print into loop I get two result(what is good) but when i use print after loop i get only one result. How to print all result after loop? I can't use arrays because I would like to send it in email. I want to send all records
As #roganjosh said, each time through the loop you're throwing away the previous contents of message.
You can address this in two ways:
Whatever you're doing with message -- printing, sending an email, writing to a file, etc. -- do it inside the loop, while you have the current value.
Or, if you need to collect all the values and then handle them all at once, save them in a list:
messages = []
for row in client_start:
streets = row[2]
house_nr = row[3]
message = str(streets) + " / " + str(house_nr)
messages.append(message)
# now handle the list of messages
Beginner here.
I have the following circumstances.
A text file with each line containing a name.
A cassandra 3.5 database
A python script
The intention is to have the script read from the file one line (one name) at a time, and query Cassandra with that name.
FYI, everything works fine except for when I try to pass the value of the list to the query.
I current have something like:
#... driver import, datetime imports done above
#...
with open(fname) as f:
content = f.readlines()
# Loop for each line from the number of lines in the name list file
# num_of_lines is already set
for x in range(num_of_lines):
tagname = str(content[x])
rows = session.execute("""SELECT * FROM tablename where name = %s and date = %s order by time desc limit 1""", (tagname, startDay))
for row in rows:
print row.name + ", " + str(row.date)
Everything works fine if I remove the tagname list component and edit the query itself with a name value.
What am I doing wrong here?
Simply building on the answer from #Vinny above, format simply replaces literal value. You need to put quotes around it.
for x in content:
rows = session.execute("SELECT * FROM tablename where name ='{}' and date ='{}' order by time desc limit 1".format(x, startDay))
for row in rows:
print row.name + ", " + str(row.date)
You can simply iterate over content:
for x in content:
rows = session.execute("SELECT * FROM tablename where name = {} and date = {} order by time desc limit 1".format(x, startDay))
for row in rows:
print row.name + ", " + str(row.date)
....
Also, you don't need to have 3 quotes for the string. Single quotes is good enough (3 quotes is used for documentation / multiple line comments in python)
Note that this might end in a different error; but you will be iterating on the lines instead of iterating over an index and reading lines.
I am new to python and with this task I'm trying to learn it. I'd like to get information out of a DNS zone file. The DNS file has the following entries:
something 600 A 192.168.123.45
600 A 192.168.123.46
someelse CNAME something
anotherone A 192.168.123.47
nextone CNAME anotherone
anotherone TXT ( "te asd as as d" )
The goal is to grab the hostnames and IF there is a coresponding TXT entry, I'd like to get the information for that entry as well.
So I started to just work through the file, if I would describe the entries, the Record type is either at [2] or [1] and right after it, I got the IP. So I have something like this for now:
for line in data:
word = line.split()
if len(word) > 2:
if "CNAME" == word[2]:
rectype = "CNAME"
arecord = word[3].replace('.domain.com.', '')
print rectype + " " + arecord
if "CNAME" == word[1]:
rectype = "CNAME"
arecord = word[2].replace('.domain.com.', '')
print rectype + " " + arecord
if "A" == word[2]:
rectype = "A"
print rectype + " " + word[3]
if "A" == word[1]:
rectype = "A"
print rectype + " " + word[2]
okay, so far so good.. but now if I like to get that corresponding TXT record, do I need to work through every line in the document for each line or is their any easier and more efficiernt way to do that?
Part of the beauty of Python is you can write code that reads like English and it uses intuitive operators.
You have a lot of duplicated code. In your example, we can see that anytime a "keyword" pops up, such as A or CNAME then you simply take the next token. Here, I used the in Python keyword which checks if an element is in a collection of some sort. This is a boolean, so if it returns true then I take the next element, ie, tokens[tokens.index(keyword) + 1].
Similarly, in can also be used for string and substring searches. I check to see if "TXT" is in the current line, and if it is, I assume you want everything after it? I use a splice operator to specify the range.
line[line.index("TXT") + 3:] means that I want everything in line after the index line.index("TXT") + 3.
KEYWORDS = ["CNAME", "A"]
for line in data:
tokens = line.split()
if len(tokens) > 2:
record = ""
for keyword in KEYWORDS:
if keyword in tokens:
record = keyword + " " + tokens[tokens.index(keyword) + 1]
if "TXT" in line:
txt_data = line[line.index("TXT") + 3:]
record += "TXT: " + txt_data
print record
I'd recommend you build a dictionary whose keys are identifiers ("anotherone" would be an example from your sample) and whose values are strings (or maybe a list of strings - I'm rusty on DNS and am not sure if multiples are possible).
As you encounter CNAME and A records, add the identifiers as keys into the dictionary and initialize their corresponding values as empty lists. Then when you hit a line with "TXT" in it, lookup that identifier in the dictionary and add the line as value.
I am trying to python to generate a script that generates unload command in redshift. I not an expert Python programmer. I need to where I can generate all columns for the unload list. If the column is of specific name, I need to replace with a function. The challenge I am facing is it appending "," to last item in the dictionary. Is there a way I can avoid the last comma? Any help would be appreciated.
import psycopg2 from psycopg2.extras
import RealDictCursor
try:
conn = psycopg2.connect("dbname='test' port='5439' user='scott' host='something.redshift.amazonaws.com' password='tiger'");
except:
print "Unable to connect to the database"
conn.cursor_factory = RealDictCursor
cur = conn.cursor()
conn.set_isolation_level( psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT )
try:
cur.execute("SELECT column from pg_table_def where schema_name='myschema' and table_name ='tab1' " );
except:
print "Unable to execute select statement from the database!"
result = cur.fetchall()
print "unload mychema.tab1 (select "
for row in result:
for key,value in row.items():
print "%s,"%(value)
print ") AWS Credentials here on..."
conn.close()
Use the join function on the list of values in each row:
print ",".join(row.values())
Briefly, the join function is called on a string which we can think of as the "glue", and takes a list of "pieces" as its argument. The result is a string of "pieces" "held together" by the "glue". Example:
>>> glue = "+"
>>> pieces = ["a", "b", "c"]
>>> glue.join(pieces)
"a+b+c"
(since row.values() returns a list, you don't actually need the comprehension, so it's even simpler than I wrote it at first)
Infact, this worked better.
columns = []
for row in result:
if (row['column_name'] == 'mycol1') or (row['column_name'] == 'mycol2') :
columns.append("func_sha1(" + row['column_name'] + "||" + salt +")")
else:
columns.append(row['column_name'])
print selstr + ",".join(columns)+" ) TO s3://"
Thanks for your help, Jon
This is effectively how i'm using _mssql.
Everything works fine, even after i use fetch_array().
The problem is, when I iterate through fetch_array(), it takes over ten minutes for 6K rows to be written to the body of an email. This just isn't acceptable.
Is there a better way for me to do this?
EDIT: haven't figured out the best way to copy python code here but here is the crappily formatted code used.
code for writing the email body:
results=mssql.fetch_array()
mssql.close()
resultTuple = results[0]
if resultTuple[1] == 0:
emailBody = 'Zero Rows returned'
if test or resultTuple[1] != 0:
i = 0
columnTuples = resultTuple[0]
listOfRows = resultTuple[2]
columns = []
emailBody = ''
if test: print 'looping results'
while i < len(columnTuples):
columns.append(columnTuples[i][0])
emailBody = emailBody + columns[i].center(20) + '\t'
i = i + 1
emailBody = emailBody + '\n'
for rowTuple in listOfRows: #row loop
for x in range(0, len(rowTuple)): #field loop
emailBody = emailBody + str(rowTuple[x]).center(20) + '\t'
if test: print x
emailBody = emailBody + '\n'
Could you show us some code of iterating through the results and creating an email? If I had to make a wild guess, I would say you are probably violating this idiom: "Build strings as a list and use ''.join at the end." If you keep adding to a string as you go you create a new (and progressively larger) string on each iteration, which runs in quadratic time.
(Turned this comment into an answer at the request of the OP. Will edit to more complete answer as needed.)
Edit:
Ok, my guess was correct. You'll want something like this:
email_lines = []
for row in result:
line = do_something_with(row)
email_lines.append(line)
email_body = '\n'.join*(email_lines)