I'm using pandas as pd and python 2.7
I have a bunch of queries and each query is returning serial numbers. Then I'm using that serial number in a bunch more queries. I want to append all the results to a list. When I append to an empty list, it just returns nothing. I don't know what I'm doing wrong.
for query in list_of_queries:
results = pd.read_sql_query(query,connection,index_col=None)
for serial_number in results['SerialNumbers']:
a = []
new_query = """ SELECT * FROM blah b where b.SerialNumber = '{}' """
new_query = new_query.format(serial_number)
results = pd.read_sql_query(new_query,connection,index_col = None)
a.append(results)
You are resetting the list to be empty at the beginning of each for loop. This should be:
a = []
for serial_number in results['SerialNumbers']:
new_query = """ SELECT * FROM blah b where b.SerialNumber = '{}' """
new_query = new_query.format(serial_number)
results = pd.read_sql_query(new_query,connection,index_col = None)
a.append(results)
# 'a' will now have all the results
Furthermore, it looks like you might be clobbering results because you use it as a variable name twice (once in each loop). I would suggest changing that too!
Related
I have a function getloantype(account_no) which I would like to call. The account numbers are in a list ['10101-2','10101-2', '10101-3'] and I would like the function to run one by one through all the account numbers and put all the results into another list. However, I cannot seem to get my code to run.
What I do: first, I get the user to input his userID and use it to fetch all the bank accounts that he owns from SQL database:
userid = input("Please enter user id")
conn=create_connection()
def getacct(userid):
query = """\
select Account_Number
from User_Account
where UserID = '{}'
""" .format(userid)
return (execute_read_query (conn, query))
account_no = getacct(userid)
As such, the account numbers would end up being in a list [account_no]. Next, I will need to use the account number to fetch his Loan ID. From this part comes the first question. Am I supposed to code it as getloanid(account_no) or getloanid(x) whereby x is for x in account_no ?
def getloanid(x):
query = """\
select SUBSTRING(LoanID, 1, 2)
from Account_Delinquency
where Account_Number = '{}'
""" .format(account_no)
return (execute_read_query (conn, query))
From here, I assume that I should do a nested for loop but the way I coded it, the list remains empty.
loanlist = []
for i in account_no:
for x in i:
getloanid(x)
loanlist.append(i[0])
I have also tried this which would return error :
'NoneType' object is not iterable
mylist = []
loanlist = []
for i in account_no:
mylist.append(i[0])
for x in mylist:
a = getloanid(x)
for i in a:
loanlist.append(i[0])
How can I code it such that I can call the function getloantype(account_no) with all the account numbers in the list account_no = getacct(userid) and have all the results be appended into a new list [loanlist]?
It is not so clear the structure of your program, and data returned from functions, but if I can undestand, a possible simple solution could be somthing like this:
result_list = [getloantype(account_no) for account_no in account_list]
I'm querying a REST API url & I'm trying to return all the dictionary sets, but only able to return one key pair.
Dictionary Output in the print statement inside for loop is the expected output, when when returned only one set of key pair is appearing.
Expected Dictionary looks like:
{'IncidentID': 'IM10265'}
{'IncidentID': 'IM10266'}
{'IncidentID': 'IM10267'}
{'IncidentID': 'IM10268'}
Code:
import json , requests
sm1 = requests.get('http://Rest Url', auth=('XX','YY'))
z = json.loads(sm1.text)
def get_im_list():
incidentlist_access = z['content']
for im_data in incidentlist_access:
Access_imslist = im_data['Incident']
print(Access_imslist)
#print(type(Access_imslist))
#return Access_imslist
data = get_im_list()
#print(data)
So when when I'm un-commentating
return Access_imslist & print(data)
I'm only receiving the output as:
{'IncidentID': 'IM10265'}
not the complete dictionary.
Every time you loop through the data, Access_imslist gets overwritten, so when you (presumably) return Access_Imlist it's only returning the last value.
You need to create a data structure outside of the for loop, add each bit of data to it, then return that instead. Something like:
def get_im_list():
incident_data = []
incidentlist_access = z['content']
for im_data in incidentlist_access:
incident_data.append(im_data['Incident'])
return incident_data
hope that helps!
you need to define a variable list , and then append the Access_imslist values to that variable.
like this :
data = []
def get_im_list():
incidentlist_access = z['content']
for im_data in incidentlist_access:
Access_imslist = im_data['Incident']
data.append(Access_imslist)
print(data)
I'm writing an ArcGIS function and I want to pass the variable and value as parameters (if possible).
Here's my code:
from arcgis.gis import *
from arcgis.features import FeatureLayer
URL = 'http://url_to_arcgis_server'
# QUERY = HERE LIES THE PROBLEM
def get_features(url, query):
features = []
f = FeatureLayer(url = url)
f_set = f.query(where = '1=1', out_sr = '4326')
for f in f_set:
features.append(f.as_dict)
return features
get_features(URL, QUERY)
where query can be chosen from this list of parameters.
Is there a way that I can pass where = '1=1' and out_sr = '4326' to QUERY? I tried encapsulating in quotes (QUERY = "where = '1=1', out_sr = '4326'"), but that does not work.
ArcGIS says that queries are based on SQL statements.
QUERY = { 'where': '1=1', 'out_sr': '4326' }
...
f_set = f.query(**QUERY)
The ** syntax causes the key/value pairs of a dictionary to be split into separate keyword parameters to a function call. Note that the syntax of a dictionary is somewhat different - the key name has to be quoted, colon instead of equals sign.
So I have data known as id_list that is coming into the function in this format [(u'SGP-3630', 1202), (u'MTSCR-534', 1244)]. The format being two values paired together, there could be 1 pair or a hundred pairs.
This is the function:
def ListParser(id_list):
list_length = len(id_list)
count = 0
table = ""
while count < list_length:
jira = id_list[count][0]
stash = id_list[count][1]
count = count + 1
table = table + RetrieveFromAPI(stash, jira)
table = TableFormatter(table)
table = TableColouriser(table)
return table
What this function does is goes through the list and extracts the pairs and puts them through a function called RetrieveFromAPI() which fetches information from a URL.
Anyone have an idea on how to impliment multithreading here? I've had a shot at splitting both lists up into their own lists and getting the pool to iterate through each list but it hasn't quite worked.
def ListParser(id_list):
pool = ThreadPool(4)
list_length = len(id_list)
count = 0
table = ""
jira_list = list()
stash_list = list()
while count < list_length:
jira_list = jira_list.extend(id_list[count][0])
print jira_list
stash_list = stash_list.extend(id_list[count][1])
print stash_list
count = count + 1
table = table + pool.map(RetrieveFromAPI, stash_list, jira_list)
table = TableFormatter(table)
table = TableColouriser(table)
return table
The error I'm getting for this attempt is TypeError: 'int' object is not iterable
EDIT 2: Okay so I've managed to get the first list with tuples split up into two different lists, but I'm unsure how to get multithreading working with it.
jira,stash= map(list,zip(*id_list))
You're working too hard! From help(multiprocessing.pool.ThreadPool)
map(self, func, iterable, chunksize=None)
Apply `func` to each element in `iterable`, collecting the results
in a list that is returned.
The second argument is an iterable of the arguments you want to pass to the worker threads. You have a list of lists and you want the first two items from the inner list for each call. id_list is already iterable, so we're close. A small function (in this case implemented as a lambda) bridges the gap.
I worked up a full mock solution just to make sure it works, so here it goes. As an aside, you can benefit from a fairly large pool size since they spend much of their time waiting on I/O.
from multiprocessing.pool import ThreadPool
def RetrieveFromAPI(stash, jira):
# boring mock of api
return '{}-{}.'.format(stash, jira)
def TableFormatter(table):
# mock
return table
def TableColouriser(table):
# mock
return table
def ListParser(id_list):
if id_list:
pool = ThreadPool(min(12, len(id_list)))
table = ''.join(pool.map(lambda item: RetrieveFromAPI(item[1], item[0]),
id_list, chunksize=1))
pool.close()
pool.join()
else:
table = ''
table = TableFormatter(table)
table = TableColouriser(table)
return table
id_list = [[0,1,'foo'], [2,3,'bar'], [4,5, 'baz']]
print(ListParser(id_list))
I am using Django 1.8 and currently am working on a Blog application. When i search for tweets( just a name instead of posts) , i want to save the search results obtained after querying the database, as text instead of list. My view function is as below:
def search(request):
query = request.GET.get('q','')
if query:
qset = (
Q(text__icontains=query)
#Q(hashes__icontains=query)
#Q(artist__icontains=query)
)
results = Tweet.objects.filter(qset).distinct()
else:
results = []
number_of_results = len(results)
search_item = query
returned_items = []
for res in results:
text = res.text
returned_items.append(text)
returns = returned_items[:]
search = Search(search_item=search_item,returns=returns)
search.save()
context = {'query':query,'results':results,'number_of_results':number_of_results,'title':'Search results for '+request.GET.get('q','')}
return render_to_response("tweets/search.html",context,context_instance=RequestContext(request))
also, the snapshot of my search table in the database is as shown below:
Please help me out friends.
you should join the returned list using the comma separted values. This will return the string.
returns = ', '.join(returned_items)
This piece of code is setting returns to a list:
returns = returned_items[:]
If you want to access the first string, set it to returned_items[0]. If you want to join all strings in the list, use join()
returns = "".join(returned_items)