I am fetching data from database which is stored in arrays, I have to match the output of this array with a string. But array outputs the result into Unicode format (u'aviesta',) thus it does not match the string.
My code.
// fblike is an arry in which the output of query stores.
for i in fblike:
if i=="Aviesta":
like=1
return render_to_response('showroom.html')
i have also try to encode this as variable.encode('utf8') but it only encode the specific element of an array such as i[0].encode('utf8') but i do not know which element of array have aviesta as a value.Thus i need to encode whole array but5 i don't know how to do that.
Updated::
In views.py is use
cursor = connection.cursor()
cursor.execute("SELECT name FROM django_facebook_facebooklike WHERE user_id = %s", request.user.id)
rowfb = cursor.fetchall() return render_to_response('showroom.html',{'rowfb':rowfbthis}
and print the {{rowfb}} variable in my template..and result array is
((u'Mukesh Chapagain',), (u'Ghrix Technologies Private Limited',), (u'FirstLALimo',), (u'Aviesta',), (u'Awkward Group',), (u'FB.Canvas.setDoneLoading',), (u'99recharge',), (u'AllThingsCustomized.com',), (u'celebrity aviesta',), (u'FTC',))
So please suggest me some way so that i can match the elements of array with the given string.
Thanks
Firstly, you should have posted the code as an update to your question, rather than a comment.
Secondly, I have no idea why you are accessing the data via a manual SQL query, rather than using Django's ORM. If you had done it the normal way, you would not be having this problem.
Finally, your problem has nothing to do with encodings. Your data is as follows (reposted for clarity):
((u'Mukesh Chapagain',), (u'Ghrix Technologies Private Limited',), (u'FirstLALimo',), (u'Aviesta',), (u'Awkward Group',), (u'FB.Canvas.setDoneLoading',), (u'99recharge',), (u'AllThingsCustomized.com',), (u'celebrity aviesta',), (u'FTC',))
This is a tuple of tuples. Each row of data is represented by a tuple, and in turn each column within that row is a tuple. In your case, since you're only selecting one column, you have a tuple of single-element tuples. That means, in each iteration of your loop, you have a tuple, not a string.
This would work:
for i in fblike:
if i[0] == "Aviesta":
like = 1
but to be honest, you would be better off going and doing a simple Python tutorial, and then going back to the Django tutorial and learning how to do queries via the ORM.
I don't know if your question has anything to do with arrays.
If you only need to find if the given string is in your array, you could simply do
if "Aviesta" in fblike:
like+=1
Related
I am using execute_values to insert a list of lists of values into a postgres database using psycopg2. Sometimes I get "not all arguments converted during string formatting", indicating that one of the values in one of the lists is not the expected data type (and also not NoneType). When it is a long list, it can be a pain to figure out which value in which list was causing the problem.
Is there a way to get postgres/psycopg2 to tell me the specific 'argument which could not be converted'?
If not, what is the most efficient way to look through the list of lists and find any incongruent data types per place in the list, excluding NoneTypes (which obviously are not equal to a value but also are not the cause of the error)?
Please note that I am not asking for help with the specific set of values I am executing it, but trying to find a general method to more quickly inspect the problem query so I can debug it.
Importing a JSON document into a pandas dataframe using records = pandas.read_json(path), where path was a pre-defined path to the JSON document, I discovered that the content of certain columns of the resulting dataframe "records" are not simply strings as expected. Instead, each "cell" in such a column is an array, containing one single element -- the string of interest. This makes selecting columns using boolean indexing difficult. For example, records[records['category']=='Python Books'] in Ipython outputs an empty dataframe; had the "cells" contained strings instead of arrays of strings, the output would have been nonempty, containing rows that correspond to python books.
I could modify the JSON document, so that "records" reads the strings in properly. But is there a way to modify "records" directly, to somehow strip the single-element arrays into the elements themselves?
Update: After clarification, I believe this might accomplish what you want while limiting it to a single iteration over the data:
nested_column_1 = records["column_name_1"]
nested_column_2 = records["column_name_2"]
clean_column_1 = []
clean_column_2 = []
for i in range(0, len(records.index):
clean_column_1.append(nested_column_1[i][0])
clean_column_2.append(nested_column_2[i][0])
Then you convert the clean_column lists to Series like you mentioned in your comment. Obviously, you make as many nested_column and clean_column lists as you need, and update them all in the loop.
You could generalize this pretty easily by keeping a record of "problem" columns and using that to create a data structure to manage the nested/clean lists, rather than declaring them explicitly as I did in my example. But I thought this might illustrate the approach more clearly.
Obviously, this assumes that all columns have the same number of elements, which maybe isn't a a valid assertion in your case.
Original Answer:
Sorry if I'm oversimplifying or misunderstanding the problem, but could you just do something like this?
simplified_list = [element[0] for element in my_array_of_arrays]
Or if you don't need the whole thing at once, just a generator instead:
simplifying_generator = (element[0] for element in my_array_of_arrays)
I am trying to get the results to a SQLAlchemy query. I know that if I loop over the query I can put the results in a list (like below), but this seems inefficient for a large set of results and looks ugly when the result will be a single number (as below). Is there a more direct and/or efficient way to return query results?
mylist = []
for item in session.query(func.max(mytable.id)):
mylist.append(item)
Looping through the result, as you do, is correct. You can also use all() to get the list of sequences (rows). Maybe more efficient is to not store the data in a list, get smaller result sets, and/or do the operation immediately on each row. You could also use a server side cursor if your DBMS supports it.
When only one row with one field is fetched, you can use first() and get the first element of the returned sequence. Code wise, this is probably most efficient:
maxid_mytable = session.query(func.max(mytable.id)).first()[0]
This will the equivalent
mylist = session.query(func.max(mytable.id)).all()
I have a SQL database that I store python lists in. Currently I convert the list to a string and then insert it into the database (using sqlite3) i.e.
foo = [1,2,3]
foo = str(foo)
#Establish connection with database code here and get cursor 'cur'
cur.execute("INSERT INTO Table VALUES(?, ?)", (uniqueKey, foo,))
It seems strange to convert my list to a string first, is there a better way to do this?
Replace your (key, listdata) table with (key, index, listitem). The unique key for the table becomes (key, index) instead of just key, and you'll want to ensure as a consistency condition that the set of indexes in the table for any given key is contiguous starting from 0.
You may or may not also need to distinguish between a key whose list is empty and a key that doesn't exist at all. One way is to have two tables (one of lists, and one of their elements), so that an empty but existing list is naturally represented as a row in the lists table with no corresponding rows in the elements table. Another way is just to fudge it and say that a row with index=null implies that the list for that key is empty.
Note that this is worthwhile if (and probably only if) you want to act on the elements of the list using SQL (for example writing a query to pull the last element of every list in the table). If you don't need to do that, then it's not completely unreasonable to treat your lists as opaque data in the DB. You're just losing the ability for the DB to "understand" it.
The remaining question then is how best to serialize/deserialize the list. str/eval does the job, but is a little worrying. You might consider json.dumps / json.loads, which for a list of integers is the same string format but with more safety restrictions in the parser. Or you could use a more compact binary representation if space is an issue.
2 ways.
Normalize tables, to you need setup new table for list value. so you get something like "TABLE list(id)" and "TABLE list_values(list_id, value)".
You can serialize the list and put in a column. Ex. Json, XML and so on (its not a very good practice in SQL).
I can see quite a few different options for doing this and would like some feedback on the most efficient or 'best practice' method.
I get a Django Queryset with filter()
c_layer_points = models.layer_points.objects.filter(location_id=c_location.pk,season_id=c_season.pk,line_path_id=c_line_path.pk,radar_id=c_radar.pk,layer_id__in=c_layer_pks,gps_time__gte=start_gps,gps_time__lte=stop_gps)
This queryset could be very large (hundreds of thousands of rows).
Now what needs to happen is a conversion to lists and encoding to JSON.
Options (that i've seen in my searches):
Loop over the queryset
Example:
gps_time = [lp.gps_time for lp in c_layer_points];
twtt = [lp.twtt for lp in c_layer_points];
Use values() or values_list()
Use iterator()
In the end I would like to encode as json something like this format:
{'gps_time':[list of all gps times],'twtt',[list of all twtt]}
Any hints on the best way to do this would be great, Thanks!
You might not be able to get the required format from the ORM. However, you can efficiently do something like this:
c_layer_points = models.layer_points.objects.filter(
location_id=c_location.pk,
season_id=c_season.pk,
line_path_id=c_line_path.pk,
radar_id=c_radar.pk,
layer_id__in=c_layer_pks,
gps_time__gte=start_gps,
gps_time__lte=stop_gps
).values_list('gps_time', 'twtt')
and now split the tuples into two lists: (Tuple unpacking)
split_lst = zip(*c_layer_points)
dict(gps_time=list(split_lst[0]), twtt=list(split_lst[1]))
I will suggest you use the iterate through the query set and conform the json dictionary element by element from the queryset.
Normally, Django's QuerySets are lazy, this means they get load into memory whenever they get accessed. If you load the entire list: gps_time = [lp.gps_time for lp in c_layer_points] you will have all those objects in memory (thousands). You'll be good by doing a simple iteration:
for item in c_layer_points:
#convert item to json and add it to the
#json dict.
As an aside note, you don't need the ; character at the end of lines in python :)
Hope this helps!