Convert web.py iterbetter to dictionary or list - python

I'm trying to get the some data from a MySQL table (ipb_members) using web.py's database.select. This is the code I'm using: members = db.select("ipb_members", where="name=\"asdfquerty\"")
it returns an instance of iterbetter. What I'm trying to figure out is how to convert that to a dictionary or a list. I saw this, which recommends using list(), but that just puts everything in the first index. I've also tried dict(), but that didn't work either. What am I doing wrong?

This should work:
members = db.select("ipb_members", where="name=\"asdfquerty\"")
for member in members #members is IterBetter
print member.posts #member is Storage
Note that you dont have to call .list() on the result of db.select if you need to iterate over it only once.

Have you tried dict(members)?

iterBetter is not very convenient as it can only iterate once, here is code convert iterBetter to list:
list_conv_from_db = []
for menber in menbers:
temp = dict()
for key in menber:
temp[key]=menber[key]
list_conv_from_db.append(temp)
then you get list_conv_from_db you can do everything you want!

Related

Add a unique id to list of nested dictionaries in python

I am working on a API request that return a list of nested dictionaries. The result is similar to this,
example=[{'transactionId':'1112234','customerID':1212,'total':360,'items':[{'productId':1a2345d23,'quantity':3,'price':160},{'productId':1a5645d99,'quantity':5,'price':200}],{
'transactionId':'11134674','customerID':1223,'total':120,'items':[{'productId':1a2345d27,'quantity':1,'price':60},{'productId':1a1145d22,'quantity':1,'price':60}}]]
I made a code to loop and append the newId to the list and was only able to insert the id to the parent dictionary.
result = [dict(item, newId= uuid.uuid4()) for item in example]
Any help would be greatly appreciated to attain my desire result like the one bellow.
example=[{'newId':1111,'transactionId':'1112234','customerID':1212,'total':360,'items':[{'newId':1111,'productId':1a2345d23,'quantity':3,'price':160},{'newId':1111,'productId':1a5645d99,'quantity':5,'price':200}],
{'newId':2222,'transactionId':'11134674','customerID':1223,'total':120,'items':[{'newId':2222,'productId':1a2345d27,'quantity':1,'price':60},{'newId':2222,'productId':1a1145d22,'quantity':1,'price':60}}]]
Your example data has syntax errors in it, so I wasn't able to test the following code, but I believe this is what you are looking for. You just need to do things the old fashioned way instead of doing list comprehension, and it becomes easy enough to do what you want.
result = []
for order in example:
order['newId'] = uuid.uuid4()
if 'items' in order:
for item in order['items']:
item['newId'] = uuid.uuid4()

Python elasticsearch-dsl sorting with multiple fields

I'm trying to form the command for sorting using elasticsearch-dsl. However I have trouble passing the variable in correct format in.
The format should be
s=Search()
s = s.sort({"time":{"order":"asc"}}, {"anoter_field":{"order":"desc"}})
s.execute()
The problem is I'm trying to put {"time":{"order":"asc"}}, {"anoter_field":{"order":"desc"}} as a variable, but I can't seem to get this in the right syntax. I tried using dict, list, and string, and none seems to work.
My input would be a dict that looks like
input = {"time":"asc", "another_field":"desc"}
data_input = {"time":"asc", "another_field":"desc"}
args = [{k:{'order':v}} for k,v in data_input.items()]
s.sort(*args)
I guess is what you are asking? Its hard to tell ...

How to get spotipy playlist results in list format

I'm using spotipy to get a list of my playlists. I use
if token:
sp = spotipy.Spotify(auth=token)
playlists = sp.user_playlists(username)
for item in playlists['items']:
id= item['uri']
print id
This returns a list of playlist uri's that looks like
spotify:user:ultramusicofficial:playlist:0gvQoG7iMMz8L5Ltsa4lkT
spotify:user:spotify:playlist:4Ha7Qja6HY3AgvNBgWz87p
spotify:user:ministryofsounduk:playlist:7grWVkJDQpcBie8oqKP6hv
But there is something weird about the way it returns them. It's not a normal list and I can't seem to make it into one. If I use
print id[1]
It would return something like
p
p
p
I want to be able to do something like
print id[1]
and have it return
spotify:user:spotify:playlist:4Ha7Qja6HY3AgvNBgWz87p
I've tried joining it and splicing it in different ways, I've tried using it as a tuple, converting it to a string. Nothing works I'm clearly very unsure what to do. I feel like it's probably a simple and I'm just missing it. Any help would be appreciated thanks.
You are just printing the id, not gathering them into a list, so your id is the last item from the loop which is a single uri (a string). You can use a list comprehension to make a list out of the for loop:
id = [item['uri'] for item in playlists['items']]
Or start with an empty list and append the result to it:
id = []
for item in playlists['items']:
id.append(item['uri'])
In your example id is a string. So id[1] is the second character, which is p

How to get a list of all indexes in python-elasticsearch

How would I get a list of the names of an index in Python? Here is what I have so far:
>>> es=e.es
>>> es
<Elasticsearch([{'host': '14555f777d8097.us-east-1.aws.found.io', 'port': 9200}])>
>>> es.indices
<elasticsearch.client.indices.IndicesClient object at 0x10de86790>
# how to get a list of all indexes in this cluster?
This question comes up when searching for information on retrieving aliases using the python-elasticsearch library. The accepted answer says to use get_aliases but that method has been removed (as of 2017). To get aliases, you can use the following:
es.indices.get_alias("*")
UPDATE
The latest usage should be with a keyword arg:
es.indices.get_alias(index="*")
how to get a list of all indexes in this cluster?
Use the wildcard. Works with elasticsearch-py.
for index in es.indices.get('*'):
print index
Here is one way to do it with the get_alias() method:
>>> indices=es.indices.get_alias().keys()
>>> sorted(indices)
[u'avails', u'hey', u'kibana-int']
If you are willing to use pyelasticsearch module they have support for the GET _mapping command, which produces the schema of the cluster. This will allow you to see the indices, and drill into each index to see doc_types, and their fields, etc. Here's an example:
import pyelasticsearch as pyes
es = pyes.ElasticSearch(["http://hostname0:9200", "http://hostname1:9200"]) ## don't accidentally type Elasticsearch, the class from the other two modules
schema = es.get_mapping() ## python dict with the map of the cluster
To get just the list of indices,
indices_full_list = schema.keys()
just_indices = [index for index in indices_full_list if not index.startswith(".")] ## remove the objects created by marvel, e.g. ".marvel-date"
This is related to this question
You can use the Cat API:es.cat.indices(h='index', s='index').split()
I use curl to call the stats API and get information about the indices. Then I parse the JSON object that is returned to find the index names.
curl localhost:9200/_stats
In Python you can call curl using the requests library. I don't know of a way to do this using the Elasticsearch or Elasticsearch-DSL Python library.
You can get _mapping to get list of all indexes by doing something like that.
requests.get(full_elastic_url + "/_mapping")
_cat API seems the right way to do this, since the _aliases way of doing will soon be removed by elasticsearch since it exposes the system indices.
indices = es.cat.indices(h='index', s='index').split()
It did the job for me.
If you want 'alias name' and not 'index name', here the perfect solution:
response = es.indices.get(indexname)
alias_names = list(response[indexname]['aliases'].keys())
In alias_names we get list of alias names on a particular index.

TypeError: documents must be a non-empty list

I'm doing a program using Twitter API and MongoDB in 2.7 Python language.
I get a timeline and put it in a dictionary, which I want to store in a MongoDB database. To do this I have next code:
def saveOnBD(self, dic):
client = MongoClient("xxxx", "port")
db = client.DB_Tweets_User_Date
collection = db.tweets
collection.insert_many(dic)
I'm debbuging and dic it's not empty but I get next error:
TypeError: documents must be a non-empty list
How can I fix it?
I trying many options, but i solved that question changing the post method.
Instead of:
collection.insert_many(dic)
I used this:
collection.insert_one(dic)
I supose that, as I try to post only a variable(dic) and "insert_many()" is for many variables that retun me the error. That change solved me the question
you can either put in an entry before running the bulk entry function or use insert()
A list of documents must be passed to insert_many method
E.g.:
collection.insert_many([dic])

Categories

Resources