I have a list of computer nodes called node_names, and I want to find the amount of free ram in each node, and store that in a second list. I then want to combine these lists into a dictionary.
I have:
for i in range(0, number_of_nodes):
sys_output = [commands.getoutput('ssh %s \'free -m\'' % node_names[i])]
free_memory = [x.split()[9] for x in sys_output]
print free_memory
For 4 nodes, this returns [mem1],[mem2],[mem3],[mem4].
How can I combine each memory value into a single list? I'm having difficulty assigning free_memory as a list instead of a string which is replaced after each loop iteration.
Once I have a memory list, I should be able to combine it with the node_names list to make a dictionary file and do any necessary sorting.
I would recommend just building the dictionary directly:
import commands
node_free_mem = {}
for n in node_names:
sys_output = commands.getoutput("ssh %s 'free -m'" % n)
free_memory = sys_output.split()[9]
node_free_mem[n] = int(free_memory)
Here's code that does exactly what you asked: it builds a list, then uses the list to make a dictionary. Discussion after the code.
import commands
def get_free_mem(node_name):
sys_output = commands.getoutput('ssh %s \'free -m\'' % node_name)
free_memory = sys_output.split()[9]
return int(free_memory)
free_list = [get_free_mem(n) for n in node_names]
node_free_mem = dict(zip(node_names, free_list))
Note that in both code samples I simply iterate over the list of node names, rather than using a range() to get index numbers and indexing the list. It's simplest and best in Python to just ask for what you want: you want the names, so ask for those.
I made a helper function for the code to get free memory. Then a simple list comprehension builds a parallel list of free memory values.
The only tricky bit is building the dict. This use of zip() is actually pretty common in Python and is discussed here:
Map two lists into a dictionary in Python
For large lists in Python 2.x you might want to use itertools.izip() instead of the built-in zip(), but in Python 3.x you just use the built-in zip().
EDIT: cleaned up the code; it should work now.
commands.getoutput() returns a string. There is no need to package up the string inside a list, so I removed the square braces. Then in turn there is no need for a list comprehension to get out the free_memory value; just split the string. Now we have a simple string that may be passed to int() to convert to integer.
Related
I have a set with multiple tuples: set1 = {(1,1),(2,1)} for example.
Now I want to pass each tuple of the set to a method with this signature: process_tuple(self, tuple).
I am doing it with a for loop like this:
for tuple in set1:
process_tuple(tuple)
Is there a better way to do it?
Your question is basically "how can I loop without using a loop". While it's possible to do what you're asking with out an explicit for loop, the loop is by far the clearest and best way to go.
There are some alternatives, but mostly they're just changing how the loop looks, not preventing it in the first place. If you want to collect the return values from the calls to your function in a list, you can use a list comprehension to build the list at the same time as you loop:
results = [process_tuple(tuple) for tuple in set1]
You can also do set or dict comprehensions if those seem useful to your specific needs. For example, you could build a dictionary mapping from the tuples in your set to their processed results with:
results_dict = {tuple: process_tuple(tuple) for tuple in set1}
If you don't want to write out for tuple in set1 at all, you could use the builtin map function to do the looping and passing of values for you. It returns an iterator, which you'll need to fully consume to run the function over the full input. Passing the map object to list sometimes makes sense, for instance, to convert inputs into numbers:
user_numbers = list(map(int, input("Enter space-separated integers: ").split()))
But I'd also strongly encourage you to think of your current code as perhaps the best solution. Just because you can change it to something else, doesn't mean you should.
I am learning lists and trying to create a list and add data to it.
mylist=[]
mylist[0]="hello"
This generates Error.
Why cant we add members to lists like this, like we do with arrays in javascript.
Since these are also dynamic and we can add as many members and of any data type to it.
In javascript this works:
var ar=[];
ar[0]=333;
Why this dosent work in Python and we only use append() to add to list.
mylist[0] = 'hello' is syntactic sugar for mylist.__setitem__(0, 'hello').
As per the docs for object.__setitem__(self, key, value):
The same exceptions should be raised for improper key values as for
the __getitem__() method.
The docs for __getitem__ states specifically what leads to IndexError:
if value outside the set of indexes for the sequence (after any
special interpretation of negative values), IndexError should be
raised.
As to the purpose behind this design decision, one can write several chapters to explain why list has been designed in this way. You should familiarise yourself with Python list indexing and slicing before making judgements on its utility.
Lists in Python are fundamentally different to arrays in languages like C. You do not create a list of a fixed size and assign elements to indexes in it. Instead you either create an empty list and append elements to it, or use a list-comprehension to generate a list from a type of expression.
In your case, you want to add to the end, so you must use the .append method:
mylist.append('hello')
#["hello"]
And an example of a list comprehension:
squares = [x**2 for x in range(10)]
#[1,4,9,16,25,36,49,64,81,100]
This question already has answers here:
dict.keys()[0] on Python 3 [duplicate]
(3 answers)
Closed 6 years ago.
I just wanna make sure that in Python dictionaries there's no way to get just a key (with no specific quality or relation to a certain value) but doing iteration. As much as I found out you have to make a list of them by going through the whole dictionary in a loop. Something like this:
list_keys=[k for k in dic.keys()]
The thing is I just need an arbitrary key if the dictionary is not empty and don't care about the rest. I guess iterating over a long dictionary in addition to creation of a long list for just randomly getting a key is a whole lot overhead, isn't it?
Is there a better trick somebody can point out?
Thanks
A lot of the answers here produce a random key but the original question asked for an arbitrary key. There's quite a difference between those two. Randomness has a handful of mathematical/statistical guarantees.
Python dictionaries are not ordered in any meaningful way. So, yes, accessing an arbitrary key requires iteration. But for a single arbitrary key, we do not need to iterate the entire dictionary. The built-in functions next and iter are useful here:
key = next(iter(mapping))
The iter built-in creates an iterator over the keys in the mapping. The iteration order will be arbitrary. The next built-in returns the first item from the iterator. Iterating the whole mapping is not necessary for an arbitrary key.
If you're going to end up deleting the key from the mapping, you may instead use dict.popitem. Here's the docstring:
D.popitem() -> (k, v), remove and return some (key, value) pair as a 2-tuple;
but raise KeyError if D is empty.
You can use random.choice
rand_key = random.choice(dict.keys())
And this will only work in python 2.x, in python 3.x dict.keys returns an iterator, so you'll have to do cast it into a list -
rand_key = random.choice(list(dict.keys()))
So, for example -
import random
d = {'rand1':'hey there', 'rand2':'you love python, I know!', 'rand3' : 'python has a method for everything!'}
random.choice(list(d.keys()))
Output -
rand1
You are correct: there is not a way to get a random key from an ordinary dict without using iteration. Even solutions like random.choice must iterate through the dictionary in the background.
However you could use a sorted dict:
from sortedcontainers import SortedDict as sd
d = sd(dic)
i = random.randrange(len(d))
ran_key = d.iloc[i]
More here:.
http://www.grantjenks.com/docs/sortedcontainers/sorteddict.html
Note that whether or not using something like SortedDict will result in any efficiency gains is going to be entirely dependent upon the actual implementation. If you are creating a lot of SD objects, or adding new keys very often (which have to be sorted), and are only getting a random key occasionally in relation to those other two tasks, you are unlikely to see much of a performance gain.
How about something like this:
import random
arbitrary_key = random.choice( dic.keys() )
BTW, your use of a list comprehension there really makes no sense:
dic.keys() == [k for k in dic.keys()]
check the length of dictionary like this, this should do !!
import random
if len(yourdict) > 0:
randomKey = random.sample(yourdict,1)
print randomKey[0]
else:
do something
randomKey will return a list, as we have passed 1 so it will return list with 1 key and then get the key by using randomKey[0]
Okay I concede that I didn't ask the question very well. I will update my question to be more precise.
I am writing a function that takes a list as an argument. I want to check the length of the list so I can loop through the list.
The problem that I have is when the list has only one entry, len(myList) returns the length of that entry (the length of the string) and not the length of the list which should be == 1.
I can fix this if I force the argument to be parsed as a single value list ['val']. But I would prefer my API to allow the user to parse either a value or a list of values.
example:
def myMethod(self,dataHandle, data,**kwargs):
comment = kwargs.get('comment','')
_dataHandle= list()
_data = list()
_dataHandle.append(dataHandle)
_data.append(data)
for i in range(_dataHandle):
# do stuff.
I would like to be able to call my method either by
myMethod('ed', ed.spectra,comment='down welling irradiance')
or by
myMethod(['ed','lu'] , [ed.spectra,lu.spectra] , comments = ['downwelling', upwelling radiance'])
Any help would be greatly appreciated. Might not seem like a big deal to parse ['ed'], but it breaks the consistency of my API so far.
The proper python syntax for a list consisting of a single item is [ 'ed' ].
What you're doing with list('ed') is asking python to convert 'ed' to a list. This is a consistent metaphor in python: when you want to convert something to a string, you say str(some_thing). Any hack you'd use to make list('ed') return a list with just the string 'ed' would break python's internal metaphors.
When python sees list(x), it will try to convert x to a list. If x is iterable, it does something more or less equivalent to this:
def make_list(x):
ret_val = []
for item in x:
ret_val.append(item)
return ret_val
Because your string 'ed' is iterable, python will convert it to a list of length two: [ 'e', 'd' ].
The cleanest idiomatic python in this case might be to have your function accept a variable number of arguments, so instead of this
def my_func(itemList):
...
you'd do this
def my_func(*items):
...
And instead of calling it like this
my_func(['ed','lu','lsky'])
You'd call it like this:
my_func('ed', 'lu', 'lsky')
In this way you can accept any number of arguments, and your API will be nice and clean.
You can ask if your variable is a list:
def my_method(my_var):
if isinstance(my_var, list):
for my_elem in my_var:
# do stuff with my_elem
else: # my_var is not iterable
# do stuff with my_var
EDIT: Another option is to try iterating over it, and if it fails (raises and exception) you assume is a single element:
def my_method(my_var):
try:
for my_elem in my_var:
# do stuff with my_elem
except TypeError: # my_var is not iterable
# do_stuff with my_var
The good thing about this second options is that it will work not only for lists, as the first one, but with anything that is iterable (strings, sets, dicts, etc.)
You do actually need to put your string in a list if you want it to be treated like a list
EDIT
I see that at some point there was a list in front of the string. list, contrary to what you may think, doesn't create a list of one item. It calls __iter__ on the string object and iterates over each item. Thus it makes a list of characters.
Hopefully this makes it clearer:
>>> print(list('abc'))
['a', 'b', 'c']
>>> print(list(('abc',)))
['abc']
list('ed') does not create a list containing a single element, 'ed'. list(x) in general does not create a list containing a single element, x. In fact, if you had been using numbers rather than strings (or anything else non-iterable), this would have been blindingly obvious to you:
>>> list('ed')
['e', 'd']
>>> list(3)
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
list(3)
TypeError: 'int' object is not iterable
>>
So you are in fact passing a list with multiple elements to your method, which is why len is returning greater than 1. It's not returning the length of the first element of the list.
For your method to allow passing either a single item or a list, you'd have to do some checking to see if it's a single item first, and if it is create a list containing it with myVar = [myVar], then run your loop.
However this sort of API is tricky to implement and use, and I would not recommend it. The most natural way to check if you've been given a collection or an item is see if myVar is iterable. However this fails for strings, which are iterable. Strings unfortunately straddle the boundry between a collection and an individual data item; we very very often use them as data items containing a "chunk of text", but they are also collections of characters, and Python allows them to be used as such.
Such an API also is likely to cause you to one day accidentally pass a list that you're thinking of as a single thing and expecting the method to treat it as a single thing. But it's a list, so suddenly the code will behave differently.
It also raises questions about what you do with other data types. A dictionary is not a list, but it can be iterated. If you pass a dictionary as myVar, will it be treated as a list containing a single dictionary, or will it iterate over the keys of the dictionary? How about a tuple? What about a custom class implementing __iter__? What if the custom class implementing __iter__ is trying to be "string-like" rather than "list-like"?
All these questions lead to surprises if the caller guesses/remembers wrongly. Surprises when programming lead to bugs. IMHO, it's better to just live with the extra two characters of typing ([ and ]), and have your API be clean and simple.
I run into this same problem frequently. Building a list from an empty list, as you are doing with the "_dataHandle= list()" line, is common in Python because we don't reserve memory in advance. Therefore, it is often the case that the state of the list will transition from empty, to one element, to multiple elements. As you found, Python treats the indexing different for one element vs. multiple elements. If you can use list comprehension, then the solution can be simple. Instead of:
for i in range(_dataHandle):
use:
for myvar in _dataHandle:
In this case, if there is only one element, the loop only iterates once as you would expect.
I have a Dictionary of Classes where the classes hold attributes that are lists of strings.
I made this function to find out the max number of items are in one of those lists for a particular person.
def find_max_var_amt(some_person) #pass in a patient id number, get back their max number of variables for a type of variable
max_vars=0
for key, value in patients[some_person].__dict__.items():
challenger=len(value)
if max_vars < challenger:
max_vars= challenger
return max_vars
What I want to do is rewrite it so that I do not have to use the .iteritems() function. This find_max_var_amt function works fine as is, but I am converting my code from using a dictionary to be a database using the dbm module, so typical dictionary functions will no longer work for me even though the syntax for assigning and accessing the key:value pairs will be the same. Thanks for your help!
Since dbm doesn't let you iterate over the values directly, you can iterate over the keys. To do so, you could modify your for loop to look like
for key in patients[some_person].__dict__:
value = patients[some_person].__dict__[key]
# then continue as before
I think a bigger issue, though, will be the fact that dbm only stores strings. So you won't be able to store the list directly in the database; you'll have to store a string representation of it. And that means that when you try to compute the length of the list, it won't be as simple as len(value); you'll have to develop some code to figure out the length of the list based on whatever string representation you use. It could just be as simple as len(the_string.split(',')), just be aware that you have to do it.
By the way, your existing function could be rewritten using a generator, like so:
def find_max_var_amt(some_person):
return max(len(value) for value in patients[some_person].__dict__.itervalues())
and if you did it that way, the change to iterating over keys would look like
def find_max_var_amt(some_person):
dct = patients[some_person].__dict__
return max(len(dct[key]) for key in dct)