When building a set of statistics from a dictionary, I process the various entries (such as by user). THus, I can build the various statistics for each user. While doing this, I also build the statistics for a dummy user that I can call "total". After the dictionary is completely built, I create a .csv file and output the statistics using the writerow method.
Since python iterates of the dictionary keys in no particular order, I want to cause the total user to print last. If I attempt to save the generated statistics into a save variable and then output it at the proper time, the save variable gets reset because python variables work by reference rather than value. That is the code
mystats = {}
totalstats = {}
for user in mydict
#perform calculations to generate mystats dictionary entries
if user == 'Total':
totalstats = mystats
else:
outfile.writerow(mystats)
outfile.writerow(totalstats)
However, the actual output of totalstats is whatever set of values had been put into mystats last.
Is there a decent way to show that totalstats is to keep the explicit values within mystats that I had at the time of the assignment or do I need to calculate all the statistics at the end or do
for stattype in mystats:
totalstats[stattype] = mystats[stattype]
While this works, I would rather have something of the type "totalstats = mystats' rather than do a large loop over the complete set of statistics or calculate the entire set of statistics for Total at the end of processing.
You can use copy.deepcopy:
from copy import deepcopy
totalstats = deepcopy(mystats)
If the dict doesn't contain mutable values then you can simply use dict.copy().
Related
I have a problem. I have a code that using selenium and getting information form different sites and put then into one list. And after all, python will delete all information in the list, I need to write them to the excel:
List = []
for values in List:
...
List.append(values)
List.append(some_information_from_selenium)
And in the end of iteration:
List.clear()
I need to save information before clear() and after cleat List, add new information to the excel. This iteration have limit - 100. Need to create a new excel file, and adding information to this. List will delete and then append new information, this iteration will be 100 times. I will have 18 columns and 100 rows. I can use whatever i want.
:UPD:
One more question: if i use
data = pd.DataFrame()
data({ "Name":List[some_index]
"Surname":List[some_index_1]
.... })
data.to_excel("Excel.xlsx")
Why I got error 'DataFrame' object is not callable and how can i solve this
I'm not 100% sure what you're trying to do from your code, but instead of clearing your list variable each time lets hold it in some sort of nested collection.
a simple dictionary will do.
from collections import defaultdict
data_dict = defaultdict(list)
for num in range(10,110,10): #call your func in iterations of 10s
your code
data_dict[i].append(some_information_from_selenium)
each iteration of 10 will hold your nested data.
data_dict[10]
which you can then pass into pandas.
So I have a list where each element is associated with a variable(s). If the user wants to read a variable value I would need to take an element of the list perform my operation and then return it to the user. The list is ~250 elements, where each element is defined as a different variable. The element number and variable do not change.
Do I need some form of lookup table, or equivalent? Does it go in the main code, or can I keep it separate as a config file, i.e. txt file containing: element 1 = variable y
I'm fairly new to Python, so just want to pointed in the right direction really.
The data structure you seem to be wanting is a dictionary. Dictionaries allow you to reference elements by their key i.e. a key could be a "variable" and the value associated with the key is essentially data that can be referenced by the "variable". Dictionaries can be made by defining a variable as {} or by using the dict() function on something. I most often will do something like:
dictionary = dict(zip(list_of_names , list_of_data))
The lists could be made by reading your txt file and then using:
string.split("your delimiter here")
My problem: I am composing a data source that I pass to the constControl(..., ds=ds, ...) that will be updated iteratively. It is the result of a series of calculations at every timestep coming from a model that I built. After passing the results of my model (loads) at every timestep, I want to call run_timeseries and store the results before going to the next iteration.
Trying to store some results (e.g. net.res_bus.vm_pu, ...) into a list fails to update the values after every iteration as if the data source object had constant values throughout the iterations, which it doesn't.
My solution: I found that calling OutputWriter() before the run_timeseries() and letting it output an excel file at every iteration successfully updates the results I am simultaneously storing in that empty list. So it works now!
My question: I don't want to be forced to output excel files every time I run_timeseries() in order for the results to be stored and/or successfully updated in the next iteration. I do not even need excel files at the moment. Is there any way to not having to call OutputWriter() but store net's result values in an empty list?
You don't neet to use OutputWriter(). It depends in what you're trying to do, if you create a loop that make the same as run_timeseries() that might work.
In my case I wanted to change the load and make a new power flow calculation every time the change was made.
A overview of the steps taken:
Create a loop
Each step change the load using map to go through all item in net.load.p_mw for example.
Call the run_control(net) to make the new power flow calculation.
Then you can create a list to collect the data and convert it in JSON.
data = all bus data
json = { dataBuses: data }
Once it's in JSON format you can even send to a database by a post request or whatever you desire.
I have a dictionary that I would like to write in whole to an NDB on App Engine. The problem is that only the last item in the dictionary is being written. I thought perhaps the writes were too fast so I put a sleep timer in with a very long wait of 20 seconds to see what would happen. I continually refreshed the Datastore Viewer and saw the transaction write, and then later get overwritten by the next transaction, etc. The table started out empty and the dictionary keys are unique. A simple example:
class Stats(ndb.Model):
desc= ndb.StringProperty(required = True)
count= ndb.IntegerProperty(required = True)
update = ndb.DateTimeProperty(auto_now_add = True)
class refresh(webapp2.RequestHandler):
def get(self):
statsStore = Stats()
dict = {"test1":0,"test2":1,"test3":2}
for key in dict:
statsStore.desc = key
statsStore.count = dict.get(key)
statsStore.put()
What will happen above is that only the final dictionary item will remain in the datastore. Again with a sleep timer I can see each being written but then overwritten. I am using this on my local machine with the local development GAE environment.
Appreciate the help.
The problem with your original code is that you're reusing the same entity (model instance).
During the first put(), a datastore key is generated and assigned to that entity. Then, all the following put() calls are using the same key.
Changing it to create a new model instance on each iteration (the solution you mention in your comment) will ensure a new datastore key is generated each time.
Another option would be to clear the key with "statsStore.key = None" before calling put(). But what you did is probably better.
Not sure what you are trying to do, but here are some hopefully helpful pointers. If you want to save the dict and then re-use it later by reading from the database, then change your string to a text property, import json, and save the dict as a json string value using json.dumps(). If you want to write an entity for every element in your dict, then you will want to move your statsStore class creation line inside the for loop, and finish the loop process by adding each Stats() classes to an array. Once the loop is done, you can batch put all the entities in the array. This batch approach is much faster than including a put() inside your loop which is most often a very non-performant design choice. If you just want to record all the values in the dict for later reference, and you have a value that you can safely use as a delimiter, then I would create two empty arrays prior to your loop, and append each desc and count inside the respective array. Once outside the array, you can save these values to two text properties in your entity by joining the arrays using the delimiter string. If you do this, then strongly suggest using urllib.quote() to escape your desc text value when appending it so at to avoid conflicts with your delimiter value.
Some final notese: You should be careful using this type of process with a StringProperty. You might easily exceed the string limit size depending on the number of items, and/or the length of your desc values. Also remember your items in the dict may not come out in the order you intend. Consider something like: "for k, v in sorted(mydict.items()):" HTH, stevep
I have a Dictionary of Classes where the classes hold attributes that are lists of strings.
I made this function to find out the max number of items are in one of those lists for a particular person.
def find_max_var_amt(some_person) #pass in a patient id number, get back their max number of variables for a type of variable
max_vars=0
for key, value in patients[some_person].__dict__.items():
challenger=len(value)
if max_vars < challenger:
max_vars= challenger
return max_vars
What I want to do is rewrite it so that I do not have to use the .iteritems() function. This find_max_var_amt function works fine as is, but I am converting my code from using a dictionary to be a database using the dbm module, so typical dictionary functions will no longer work for me even though the syntax for assigning and accessing the key:value pairs will be the same. Thanks for your help!
Since dbm doesn't let you iterate over the values directly, you can iterate over the keys. To do so, you could modify your for loop to look like
for key in patients[some_person].__dict__:
value = patients[some_person].__dict__[key]
# then continue as before
I think a bigger issue, though, will be the fact that dbm only stores strings. So you won't be able to store the list directly in the database; you'll have to store a string representation of it. And that means that when you try to compute the length of the list, it won't be as simple as len(value); you'll have to develop some code to figure out the length of the list based on whatever string representation you use. It could just be as simple as len(the_string.split(',')), just be aware that you have to do it.
By the way, your existing function could be rewritten using a generator, like so:
def find_max_var_amt(some_person):
return max(len(value) for value in patients[some_person].__dict__.itervalues())
and if you did it that way, the change to iterating over keys would look like
def find_max_var_amt(some_person):
dct = patients[some_person].__dict__
return max(len(dct[key]) for key in dct)