I need to create a new dataset variable everytime within a for loop
using .append as below wont work. Note the shape of each numpy array type variable is (56, 25000)
ps=[1,2,3,4]
for subj in ps:
datapath = '/home/subj%d' % (subj)
mydata.append = np.genfromtext(datapath, mydatafile)
so basically I need her 4 instances of mydata, each with a shape of (56, 25000), or that for each loop a new dataset variable is created eg mydata1, ..., mydata4....however .append won't do it. I could do this with
if ps==1: mydata1 = np.genfromtext(datapath, mydatafile)
if ps==2: mydata2 = np.genfromtext(datapath, mydatafile)
etc but I have far to many instances of ps, so would be nice to loop it
thanks!
It's hard to say without more code, but .append is generally a method, and should be called like this:
some_container.append(your_object)
Note I'm also initializing mydata to be an empty list -- you don't show how you initialize it (if you do at all), so just be aware:
mydata = []
for subj in [1,2,3,4]:
datapath = '/home/subj%d' % (subj)
mydata.append( np.genfromtext(datapath, mydatafile) )
Then, mydata will be a 4-element Python list of numpy arrays.
There is also numpy's vstack() and concatenate() functions which may be worth looking in to.
Lastly, just wanted to point out that
ps = [1,2,3,4]
for sub in ps:
...
Can be written as (as I do above):
for sub in [1,2,3,4]:
...
but also as:
for sub in range(1,5):
...
# or
for sub in range(4):
datapath = '/home/subj%d' % (subj + 1)
...
Related
I notice it is possible to access a variable in a Nd Array items Container using dot notation.
This is especially true for a file produced by the loadmat package of scipy.
For example,the following is a variable from the Nd Array items Container, that accessed using the dot notation.
dot_notation_output=stru[0].fieldA
Im curios how to reproduce something similar for a given nested dict as below.
struc=[{'fieldA': 11.02, 'fieldB': 2.69,'fieldC': 2.69}, {"fieldA": 21.4, "fieldB": 66.69,'fieldC': 2.69},
{"fieldA": 100,"fieldB": 200,'fieldC': 2.69}]
Creating directly from np.array does not reproduce the finding above
np.array(struc)
The struc from Matlab was created as follow
for idx = 1:3
stru(idx) = create_Structure();
stru(idx).fieldA = '1';
stru(idx).fieldB = 3;
stru(idx).fieldC = 44;
end
save('struc_mat.mat','stru')
function s = create_Structure()
%% Create a structure
s = struct( ...
'fieldA', NaN,'fieldB', NaN,'fieldC',NaN);
end
The struc produced in Matlab, can be open in Python with the scipy-loadmat
from scipy.io import loadmat
stru = loadmat ( 'struc_mat.mat', squeeze_me=True, struct_as_record=False )
stru = stru['stru']
dot_notation_output=stru[0].fieldA
Once you know how to define a record array, the process to create one from your dictionary is relatively straightforward.
First you have to get the dictionary keys (to use as field names).
Use these to define the "dtype" for the recarray. (Note: You also need to define each fields "dtype". I assumed floats. You can add logic to check the dictionary value types to ensure an appropriate type is used.)
Use the dtype to create an empty recarray.
Finally, loop thru the dictionary (again) to populate the array based on field names and list position (used as array index).
Note, you can reference array values in 2 ways: 1) with dot notation you described: recarr[0].fieldA, or 2) with the name as an array index: recarr[0]['fieldA']. I prefer the second method as it gives a programtic way to access the values when the field name is a variable, and not hard-coded.
Code to create a recarray with your data below:
import numpy as np
struc=[{'fieldA': 11.02,'fieldB': 2.69, 'fieldC': 2.69},
{'fieldA': 21.4, 'fieldB': 66.69,'fieldC': 2.69},
{'fieldA': 100, 'fieldB': 200, 'fieldC': 2.69}]
keys = []
for d in struc:
for k in d.keys():
if k not in keys:
keys.append(k)
dt = np.dtype([ (name,float) for name in keys ])
recarr = np.recarray((len(struc),),dtype=dt)
print(recarr.dtype)
for i, d in enumerate(struc):
for key,val in d.items():
recarr[i][key] = val
print(recarr)
In Matlab I can assign values inside arrays as follows.
a = [];
a(end+1, 1:2) = [1,2];
a(end,3:4) = [3,4];
a(end+1, 1:2) = [5,6];
a(end,3:4) = [7,8];
and so on. But in Python I can use the append command to append an array to the existing array. e.g.
a = []
a.append([1,2,3,4])
a.append([5,6,7,8])
My problem is I should assign the first two values at some point and the next two values in some other point as shown in my Matlab code. How can I do that?
I think you're looking for extend:
a = []
a.append([1,2]) # [[1,2]]
a[-1].extend([3,4]) # [[1,2,3,4]]
When I run the following code and print "array" and "main", I am getting the same answer as "main" and "array". I am only modifying "main" but why is it affecting "array" as well. Can anyone help ?
array = ["a",0,0,"b","c","d",0,1,0,1,0,3,0,1,9,0,0,0,0,9]
main = array
for i in range(0,len(main)-1):
if main[i] == 0 or main[i] == 0.0:
main.pop(i)
main.append(0)
print main
print array
Variables and arrays (they are called "lists" in python) works differently. You are creating only an "alias" in that way. This happens because the two names ("main" and "array") will point out to the same memory address.
To avoid that you have different options:
1) Use copy() function in this way:
main = array.copy()
2) or:
import copy
main = copy.copy(array)
3) using "list" built-in method:
main = list(array)
4) creating a new empy list and extend it:
main = []
main.extend(array)
5) using slicing:
main = array[:]
6) If you are going to create a multi-dimensional list, above example are not enough. In this particular case you need to use deepcopy() function:
import copy
main = copy.deepcopy(array)
In python variables are references to an instance in memory. When you do:
main = array
you're assigning to main a reference to the same instance, that you can check with:
print(id(main))
print(id(array))
when you mutate a list you're iterating over, you're opening the door to scary and awful issues.
Your solution, as stated in a comment is to make a copy of the list:
main = array[:]
which is a shorter equivalent of:
main = []
for elt in array:
main.append(array)
meaning that the instances in the list are the same in both list, but the container is different.
But because that solution would mean you iterate twice over the list (once to make a copy, and a second time to change the values), a better way would be to actually iterate over a list, and then create a second list as you go:
main = []
for item in array:
if item in (0, 0.0):
main.append(0)
else:
main.append(item)
Nota Bene: In your snippet you're using the following way to iterate over your list:
for i in range(0,len(main)-1):
# use main[i]
that's not pythonic at all, instead of that, you just need to use the following more readable syntax:
for item in main:
# use item
Don't modify lists while iterating. Generate a new list:
array = ["a",0,0,"b","c","d",0,1,0,1,0,3,0,1,9,0,0,0,0,9]
main = [x for x in array if x != 0]
main.extend([0] * (len(main)-len(array)))
print main
print array
I want to know that when I defined a multi-dimension variables in Gurobi, how can I extract all the value of the solution and organize them in to a Numpy array according to the original coordinate of the variable.
I have the following decision variables defined in Gurobi using Python API:
for i in range(N):
for t in range(M):
Station_Size[i,t] = m.addVar(ub=Q, name = 'Station_Size_%s_%s' %(i,t))
for j in range(N):
Admission[i,j,t] = m.addVar(ub = Arrival_Rate[t,i,j], obj=-1, name = 'Admission_Rate_%s_%s_%s' %(i,j,t))
Return[i,j,t] = m.addVar(name = 'Return_Rate_%s_%s_%s' %(i,j,t))
I have the problem solved and I have three dictionary:
Station_Size, Admission and Return
I know that the solution can be accessed as:
Station_Size[i,t].X, Admission[i,j,t].X and Return[i,j,t].X
I want to creat three Numpy array such that:
Array_Station_Size[i,t] = Station_Size[i,t].X
Array_Admission[i,j,t] = Admission[i,j,t].X
I can definitely do this by creating three loops and creat the Numpy Array element by element. It's do-able if the loop doesn't take a lot of time. But I just want to know if there is a better way to do this. Please comment if I did not make myself clear.
Assuming your model's name is m, do the following:
Array_Station_Size = m.getAttr('x', Station_Size)
which is a gurobipy.tupledict now.
see gurobi doc here
http://www.gurobi.com/documentation/8.1/quickstart_windows/py_results.html
I figured this problem out.
Do the following:
Array_Station_Size = np.array()
Array_Station_Size[i,] = [Station_Size[i,t].X for t in rang(T)]
all:
I want to create a string array and then pass it to a class in python as following:
from plottert import plotter
at[0]='./Re100/17/0.001/R/Vx-H'
at[1]='./Re100/33/0.001/R/Vx-H'
at[2]='./Re100/65/0.001/R/Vx-H'
b[0]='./U-0.001-H'
plotter (at,b)
but I got an error showing name 'at' is not defined.
I know that at.append() will do work. But, what I really want is to add the value to a SPECIFIC index of the array I want. Any help?
You could simply fill it with empty strings if you want
at = [''] * n #n = length of list
at[0]='./Re100/17/0.001/R/Vx-H'
at[1]=...
However as others have mentioned, you never initialized your list in the first place.
If you want to assign to indexes without having to know the final size of your data structure, use a dictionary instead:
at = {}
at[0] = 'zero'
at[4] = 'four' # look, it's sparse
As you can see, this also has the advantage (over append) that you can assign in any order.
If you want to convert this to an array later, you can do something like this:
at_arr = [at[i] if i in at else None
for i in range(max(at.keys())+1)]
# at_arr now holds the array ['zero', None, None, None, 'four']
First, create the lists (there is no non-basic-type arrays):
at = [''] * n # n = size of at
b = [''] * m # m = size of b
then execute your code.
You cant use lists you havent defined.