Develop Reference

Why is the change of values in one array is affecting the value on another array (with exact same values) that isn't involved in the operation?

When I run the command "negative_only[negative_only>0]=0" (which should make positive values = 0 on the array "negative_only") the values on a similar array ("positive_only") are also changed. Why is this happening? I'm using Python 3.7 (Windows 10 / Spyder IDE).
The code where the two arrays are being manipulated is below. The "long_dollars_ch" is an array of ~2700 x 60 with some positive values, some negative values and a lot of zeros. This code is part of a loop that is cycling through each row of the array "long_dollars_ch".
# calculations to isolate top contributors to NAV change for audits
top_check = 3 # number of top values changes to track
# calculate dollar change (for longs), and create array with most positive/negative values
long_dollars_ch[c_day,:] = long_shares[c_day,:]*hist_prices_zeros[c_day,:]-long_shares[c_day,:]*hist_prices_zeros[c_day-1,:]
positive_only = long_dollars_ch[c_day,:]
positive_only[positive_only<0]=0 #makes non-positive values zero
idx = np.argsort(positive_only) #creat index representing sorted values for only_positive for c_day
non_top_vals = idx[:-top_check]
negative_only = long_dollars_ch[c_day,:]
negative_only[negative_only>0]=0 #makes non-negative values zero
idx = np.argsort(negative_only) #creat index representing sorted values for only_negative for c_day
non_bottom_vals = idx[:-top_check]
# create array that shows the most positive/negative dollar change for "top-check" securities
long_dollars_ch_pos[c_day,:] = positive_only
long_dollars_ch_pos[c_day,:][non_top_vals] *= 0
long_dollars_ch_neg[c_day,:] = negative_only
long_dollars_ch_neg[c_day,:][non_bottom_vals] *= 0
The objective of this code is to create two arrays. One that has only the top "top_check" positive values (if any) and another that has the bottom "top_check" negative values (if any) for each row of the original array "long_dollars_ch". However, it appears that Python is considering the "positive_only" and "negative_only" the same "variable". And therefore, the operation with one of them affects the values inside the other (that was not part of the operation).

its quite simple.
In numpy np.array x = np.array y you do not copy array :)
You make a reference to array x.
In other words, you do not have two array after using "=". You still have one array x, and a reference to that array (y is that reference).
positive_only = long_dollars_ch[c_day,:]
.
.
,
negative_only = long_dollars_ch[c_day,:]
do not make a copy of long_dollars_ch, but only makes references to it.
you need to use copy method, or other method (numpy provides few of them) to make it work.
Here is a documentation
EDIT: I posted wrong link, now it is ok.

Storing data with Python arrays (HAR-RV Credit Risk model implementation)

that's my first time here at Stack and I'm on my first days with Python.
I'm dealing with the HAR-RV model, trying to run this equation but having no success at all to store my operations on the array
Here what I'm trying to calculate:
r_[t,i] = Y_[t,i] - Y_[t,i-1]
https://cdn1.imggmi.com/uploads/2019/8/30/299a4ab026de7db33c4222b30f3ed70a-full.png
Im using the first relation here, where "r" means return and "Y" the stock prices
t = 10 # daily intervals
i = 30 # number of days
s = 1
# (Here I've just created some fake numbers, intending to simulate some stock prices)
Y = abs(np.random.random((10,30)) + 1)
# initializing my return array
return = np.array([])
# (I also tried to initialize it as a Matrix before..) :
return = np.zeros((10,30))
# here is my for loop to store each daily return at its position on the "return" Array. I wanted an Array but got just "() size"
for t in range(0,9):
for i in range(1,29):
return = np.array( Y.item((t,i)) - Y.item((t,i-1)) )
... so, I was expecting something like this:
return = [first difference, second difference, third difference...]
how can I do that ?

first don't use return as a variable name in python as it is a python keyword (it signifies the result of a function). I have changed your variable return to ret_val.
You are wanting to make a change at each position in your array, so make the following change to your for loop:
for t in range(0,10):
for i in range(1,30):
ret_val[t][i] = Y[t][i] - Y[t][i-1]
print(ret_val)
This is saying to change the value at index ret_val[t][i] with the result of subtracting the values at that specific index in Y. You should see an array of the same shape when you print.
Also, the range function in python does not include the upper number. So when you say, for i in range(0,9) you are saying to include numbers 0-8. For your array, you'll want to do for i in range(0,10) to include all values in your array. Correspondingly, you'll want to do the same for i in range(1,30).

Octave/Matlab version of this Python for-loop

I just want to know if there is any Octave/Matlab equivalent syntax for this particular for-loop in python:
for (i,j) in [(1,2),(2,3),(3,4),(4,5),(5,6),(6,7)]:
a[i,j] = 1
I need it to ease out my Image processing assignments where I can easily construct Image matrix without having to enter each pixel value for almost each element of the Image matrix. So, if there are any other ways of implementing the above functionality in Octave/Matlab, then please let me know.
Thanks.

In Octave ,I guess also in MATLAB, you can do:
for ij = [{1;2} {2;3} {3;4} {4;5} {5;6} {6;7}]
a(ij{:}) = 1;
end
But in general In MATLAB and Python it is better to prevent loops. There are much efficient indexing methods both in Python and MATLAB.

If you want to set a series of pixels in a, given by coordinates, to the same value, you can do as follows:
coord = [1,2; 2,3; 3,4; 4,5; 5,6; 6,7];
ind = sub2ind(size(a), coord(:,1), coord(: 2));
a(ind) = 1;
You can replace that last 1 with a vector with as many elements as coordinates in coord to assign a different value to each pixel.
Note that MATLAB indexes rows with the first index, so the first column of coord corresponds to the y coordinate.

The simplest here would be:
for i = 1 : 6
a(i, i+1) = 1; % Alternatively: j=i+1; a(i,j)=1;
end
The more flexible alternative is to construct the pairs:
vals = [1,2; … ; 6,7]; % Your i,j pairs. Possibly even put 3 numbers there, i,j,value.
for i = 1 : size(vals, 1)
a(vals(i,1), vals(i,2)) = 1;
end

create loop using values from an array

I have an array D of variable length,
I want to create a loop that performs a sum based on the value of D corresponding to the number of times looped
i.e. the 5th run through the loop would use the 5th value in my array.
My code is:
period = 63 # can be edited to an input() command for variable periods.
Mrgn_dec = .10 # decimal value of 10%, can be manipulated to produce a 10% increase/decrease
rtn_annual = np.arange(0.00,0.15,0.05) # creates an array ??? not sure if helpful
sig_annual = np.arange(0.01,0.31,0.01) #use .31 as python doesnt include the upper range value.
#functions for variables of daily return and risk.
rtn_daily = (1/252)*rtn_annual
sig_daily = (1/(np.sqrt(252)))*sig_annual
D=np.random.normal(size=period) # unsure of range to use for standard distribution
for i in range(period):
r=(rtn_daily+sig_daily*D)
I'm trying to make it so my for loop is multiplied by the value for D of each step.
So D has a random value for every value of period, where period represents a day.
So for the 8th day I want the loop value for r to be multiplied by the 8th value in my array, is there a way to select the specific value or not?
Does the numpy.cumprod command offer any help, I'm not sure how it works but it has been suggested to help the problem.

You can select element in an iterative object (such as D in your code) simply by choosing its index. Such as:
for i in range(period):
print D[i]
But in your code, rtn_daily and sig_daily are not in the same shape, I assume that you want to add sig_daily multiply by D[i] in each position of rtn. so try this:
# -*- coding:utf-8 -*-
import numpy as np
period = 63 # can be edited to an input() command for variable periods.
Mrgn_dec = .10 # decimal value of 10%, can be manipulated to produce a 10% increase/decrease
rtn_annual = np.repeat(np.arange(0.00,0.15,0.05), 31) # creates an array ??? not sure if helpful
sig_annual = np.repeat(np.arange(0.01,0.31,0.01), 3) #use .31 as python doesnt include the upper range value.
#functions for variables of daily return and risk.
rtn_daily = (float(1)/252)*rtn_annual
sig_daily = (1/(np.sqrt(252)))*sig_annual
D=np.random.normal(size=period) # unsure of range to use for standard distribution
print D
for i in range(period):
r=(rtn_daily[i]+sig_daily[i]*D[i])
print r
Last of all, if you are using python2, the division method is for integer, so that means 1/252 will give you zero as result.
a = 1/252 >-- 0
to solve this you may try to make it float:
rtn_daily = (float(1)/252)*rtn_annual

Right now, D is just a scalar.
I'd suggest reading https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.random.normal.html to learn about the parameters.
If you change it to:
D=np.random.normal(mean,stdev,period)
you will get a 1D array with period number of samples, where mean and stdev are your mean and standard deviation of the distribution. Then you change the loop to:
for i in range(period):
r=(rtn_daily+sig_daily*D[i])
EDIT: I don't know what I was thinking when I read the code the first time. It was a horribly bad read on my part.
Looking back at the code, a few things need to happen to make it work.
First:
rtn_annual = np.arange(0.00,0.15,0.05)
sig_annual = np.arange(0.01,0.31,0.01)
These two lines need to be fixed so that the dimensions of the resulting matricies are the same.
Then:
rtn_daily = (1/252)*rtn_annual
Needs to be changed so it doesn't zero everything out -- either change 1 to 1.0 or float(1)
Finally:
r=(rtn_daily+sig_daily*D)
needs to be changed to:
r=(rtn_daily+sig_daily*D[i])
I'm not really sure of the intent of the original code, but it appears as though the loop is unnecessary and you could just change the loop to:
r=(rtn_daily+sig_daily*D[day])
where day is the day you're trying to isolate.

Making histogram out of matrix entries?

Today my task is to make a histogram to represent the operation of A^n where A is a matrix, but only for specific entries in the matrix.
For example, say I have a matrix where the rows sum to one. The first entry is some specific decimal number. However, if I raise that matrix to the 2nd power, that first entry becomes something else, and if I raise that matrix to the 3rd power, it changes again, etc - ad nauseum, and that's what I need to plot.
Right now my attempt is to create an empty list, and then use a for loop to add the entries that result from matrix multiplication to the list. However, all that it does is print the result from the final matrix multiplication into the list, rather than printing its value at each iteration.
Here's the specific bit of code that I'm talking about:
print("The intial probability matrix.")
print(tabulate(matrix))
baseprob = []
for i in range(1000):
matrix_n = numpy.linalg.matrix_power(matrix, s)
baseprob.append(matrix_n.item(0))
print(baseprob)
print("The final probability matrix.")
print(tabulate(matrix_n))
Here is the full code, as well as the output I got.
http://pastebin.com/EkfQX2Hu

Of course it only prints the final value, you are doing the same operation, matrix^s, 1000 times. You need to have s change each of those 1000 times.
If you want to calculate all values in location matrix(0) for matrix^i where i is each value from 1 to s (your final power) do:
baseprob = []
for i in range(1,s): #changed to do a range 1-s instead of 1000
#must use the loop variable here, not s (s is always the same)
matrix_n = numpy.linalg.matrix_power(matrix, i)
baseprob.append(matrix_n.item(0))
Then baseprob will hold matrix(0) for matrix^1, matrix^2, etc. all the way to matrix^s.