Why aren't my data being masked? - python

data = [[0, 1, 1, 5, 5, 5, 0, 2, 2, 2, 2, 2, 2, 2, 6, 6, 6, 6, 6, 6, 6, 6],
[1, 1, 1, 0, 5, 5, 5, 0, 2, 2, 0, 0, 2, 0, 0, 6, 6, 6, 0, 0, 6, 6],
[1, 1, 1, 0, 0, 0, 0, 0, 2, 2, 0, 2, 2, 2, 0, 0, 2, 6, 0, 0, 6, 6]]
The data object i have is a <class 'numpy.ndarray'>
Knowing data is a numpy object i did the following:
data = np.array(data)
i want to set the numbers inside a list i give as input to 0, what i tried:
data[~np.isin(data,[2,4])] = 0
i expect all the 2 and 4 occurrences in the previous matrix to be 0 and the rest to keep their values, what i got:
TypeError: only integer scalar arrays can be converted to a scalar index
also tried to give data as a numpy array using np.array gave error as well.


You should not negate the mask from np.isin check if you intend to set those matching values to 0. The below code works just fine:
Also, you should make the data a numpy array instead of list of lists.
In [10]: data = np.array([[0, 1, 1, 5, 5, 5, 0, 2, 2, 2, 2, 2, 2, 2, 6, 6, 6, 6, 6, 6, 6, 6],
...: [1, 1, 1, 0, 5, 5, 5, 0, 2, 2, 0, 0, 2, 0, 0, 6, 6, 6, 0, 0, 6, 6],
...: [1, 1, 1, 0, 0, 0, 0, 0, 2, 2, 0, 2, 2, 2, 0, 0, 2, 6, 0, 0, 6, 6]])
...:
In [11]: data[np.isin(data, [2, 4])] = 0
In [12]: data
Out[12]:
array([[0, 1, 1, 5, 5, 5, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 6, 6, 6, 6, 6, 6],
[1, 1, 1, 0, 5, 5, 5, 0, 0, 0, 0, 0, 0, 0, 0, 6, 6, 6, 0, 0, 6, 6],
[1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 6, 6]])
Just to reproduce your error:
In [13]: data = [[0, 1, 1, 5, 5, 5, 0, 2, 2, 2, 2, 2, 2, 2, 6, 6, 6, 6, 6, 6, 6, 6],
...: [1, 1, 1, 0, 5, 5, 5, 0, 2, 2, 0, 0, 2, 0, 0, 6, 6, 6, 0, 0, 6, 6],
...: [1, 1, 1, 0, 0, 0, 0, 0, 2, 2, 0, 2, 2, 2, 0, 0, 2, 6, 0, 0, 6, 6]]
...:
In [14]: data[np.isin(data, [2, 4])] = 0
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-14-06ee1662f1f2> in <module>()
----> 1 data[np.isin(data, [2, 4])] = 0
TypeError: only integer scalar arrays can be converted to a scalar index

Related

How to create a list of scales using functional programming in Python?

I'm adapting code I have C to Python. I would like to re-write this piece of my code using the functional paradigm in Python. My current code:
X = [0, 1, 2, 3, 4, 3, 2, 2, 3, 3, 2, 3, 3, 3, 2, 3, 2, 1, 1, 4, 3, 4, 4, 5, 1, 4, 2, 3, 3, 3, 2, 4, 4, 1, 3, 3, 2, 4, 1, 3, 3, 5, 5, 4, 4, 3, 2, 3, 4, 2, 1, 3, 4, 1, 2, 3, 5, 2, 3, 4, 3, 2, 3, 2, 3, 3, 2, 3, 3, 3, 4, 3, 1, 3, 3, 3, 2, 3, 5, 2, 1, 2, 3, 2, 1, 4, 1, 3, 2, 1, 3, 2, 3, 5, 2, 4, 1, 3, 4, 3, 3, 2, 4, 3, 4, 4, 3, 2, 1, 2, 3, 3, 4, 1, 4, 4, 3, 3, 3, 4, 3]
res = [0] * 121
for i in range(1, 31):
k = 0
for _ in range(0, 4):
res[i] += X[i + k]
k = k + 30
print(res)
How can I make this more Pythonic? Here reproduce a necessary scale in my algorithm with the following output:
[0, 9, 12, 11, 12, 13, 7, 12, 11, 12, 12, 13, 13, 13, 13, 13, 9, 9, 11, 10, 8, 12, 15, 9, 8, 15, 11, 11, 11, 12, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Thanks,
EDMC.

You might consider re-thinking your code as it basically re-assigns the zero-filled list of the length of the input list.
That shall be the thing that, to my current understanding, protects your code from being re-written in functional style.
However, packed your loop into more compact form:
for j in range(30):
for i in range(4):
res[1 + j] += X[1 + i * 30 + j]
UPD: After Re-Thinking This, Came up with the Following:
Suggest dropping the leading 0 as it does not seemingly bear any meaningful data:
X.pop(0)
Then, construct res without those trailing 0-s in functional style as desired:
res = [sum(X[_::30]) for _ in range(30)]
Trust that solves your task.

Permutation without repetition, efficient way

N = 14
SIZE = 6
lst = range(N+1)
sum_n_combs = [
list(comb) for comb in it.combinations_with_replacement(lst, SIZE)
if sum(comb) == N
]
print(sum_n_combs)
output [[0, 0, 0, 0, 0, 14], [0, 0, 0, 0, 1, 13], [0, 0, 0, 0, 2, 12], [0, 0, 0, 0, 3, 11], [0, 0, 0, 0, 4, 10], [0, 0, 0, 0, 5, 9], [0, 0, 0, 0, 6, 8], [0, 0, 0, 0, 7, 7], [0, 0, 0, 1, 1, 12], [0, 0, 0, 1, 2, 11], [0, 0, 0, 1, 3, 10], [0, 0, 0, 1, 4, 9], [0, 0, 0, 1, 5, 8], [0, 0, 0, 1, 6, 7], [0, 0, 0, 2, 2, 10], [0, 0, 0, 2, 3, 9], [0, 0, 0, 2, 4, 8], [0, 0, 0, 2, 5, 7], [0, 0, 0, 2, 6, 6], [0, 0, 0, 3, 3, 8], [0, 0, 0, 3, 4, 7], [0, 0, 0, 3, 5, 6], [0, 0, 0, 4, 4, 6], [0, 0, 0, 4, 5, 5], [0, 0, 1, 1, 1, 11], [0, 0, 1, 1, 2, 10], [0, 0, 1, 1, 3, 9], [0, 0, 1, 1, 4, 8], [0, 0, 1, 1, 5, 7], [0, 0, 1, 1, 6, 6], [0, 0, 1, 2, 2, 9], [0, 0, 1, 2, 3, 8], [0, 0, 1, 2, 4, 7], [0, 0, 1, 2, 5, 6], [0, 0, 1, 3, 3, 7], [0, 0, 1, 3, 4, 6], [0, 0, 1, 3, 5, 5], [0, 0, 1, 4, 4, 5], [0, 0, 2, 2, 2, 8], [0, 0, 2, 2, 3, 7], [0, 0, 2, 2, 4, 6], [0, 0, 2, 2, 5, 5], [0, 0, 2, 3, 3, 6], [0, 0, 2, 3, 4, 5], [0, 0, 2, 4, 4, 4], [0, 0, 3, 3, 3, 5], [0, 0, 3, 3, 4, 4], [0, 1, 1, 1, 1, 10], [0, 1, 1, 1, 2, 9], [0, 1, 1, 1, 3, 8], [0, 1, 1, 1, 4, 7], [0, 1, 1, 1, 5, 6], [0, 1, 1, 2, 2, 8], [0, 1, 1, 2, 3, 7], [0, 1, 1, 2, 4, 6], [0, 1, 1, 2, 5, 5], [0, 1, 1, 3, 3, 6], [0, 1, 1, 3, 4, 5], [0, 1, 1, 4, 4, 4], [0, 1, 2, 2, 2, 7], [0, 1, 2, 2, 3, 6], [0, 1, 2, 2, 4, 5], [0, 1, 2, 3, 3, 5], [0, 1, 2, 3, 4, 4], [0, 1, 3, 3, 3, 4], [0, 2, 2, 2, 2, 6], [0, 2, 2, 2, 3, 5], [0, 2, 2, 2, 4, 4], [0, 2, 2, 3, 3, 4], [0, 2, 3, 3, 3, 3], [1, 1, 1, 1, 1, 9], [1, 1, 1, 1, 2, 8], [1, 1, 1, 1, 3, 7], [1, 1, 1, 1, 4, 6], [1, 1, 1, 1, 5, 5], [1, 1, 1, 2, 2, 7], [1, 1, 1, 2, 3, 6], [1, 1, 1, 2, 4, 5], [1, 1, 1, 3, 3, 5], [1, 1, 1, 3, 4, 4], [1, 1, 2, 2, 2, 6], [1, 1, 2, 2, 3, 5], [1, 1, 2, 2, 4, 4], [1, 1, 2, 3, 3, 4], [1, 1, 3, 3, 3, 3], [1, 2, 2, 2, 2, 5], [1, 2, 2, 2, 3, 4], [1, 2, 2, 3, 3, 3], [2, 2, 2, 2, 2, 4], [2, 2, 2, 2, 3, 3]]
As "combinations with replacement" does, this function only produces the combination. I want permutation of each combination without repetition.
For example
[[0, 0, 0, 0, 0, 14], [0, 0, 0, 0, 14, 0] ... [3, 2, 3, 2, 2, 2], [3, 3, 2, 2, 2]]
When I tried to do this by
ret=[]
for i in range(90):
ret.extend(it.permutations(sum_n_combs[i], SIZE))
Time complexity was exponential, and made repititions
When I tested with one list sum_n_combs[0], which is [0, 0, 0, 0, 0, 14] produced 720 permutations when I only want 6 of them(14 at each different place).
How can I make permutation without repetition for each combination in an efficient way?

You could separate this in two steps:
generate partitions of the targeted sum
generate distinct permutations of each partition
Recursive generators will allow you to get the results efficiently without trial/error filtering and without storing everything in memory:
def partitions(N,size):
if size == 1 :
yield (N,) # base case, only 1 part
return
for a in range(N//size+1): # smaller part followed by
for p in partitions(N-a*size,size-1): # equal or larger ones
yield (a, *(n+a for n in p)) # recursing on delta only
def permuteDistinct(A):
if len(A) == 1:
yield tuple(A) # single value
return
used = set() # track starting value
for i,n in enumerate(A): # for each starting value
if n in used: continue # not yet used
used.add(n)
for p in permuteDistinct(A[:i]+A[i+1:]):
yield (n,*p) # starting value & rest
output:
N = 14
SIZE = 6
PARTITIONS...
for part in partitions(N,SIZE):
print(part)
(0, 0, 0, 0, 0, 14)
(0, 0, 0, 0, 1, 13)
(0, 0, 0, 0, 2, 12)
(0, 0, 0, 0, 3, 11)
(0, 0, 0, 0, 4, 10)
(0, 0, 0, 0, 5, 9)
(0, 0, 0, 0, 6, 8)
(0, 0, 0, 0, 7, 7)
(0, 0, 0, 1, 1, 12)
(0, 0, 0, 1, 2, 11)
(0, 0, 0, 1, 3, 10)
(0, 0, 0, 1, 4, 9)
(0, 0, 0, 1, 5, 8)
(0, 0, 0, 1, 6, 7)
(0, 0, 0, 2, 2, 10)
(0, 0, 0, 2, 3, 9)
(0, 0, 0, 2, 4, 8)
(0, 0, 0, 2, 5, 7)
(0, 0, 0, 2, 6, 6)
(0, 0, 0, 3, 3, 8)
(0, 0, 0, 3, 4, 7)
(0, 0, 0, 3, 5, 6)
(0, 0, 0, 4, 4, 6)
(0, 0, 0, 4, 5, 5)
...
PERMUTED PARTITIONS (DISTINCT):
for part in partitions(N,SIZE):
for permutedPart in permuteDistinct(part):
print(permutedPart)
(0, 0, 0, 0, 0, 14)
(0, 0, 0, 0, 14, 0)
(0, 0, 0, 14, 0, 0)
(0, 0, 14, 0, 0, 0)
(0, 14, 0, 0, 0, 0)
(14, 0, 0, 0, 0, 0)
(0, 0, 0, 0, 1, 13)
(0, 0, 0, 0, 13, 1)
(0, 0, 0, 1, 0, 13)
(0, 0, 0, 1, 13, 0)
(0, 0, 0, 13, 0, 1)
(0, 0, 0, 13, 1, 0)
(0, 0, 1, 0, 0, 13)
(0, 0, 1, 0, 13, 0)
(0, 0, 1, 13, 0, 0)
(0, 0, 13, 0, 0, 1)
(0, 0, 13, 0, 1, 0)
(0, 0, 13, 1, 0, 0)
...

Adding link annotations to a PDF document

How can I add annotations (in a particular shape) to a PDF?
I want to be able to control:
the link target
the color
the shape of the link annotation
the location of the link annotation

Disclaimer: I am the author of the library being used in this answer
To showcase this behaviour, this example is going to re-create a shape using "pixel-art".
This array, together with these colors define the shape of super-mario:
m = [
[0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0],
[0, 0, 0, 2, 2, 2, 3, 3, 2, 3, 0, 0, 0, 0],
[0, 0, 2, 3, 2, 3, 3, 3, 2, 3, 3, 3, 0, 0],
[0, 0, 2, 3, 2, 2, 3, 3, 3, 2, 3, 3, 3, 0],
[0, 0, 2, 2, 3, 3, 3, 3, 2, 2, 2, 2, 0, 0],
[0, 0, 0, 0, 3, 3, 3, 3, 3, 3, 3, 0, 0, 0],
[0, 0, 0, 1, 1, 4, 1, 1, 1, 1, 1, 0, 0, 0],
[0, 0, 1, 1, 1, 4, 1, 1, 4, 1, 1, 1, 0, 0],
[0, 1, 1, 1, 1, 4, 4, 4, 4, 1, 1, 1, 1, 0],
[0, 3, 3, 1, 4, 5, 4, 4, 5, 4, 1, 3, 3, 0],
[0, 3, 3, 3, 4, 4, 4, 4, 4, 4, 3, 3, 3, 0],
[0, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 3, 3, 0],
[0, 0, 0, 4, 4, 4, 0, 0, 4, 4, 4, 0, 0, 0],
[0, 0, 2, 2, 2, 0, 0, 0, 0, 2, 2, 2, 0, 0],
[0, 2, 2, 2, 2, 0, 0, 0, 0, 2, 2, 2, 2, 0],
]
c = [
None,
X11Color("Red"),
X11Color("Black"),
X11Color("Tan"),
X11Color("Blue"),
X11Color("White"),
]
To manipulate the PDF, I am going to use pText.
First we are going to read an existing PDF:
# attempt to read PDF
doc = None
with open("boring-input.pdf", "rb") as in_file_handle:
print("\treading (1) ..")
doc = PDF.loads(in_file_handle)
Then we are going to add the annotations, using the array indices as references (and keeping in mind the coordinate system for PDF starts at the bottom left):
# add annotation
pixel_size = 2
for i in range(0, len(m)):
for j in range(0, len(m[i])):
if m[i][j] == 0:
continue
x = pixel_size * j
y = pixel_size * (len(m) - i)
doc.get_page(0).append_link_annotation(
page=Decimal(0),
color=c[m[i][j]],
location_on_page="Fit",
rectangle=(
Decimal(x),
Decimal(y),
Decimal(x + pixel_size),
Decimal(y + pixel_size),
),
)
Then we store the output PDF:
# attempt to store PDF
with open("its-a-me.pdf, "wb") as out_file_handle:
PDF.dumps(out_file_handle, doc)
This is a screenshot of Okular opening the PDF:

Count specific values in a pandas series

I have a pandas series in python.
Is there a function/easy way to construct a series which contains the number of appearances of given values?
For demonstration, suppose I have the following Series: 1, 3, 1, 5, 10.
I want to count how many appearances each value has, from the following list: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].
The series that should return is 2, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1

We do value_counts + reindex
l=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
l1=[1, 3, 1, 5, 10]
pd.Series(l1).value_counts().reindex(l,fill_value=0).tolist()
[2, 0, 1, 0, 1, 0, 0, 0, 0, 1]

Use numpy.bincount
import numpy as np
l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
s = pd.Series([1, 3, 1, 5, 10])
out = list(np.bincount(s)[[l]])
out
[2, 0, 1, 0, 1, 0, 0, 0, 0, 1]

With map:
s = pd.Series([1, 3, 1, 5, 10])
inp_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
pd.Series(inp_list).map(s.value_counts()).fillna(0).astype(int).tolist()
Or list comp with get
c = s.value_counts()
[c.get(i,0) for i in inp_list]
#or [*map(lambda x: c.get(x,0),inp_list)]
[2, 0, 1, 0, 1, 0, 0, 0, 0, 1]

Using text on y-axis instead of numbers for line plot using matplotlib

I am trying to build a line plot with y-axis as text instead of numbers and I have tried this solution but the y-axis still have numbers instead of 'text'. What am I doing wrong.
CurrentCoresLE is a list of strings.
Y_AXIS_VALUES = list(numpy.sort(list(set(CurrentCoresLE))))
Y_AXIS = [Y_AXIS_VALUES.index(CurrentCoresLE[i]) for i in range(len(CurrentCoresLE))]
one = axes4.plot(Y_AXIS, color = '0.2', label='CoresUsed')
axes4.set_ylabel('Cores')
#https://stackoverflow.com/questions/17074772/using-text-on-y-axis-in-matplotlib-instead-of-numbers
#plt.yticks(Y_AXIS, Y_AXIS_VALUES)
axes4.set_yticks(Y_AXIS, Y_AXIS_VALUES)
axes4.legend(prop={'size':6}, ncol=4)
axes4.xaxis.grid(True)
And when I try plt instead of axes4, it does give text but in only one spot
here is an example:
Here is an exmaple Y_AXIS_VALUES and Y_AXIS
['0B2S', '0B3S', '0B4S', '1B3S', '2B0S', '2B2S']
[4, 4, 5, 5, 5, 3, 3, 3, 2, 1, 0, 1, 0, 1, 1, 0, 0, 1, 2, 1, 2, 2, 1, 0, 1, 2, 2, 1, 0, 1, 2, 1, 1, 2, 2, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 3, 3, 3, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 3, 3, 3, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 1, 2, 2, 1, 2, 2, 2, 2, 2, 1, 1]

This is how I fixed it.
axes4 = fig.add_subplot(514)
Y_AXIS_VALUES = list(numpy.sort(list(set(CurrentCoresLE))))
Y_AXIS = [Y_AXIS_VALUES.index(CurrentCoresLE[i]) for i in range(len(CurrentCoresLE))]
one = axes4.plot(Y_AXIS, color = '0.2', label='CoresUsed')
axes4.set_ylabel('Cores')
ytickNames = axes4.set_yticklabels(Y_AXIS_VALUES)
fontnum = 10
plt.setp(ytickNames, fontsize=fontnum)
axes4.legend(prop={'size':6}, ncol=4)
axes4.xaxis.grid(True)
I guess the trick was to set the yticktabels first and use setp

Categories

Resources