Develop Reference

Limited shape as output in tensorflow

I am trying to randomly generate timeseries data using keras as follows:
import tensorflow as tf
import pandas as pd
import random
input_data = [random.uniform(10,100) for _ in range(350000)]
targets = [random.uniform(10,100) for _ in range(350000)]
dataset = tf.keras.utils.timeseries_dataset_from_array(
input_data, targets, sequence_length=10000)
for batch in dataset:
inputs, targets = batch
break
But the final shape is reduced and coming as:
<tf.Tensor: shape=(128, 10000), dtype=float32, numpy=
array([[22.922523, 44.253967, 41.80049 , ..., 60.444836, 14.977458,
17.970036],
[44.253967, 41.80049 , 34.09485 , ..., 14.977458, 17.970036,
68.27751 ],
[41.80049 , 34.09485 , 37.27845 , ..., 17.970036, 68.27751 ,
98.05703 ],
...,
[13.941159, 51.48634 , 61.248505, ..., 98.093346, 67.3885 ,
34.01148 ],
[51.48634 , 61.248505, 77.34204 , ..., 67.3885 , 34.01148 ,
27.165142],
[61.248505, 77.34204 , 54.856853, ..., 34.01148 , 27.165142,
97.55085 ]], dtype=float32)>
How can i increase size array or is there any limitation?

change the VARIABLE with the number of sample that you want (batch_size).
If you want the whole data you can make batch_size=None
dataset = tf.keras.utils.timeseries_dataset_from_array(
input_data, targets, batch_size=VARIABLE, sequence_length=10000)

Tensorflow sort changes values in output list to 0 when the tensor datatype is tf.float32, but not tf.float64

Actions speak louder than words so here is the MWE:
import tensorflow as tf
n=17
a = tf.random.uniform(shape=[n], dtype=tf.float32)
print(a)
print(tf.sort(a))
When n<=16 it sorts the values just fine, but when n>16 it sorts the list and then turns the values at position 16 and higher to have value = -0. Example output:
tf.Tensor(
[0.41191268 0.48915362 0.65293264 0.6125376 0.00088847 0.03644979
0.13768506 0.528106 0.27231824 0.4003389 0.5799836 0.83420205
0.06494105 0.39109504 0.8135816 0.153288 0.07945895], shape=(17,), dtype=float32)
tf.Tensor(
[ 0.00088847 0.03644979 0.06494105 0.07945895 0.13768506 0.153288
0.27231824 0.39109504 0.4003389 0.41191268 0.48915362 0.528106
0.5799836 0.6125376 0.65293264 0.8135816 -0. ], shape=(17,), dtype=float32)
But when I make the dtype tf.float64 there is no problem for seemingly arbitrary n:
tf.Tensor(
[0.91347295 0.60086058 0.0271204 0.83564393 0.49664206 0.96215479
0.60472639 0.64395121 0.58394402 0.93489432 0.50379539 0.14087138
0.51662724 0.29758834 0.5657154 0.08638131 0.47912787], shape=(17,), dtype=float64)
tf.Tensor(
[0.0271204 0.08638131 0.14087138 0.29758834 0.47912787 0.49664206
0.50379539 0.51662724 0.5657154 0.58394402 0.60086058 0.60472639
0.64395121 0.83564393 0.91347295 0.93489432 0.96215479], shape=(17,), dtype=float64)
Not sure if this is a bug or expected behavior. It does not depend on eager execution. I was noticing issues when using the tensorflowprobability percentile function where it was giving me -0.0 as the value, so I made my own percentile function and observed the same issue (which I suspect is due to this underlying issue with tf.sort). Numpy sorting works fine regardless of the data type, but I was trying to keep things within tensorflow.
Any reason why this might be happening or should I make a bug report?
Hardware: I am on an M1 Macbook Air using tensorflow 2.5.0

I also checked the same with tensorflow==2.5.0 as you mentioned and did not notice any issue.
import tensorflow as tf
print(tf.__version__)
Output:
2.5.0
and then
import tensorflow as tf
n=17
a = tf.random.uniform(shape=[n], dtype=tf.float32)
print(a)
print(tf.sort(a))
Output:
tf.Tensor(
[0.7946081 0.84397626 0.04671419 0.276353 0.8124876 0.66761124
0.21016991 0.28140187 0.22393394 0.20382321 0.667526 0.1714747
0.9672215 0.17870915 0.9914366 0.32059753 0.5422765 ], shape=(17,), dtype=float32)
tf.Tensor(
[0.04671419 0.1714747 0.17870915 0.20382321 0.21016991 0.22393394
0.276353 0.28140187 0.32059753 0.5422765 0.667526 0.66761124
0.7946081 0.8124876 0.84397626 0.9672215 0.9914366 ], shape=(17,), dtype=float32)
with n>16:
import tensorflow as tf
n=18
a = tf.random.uniform(shape=[n], dtype=tf.float32)
print(a)
print(tf.sort(a))
Output:
tf.Tensor(
[0.1922586 0.6136733 0.7517139 0.3762852 0.52895963 0.7804493
0.9869323 0.08194113 0.3963052 0.6049119 0.9553219 0.18031311
0.58210933 0.92059183 0.4442644 0.91004515 0.4451145 0.8300687 ], shape=(18,), dtype=float32)
tf.Tensor(
[0.08194113 0.18031311 0.1922586 0.3762852 0.3963052 0.4442644
0.4451145 0.52895963 0.58210933 0.6049119 0.6136733 0.7517139
0.7804493 0.8300687 0.91004515 0.92059183 0.9553219 0.9869323 ], shape=(18,), dtype=float32)
Would you mind checking it back again and let us know if the issue still persists?

Why there are so many array of weights?

I am working on python with keras. I learned in my theory study that in a neural network the weights are only between the input layer and a hidden layer or between hidden layers.
I wrote this code, where I added two layers:
NN.add(Dense(4, input_shape=array_input.shape, activation='relu', name="Layer", kernel_constraint=changeWeight()))
NN.add(Dense(4, activation='relu', name="Output"))
NN.compile(loss='mean_squared_error', optimizer=Adam(learning_rate=0.3), metrics=['accuracy'])
print(NN.summary())
a = NN.fit(array_input, array_input, epochs=100)
for lay in NN.layers:
print(lay.name)
print(lay.get_weights())
I think that one is the hidden layer (the one renamed "Layer") and the other is the output layer. The problem is that if i printed "lay.get_weights()" there are two arrays of weights, one for each layer. Like this:
[array([[-1.5516974 , -1.600516 , -0. , 0. ],
[-0. , -2.1766946 , 0.32734624, -0. ],
[-0. , -0. , 0.32156652, -0.812184 ],
[-0. , -0. , -0. , -0.7288372 ]],
dtype=float32), array([-1.8015273, -1.801546 , -0.1462403, 0. ], dtype=float32)]
Output
[array([[-1.5045888 , -0.14155084, -0.29977936, -0.0492779 ],
[-1.2379107 , -0.44411597, -0.41499865, -0.2560569 ],
[ 1.2397875 , -0.3541401 , 1.2223543 , 1.5617256 ],
[ 0.18388063, 0.44298917, -0.2201969 , -0.1165269 ]],
dtype=float32), array([-0.82720596, 0. , 1.1942271 , 1.7084894 ], dtype=float32)]
Can someone explain to me where is the problem. I don't understand keras API, do I?

https://www.tensorflow.org/api_docs/python/tf/keras/layers/Layer#get_weights
get_weights() returns the weight and value of the bias in an array.
Each of your inputs is connected to the first layers. So the weight matrix has a shape of (input.shape, number of neurons in the current layer) and the bias vector has a shape of (number of neurons in the current layer, ).
Therefore, without knowing what your input array contains, I know that this array has a shape of (4,).
For the second layer, the same process is repeated
weight : (number of neurons of the last layer, number of neurons of the current layer)
bias (number of neurone of the current layer,)
Try this example:
NN = Sequential()
NN.add(Dense(2, input_shape=(3,), activation='relu', name="Layer"))
NN.add(Dense(4, activation='relu', name="Output"))
for lay in NN.layers:
print(lay.name)
print(lay.get_weights())
Output:
Layer
[array([[-0.674668 , -0.34347552],
[ 0.63090587, 0.8558588 ],
[-0.5063792 , -0.23311883]], dtype=float32), array([0., 0.], dtype=float32)]
Output
[array([[-0.07787323, 0.22444701, 0.52729607, 0.07616615],
[-0.5380094 , -0.3146367 , -0.73177123, -0.9248886 ]],
dtype=float32), array([0., 0., 0., 0.], dtype=float32)]
Graphical representation :

Keras:How to get weights, make weights become 1D array, then make weights shape become initial shape?

I want to do some experiment. and i need to get Keras model weights, make it 1D array , and make the shape like initial shape
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
model = tf.keras.Sequential()
# Adds a densely-connected layer with 64 units to the model:
model.add(layers.Dense( 4, input_dim = 5 ,activation='relu'))
# Add another:
model.add(layers.Dense(3, activation='relu'))
# Add an output layer with 10 output units:
model.add(layers.Dense(2))
# Configure a model for mean-squared error regression.
model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
loss='mse', # mean squared error
metrics=['mae']) # mean absolute error
weights = (model.get_weights())
#make weight become 1D array
#maka 1D array become like inital shape
model.set_weights(weights)
why iwant to do this ?
because i want to do some mutation using other module, that's necessary to pass 1D array
how to do this ?
as we know the shape of Keras model weights look like this
[array([[-0.24053234, 0.4722855 , 0.29863954, 0.22805429],
[ 0.45101106, -0.00229341, -0.6142864 , -0.2751704 ],
[ 0.159172 , 0.43983865, 0.61577237, 0.24255097],
[ 0.24160242, 0.422235 , 0.8066592 , -0.2711717 ],
[-0.30763668, -0.4841219 , 0.767977 , 0.23558974]],
dtype=float32), array([0., 0., 0., 0.], dtype=float32), array([[ 0.24129152, -0.4890638 , 0.18787515],
[ 0.8663894 , -0.09163451, -0.86416066],
[-0.01754427, 0.32654428, -0.78837514],
[ 0.589849 , 0.5886531 , 0.27824092]], dtype=float32), array([0., 0., 0.], dtype=float32), array([[ 0.8456359 , -0.26292562],
[-1.0447757 , -0.43539298],
[ 1.0835328 , -0.43536085]], dtype=float32), array([0., 0.], dtype=float32)]

Can Convolution2D work on rectangular images?

Let's say I have a 360px by 240px image. Instead of cropping my (already small) image to 240x240, can I create a convolutional neural network that operates on the full rectangle? Specifically using the Convolution2D layer.
I ask because every paper I've read doing CNNs seems to have square input sizes, so I wonder if what I propose will be OK, and if so, what disadvantages I may run into. Are all the settings (like border_mode='same') going to work the same?

No issues with a rectangle image... Everything will work properly as for square images.

Yes.
But why don't you give it a try
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function
import numpy as np
np.random.seed(1234)
from keras.layers import Input
from keras.layers.convolutional import Convolution2D
from keras.models import Model
print("Building Model...")
inp = Input(shape=(1,None,None))
output = Convolution2D(1, 3, 3, border_mode='same', init='normal',bias=False)(inp)
model_network = Model(input=inp, output=output)
w = np.asarray([
[[[
[0,0,0],
[0,2,0],
[0,0,0]
]]]
])
input_mat = np.asarray([
[[
[1.,2.,3.,10.],
[4.,5.,6.,11.],
[7.,8.,9.,12.]
]]
])
model_network.layers[1].set_weights(w)
print("Weights after change:")
print(model_network.layers[1].get_weights())
print("Input:")
print(input_mat)
print("Output:")
print(model_network.predict(input_mat))
Build a sample model
inp = Input(shape=(1,None,None))
output = Convolution2D(1, 3, 3, border_mode='same', init='normal',bias=False)(inp)
model_network = Model(input=inp, output=output)
Give it some weights and set them so you could predit the output, say:
w = np.asarray([
[[[
[0,0,0],
[0,2,0],
[0,0,0]
]]]
])
model_network.layers[1].set_weights(w)
So that the convolution would simply double your input.
Give it your rectangular image:
input_mat = np.asarray([
[[
[1.,2.,3.,10.],
[4.,5.,6.,11.],
[7.,8.,9.,12.]
]]
])
And check the output to see if it works
print("Output:")
print(model_network.predict(input_mat))
Sample output:
Using Theano backend.
Building Model...
Weights after change:
[array([[[[ 0., 0., 0.],
[ 0., 2., 0.],
[ 0., 0., 0.]]]], dtype=float32)]
Input:
[[[[ 1. 2. 3. 10.]
[ 4. 5. 6. 11.]
[ 7. 8. 9. 12.]]]]
Output:
[[[[ 2. 4. 6. 20.]
[ 8. 10. 12. 22.]
[ 14. 16. 18. 24.]]]]
original post with some changes