Time missmatch when using NetCDF4 - python

I have a netcdf file with a time variable time. When I print this variable I get this:
<class 'netCDF4._netCDF4.Variable'>
float32 time(time)
long_name: time
units: hours since 1900-01-01 00:00:00.0
calendar: gregorian
unlimited dimensions: time
current shape = (31,)
filling on, default _FillValue of 9.969209968386869e+36 used
I would like to see the dates stored in my time variable. If I type and run print(time[:]) I will not really be able to see the dates of my time variable. Instead I will see this:
[1004406. 1004430. 1004454. 1004478. 1004502. 1004526. 1004550. 1004574.
1004598. 1004622. 1004646. 1004670. 1004694. 1004718. 1004742. 1004766.
1004790. 1004814. 1004838. 1004862. 1004886. 1004910. 1004934. 1004958.
1004982. 1005006. 1005030. 1005054. 1005078. 1005102. 1005126.]
Now, This is what I have done to see the time in a date format:
dtime = nc.num2date(time[:],time.units)
However, when I print dtime this is my result:
[cftime.DatetimeGregorian(2014, 8, 1, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 2, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 3, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 4, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 5, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 6, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 7, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 8, 6, 0, 0, 0, has_year_zero=False)
cftime.DatetimeGregorian(2014, 8, 9, 6, 0, 0, 0, has_year_zero=False)...
Why is it showing 2014 instead of 1990 and how can I fix this using ndtcdf4?
I do not know what is happening on the background.

Well, the time values are hours since 1900-01-01 00:00:00.0 meaning that for instance if you take the first value 1004406, it should correspond to:
import datetime
datetime.datetime(1900,1,1,)+datetime.timedelta(seconds=1004406*3600)
i.e. datetime.datetime(2014,8,1,6,0) exactly as you get it. Why should it be 1990 instead of 2014?
I suggest using num2date with following options:
only_use_cftime_datetimes=False and only_use_python_datetimes=True.
Latter would give dtime as datetime objects, which can be easily used with matplotlib. Somehow the cftime are not so easily supported...
So, in your script:
dtime = nc.num2date(time[:],time.units,only_use_cftime_datetimes=False,only_use_python_datetimes=True)

Related

Slicing lists for getting just quarter of them in the same length

I have a list with lengths of A. Is there a way using slicing to take just every fourth value but maintaining it's length? For example, If this is my list:
A = [0,1,2,3,4,5,6,7,8....]
I want to use slicing to receive:
A = [0,0,0,0,4,4,...]
Using the [::4] is doing the job for the taking every fourth value, but without the other values I want.
If you have:
A = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
Then you can do:
>>> [v for v in A[::4] for _ in range(4)]
[0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12, 12, 12, 12]
>>>
I should mention that this rounds the result list length up to a multiple of 4, so you always get 4 of each value included in the result. If you want to drop the extra values at the end, you can append [:len(A)] to the end.
you could do :
A = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
repeatedA = [ A[(i//4) * 4 ] for i in range(len(A))]
the i//4 here allows to get the quotient of the division of the index by 4, then (i//4) * 4 allows us to find the closest multiple of 4 to i smaller than i, so here 1,2,3 are not dividors of 4 but 0 is , which is the closest multiple to them, it's the same with 5,6, and 7, 4 is the closest multiple ( given that it should always be smaller than either of them ), it's equivalent to i - i%4, which allows us to round i down to the closest dividor of 4 given that i = 4* (i//4 ) + 4%i
This code prints elements of A if their index residual from division by 4 equals 0.
Instead of i%4==0 you may enter i%4==1 or 2 or 3 if you need different offset.
A = [0,1,2,3,4,5,6,7,8,9,10,11,12]
print ([A[i] if i%4==0 else 0 for i in range (len(A))])
[0, 0, 0, 0, 4, 0, 0, 0, 8, 0, 0, 0, 12]
If you want to repeat each fourth value 4 times, try this:
print ( [A[i//4*4] for i in range (len(A))])
[0, 0, 0, 0, 4, 4, 4, 4, 8, 8, 8, 8, 12]

verify continuous segments in list [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I want to verify if a condition of sequential segments is appearing in a list or not
An example:
I can have two options, either a list would be like this
[7, 7, 7, 7, 0, 0, 0, 10, 10, 10, 10, 10, 10, 10, 10, 0, 0, 0, 0, 0, 7, 7, 7 ....]
or
[......, 10, 10, 10, 10, 0, 0, 0, 10, 10, 10, 10, ......]
What I want here
is that I want to go through the list and verify if I find 0's between 10's or 10's between 0's .. I return true
Note:
1- Whatever else inside the list is not important for me
2- The number of 0's or 10's is not fixed, they can change .. however, the most important manner, that if there are 0's there have to be 10's before or after .. or the other way around 0's before and after 10's
What about creating a small function that turns the lists into strings and runs a regex against it, You could tidy this up, or I could if you approve this concept
I'm not a regex, expert I'm sure there is another way to do this without using the AttribeError but this will work, someone can improve this, but it works.
import re
def finder(s):
s = ''.join(str(s))
match = re.search(r'0\,\s10,(\s10,)*\s0', s)
try:
if (match.group(0)):
return True
except AttributeError:
pass
match = re.search(r'10\,\s0,(\s0,)*\s10', s)
try:
if (match.group(0)):
return True
except AttributeError:
return False
l1 = [7, 7, 7, 7, 0, 0, 0, 10, 10, 10, 10, 10, 10, 10, 10, 0, 0, 0, 0, 0, 7, 7, 7]
l2 = [10, 10, 10, 10, 0, 0, 0, 10, 10, 10, 10]
l3 = [5, 5, 3, 1, 0, 1, 1, 1, 8, 10, 0, 8, 9, 9, 5]
print(finder(l1))
print(finder(l2))
print(finder(l3))
(xenial)vash#localhost:~/python/stack_overflow/sept$ python3.7 check.py
True
True
False
from my understanding you can do something like below.
First find the indexes of 0 ,10s then iterate over them to check if main list is having 0 or 10 in between them. Later check for True/False if some found elements it will return True else False will be returned.
from more_itertools import locate
l = [7, 7, 7, 7, 0, 0, 0, 10, 10, 10, 10, 10, 10, 10, 10, 0, 0, 0, 0, 0, 7, 7, 7]
# get the indexes of 0's and 10's in given list/array
zeros = list(locate(l, lambda x: x == 0))
tens = list(locate(l, lambda x: x == 10))
# check for each combination and return true/false in each time
find_10s = [(10 in (l[zeros[i]:zeros[i]+1])) for i in range(len(zeros))]
find_0s = [(0 in (l[tens[i]:tens[i]+1])) for i in range(len(tens))]
# this line is to check if anything found in above results
print(any(find_10s) or any(find_0s)) #False

Puzzle move maximum number from start rows to end rows in a grid of rooms and corridors

I have this puzzle that provides a grid of rooms connected together with corridors, in the entrances rooms there are groups of people that you need to move them through the corridors to the exit rooms, the puzzle has the following rules:
The grid element path[A][B] = C describes that the corridor going
from A to B can fit C number of people at each time step.
There are at most 50 rooms connected by the corridors and at most 2000000
people that will fit at a time.
entrances and exits never
overlap.
So I need to find out how many people can fit through at a time in each direction of every corridor in between. For example to solve the following grid:
entrances = [0, 1]
exits = [4, 5]
grid = [
[0, 0, 4, 6, 0, 0],
[0, 0, 5, 2, 0, 0],
[0, 0, 0, 0, 4, 4],
[0, 0, 0, 0, 6, 6],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
]
In each time step, the following might happen:
0 sends 4/4 people to 2 and 6/6 people to 3
1 sends 4/5 people to 2 and 2/2 people to 3
2 sends 4/4 people to 4 and 4/4 people to 5
3 sends 4/6 people to 4 and 4/6 people to 5
So, in total, 16 people could make it to the exit rows at 4 and 5 at each time step. (Note that in this example, room 3 could have sent any variation of 8 people to 4 and 5, such as 2/6 and 6/6, but the final answer remains the same.)
I tried to solve it by starting from top and moving people down the grid through the next available corridor in the next room until I reach the exits then I count the number of people that reached the exits. This works for simple cases like the example above but it doesn't take into account the best corridor to choose from multiple available corridors in a one room that would allow the maximum number of people to pass through to exits, also it doesn't consider that you can send any combination of people from any room. This is the code I have so far:
grid2 = [
# 0 1 2 3 4 5 6 7 8 9 10 11
[0, 0, 4, 6, 0, 9, 8, 0, 5, 0, 0, 0], # 0
[0, 0, 5, 2, 7, 0, 9, 9, 0, 6, 0, 0], # 1
[0, 0, 0, 3, 4, 9, 0, 2, 8, 0, 8, 0], # 2
[0, 0, 0, 0, 6, 6, 1, 8, 0, 7, 0, 9], # 3
[0, 0, 0, 0, 0, 3, 9, 0, 4, 0, 0, 0], # 4
[0, 0, 0, 0, 0, 0, 9, 6, 0, 4, 9, 0], # 5
[0, 0, 0, 0, 0, 0, 0, 7, 2, 3, 6, 1], # 6
[0, 0, 0, 0, 0, 0, 0, 0, 9, 9, 9, 9], # 7
[0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 6, 2], # 8
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 6], # 9
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], # 10
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], # 11
]
def solve (entrances, exits, path):
rows = len (path)
cols = len (path[0])
station = [x for x in path]
for r in range (rows):
for c in range (cols):
if path[r][c]:
val = station[r][c]
station[r][c] = [val, val] if r in entrances else [val, 0]
total = 0
for r in [x for x in range (rows) if x not in exits]:
for c in range (cols):
if station[r][c] and station[r][c][1]:
count = station[r][c][1]
if c not in exits:
for i in range (cols):
if station[c][i] and not station[c][i][1]:
num = min (count, station[c][i][0])
station[c][i][1] = num
break
else: total += count
return total
What's the best approach to solve this puzzle and is there a good algorithm to implement here?
This question seems a little bit ambiguous, but here's my best attempt:
Max-Flow. Basically what you're trying to do here, judging from the example, is you're trying to find a max-flow for the residual network given to you in that matrix. This is not said in the specification, but it looks like:
At any given time people can only be in the corridors - edges (nodes have capacity 0).
People can only enter the network in one 'wave'.
Thanks to those you can think of this as if it was a classic flow problem - you have a network of pipes of certain capacities and you're trying to see what the maximum flow of a liquid can be through those at any time.
Apart from the max-flow problem for which you can use a number of popular algorithms, there's one trick to use here.
Since you have multiple entry and exit nodes in your network, you will need to add 2 artificial nodes, let's call them -1 and +inf. They will be 'superentrance' and 'superexit' respectively. -1 will be connected to all the entrances with 'virtual' edges of infinite capacity (you can hardcode it) and similarily all exit nodes will connect to +inf with 'virtual' edges of capacity infinity. This way you can treat -1 and +inf as the only entrance and exit of your network - their capacity will be the same as the min-cut (the bottleneck; the maximum flow at a given time is equal to the min-cut as follows from Max-flow min-cut theorem), so adding them will not change the total flow. It will however give you an easy way to have single inputs and outputs of your network as opposed to multiple ones.
As for the choice of the algorithm, it depends on the size of your graph and the structure of it. Here's a list of popular algorithms - you can pick the one that is good for your case. If your data allows you to do this, you might want to stick with the popular and easier ones like Ford-Fulkerson or Edmonds-Karp, especially that you will find many implementations online.

How does the numpy.resize and numpy.reshape function works internally in python ?

In the package numpy their are two function resize and reshape. How internally they work? What kind of interpolation does they use ?I looked into the code, but didnt get it. Can anyone help me out. Or how does an image get resized. What happens with its pixels ?
Neither interpolates. And if you are wondering about interpolation and pixels of an image, they probably aren't the functions that you want. There some image packages (e.g in scipy) that manipulate the resolution of images.
Every numpy array has a shape attribute. reshape just changes that, without changing the data at all. The new shape has to reference the same total number of elements as the original shape.
x = np.arange(12)
x.reshape(3,4) # 12 elements
x.reshape(6,2) # still 12
x.reshape(6,4) # error
np.resize is less commonly used, but is written in Python and available for study. You have to read its docs, and x.resize is different. Going larger it actually repeats values or pads with zeros.
examples of resize acting in 1d:
In [366]: x=np.arange(12)
In [367]: np.resize(x,6)
Out[367]: array([0, 1, 2, 3, 4, 5])
In [368]: np.resize(x,24)
Out[368]:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 0, 1, 2, 3, 4,
5, 6, 7, 8, 9, 10, 11])
In [369]: x.resize(24)
In [370]: x
Out[370]:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0])
A recent question about scipy.misc.imresize. It also references scipy.ndimage.zoom:
Broadcasting error when vectorizing misc.imresize()
As far as I know numpy.reshape() just reshapes a matrix (does not matter if it is an image or not). It does not do any interpolation and just manipulates the items in a matrix.
a = np.arange(12).reshape((2,6))
a= [[ 0 1 2 3 4 5]
[ 6 7 8 9 10 11]]
b=a.reshape((4,3))
b=[[ 0 1 2]
[ 3 4 5]
[ 6 7 8]
[ 9 10 11]]

Fill zero values of 1d numpy array with last non-zero values

Let's say we have a 1d numpy array filled with some int values. And let's say that some of them are 0.
Is there any way, using numpy array's power, to fill all the 0 values with the last non-zero values found?
for example:
arr = np.array([1, 0, 0, 2, 0, 4, 6, 8, 0, 0, 0, 0, 2])
fill_zeros_with_last(arr)
print arr
[1 1 1 2 2 4 6 8 8 8 8 8 2]
A way to do it would be with this function:
def fill_zeros_with_last(arr):
last_val = None # I don't really care about the initial value
for i in range(arr.size):
if arr[i]:
last_val = arr[i]
elif last_val is not None:
arr[i] = last_val
However, this is using a raw python for loop instead of taking advantage of the numpy and scipy power.
If we knew that a reasonably small number of consecutive zeros are possible, we could use something based on numpy.roll. The problem is that the number of consecutive zeros is potentially large...
Any ideas? or should we go straight to Cython?
Disclaimer:
I would say long ago I found a question in stackoverflow asking something like this or very similar. I wasn't able to find it. :-(
Maybe I missed the right search terms, sorry for the duplicate then. Maybe it was just my imagination...
Here's a solution using np.maximum.accumulate:
def fill_zeros_with_last(arr):
prev = np.arange(len(arr))
prev[arr == 0] = 0
prev = np.maximum.accumulate(prev)
return arr[prev]
We construct an array prev which has the same length as arr, and such that prev[i] is the index of the last non-zero entry before the i-th entry of arr. For example, if:
>>> arr = np.array([1, 0, 0, 2, 0, 4, 6, 8, 0, 0, 0, 0, 2])
Then prev looks like:
array([ 0, 0, 0, 3, 3, 5, 6, 7, 7, 7, 7, 7, 12])
Then we just index into arr with prev and we obtain our result. A test:
>>> arr = np.array([1, 0, 0, 2, 0, 4, 6, 8, 0, 0, 0, 0, 2])
>>> fill_zeros_with_last(arr)
array([1, 1, 1, 2, 2, 4, 6, 8, 8, 8, 8, 8, 2])
Note: Be careful to understand what this does when the first entry of your array is zero:
>>> fill_zeros_with_last(np.array([0,0,1,0,0]))
array([0, 0, 1, 1, 1])
Inspired by jme's answer here and by Bas Swinckels' (in the linked question) I came up with a different combination of numpy functions:
def fill_zeros_with_last(arr, initial=0):
ind = np.nonzero(arr)[0]
cnt = np.cumsum(np.array(arr, dtype=bool))
return np.where(cnt, arr[ind[cnt-1]], initial)
I think it's succinct and also works, so I'm posting it here for the record. Still, jme's is also succinct and easy to follow and seems to be faster, so I'm accepting it :-)
If the 0s only come in strings of 1, this use of nonzero might work:
In [266]: arr=np.array([1,0,2,3,0,4,0,5])
In [267]: I=np.nonzero(arr==0)[0]
In [268]: arr[I] = arr[I-1]
In [269]: arr
Out[269]: array([1, 1, 2, 3, 3, 4, 4, 5])
I can handle your arr by applying this repeatedly until I is empty.
In [286]: arr = np.array([1, 0, 0, 2, 0, 4, 6, 8, 0, 0, 0, 0, 2])
In [287]: while True:
.....: I=np.nonzero(arr==0)[0]
.....: if len(I)==0: break
.....: arr[I] = arr[I-1]
.....:
In [288]: arr
Out[288]: array([1, 1, 1, 2, 2, 4, 6, 8, 8, 8, 8, 8, 2])
If the strings of 0s are long it might be better to look for those strings and handle them as a block. But if most strings are short, this repeated application may be the fastest route.

Categories

Resources