I'm having a terrible time coming up with an algorithm to flip a "stack" of pancakes,which are really a list of numbers. You're allowed to flip the top at a certain index and above. Even something in pseudo code would be a rolling start, if you don't want to do full Python. i just have trouble with recursion. Not sure if base case should be pancakes.is_in_order() == True or something more like len(pancakes) == 1: return pancakes.
Input example: pancakes = [2,3,1,4] (Top here would be 2. You can flip pancakes[:index] by doing a reverse() on that slice of list)
Need to flip such that you have [1,2,3,4]
Output: # of flips it takes (don't need to show flips, order, etc. Just number of flips. If it's over a dozen, then timing out is fine)
Related
I need to calculate a huge table value (157954 rows and 365 columns) by checking three conditions in a dataframe with 11 mil rows. Do you have any way to speed up the calculation, which is taking more than 10 hours now?
I have 367 stations in total.
for station in stations:
no_pickup_array = []
for time_point in data_matrix['Timestamp']:
time_point_2 = time_point + timedelta(minutes=15)
no_pickup = len(dataframe[(time_point <= dataframe["departure"]) & (dataframe["departure"] < time_point_2)
& (dataframe['departure_name'] == station)])
no_pickup_array.append(no_pickup)
print(f"Station name: {station}")
data_matrix[station] = no_pickup_array
I appreciate any of your help.
# To all: Thank you for your comments, I add more info for my problem.
Each row of dataframe is info of each renting bike. I want to create a matrix with number of bikes picked up at each station for each 15 minutes interval. Then I also want to calculate the average speed, average time,.. as well.
The solution from #Jérôme Richard could reduce the number of calculations, but I still struggle to understand and implement indexing steps and apply logarithmic search or binary search.
index = {name: df for name, df.sort_values('departure')['departure'].to_numpy() in dataframe.groupby('departure_name')}
# code #Jérôme Richard recommended
The main problem is the right-hand-side of the no_pickup assignment expression which is algorithmically inefficient because it makes a linear search while a logarithmic search is possible.
The first thing to do is to do a groupby of dataframe so to build an index enabling to fetch the dataframe subset having a given name. Then, you can sort each dataframe subset by departure so to be able to perform a binary search enabling you to know the number of item fitting the condition.
The index can be built with something like:
index = {name: df for name, df.sort_values('departure')['departure'].to_numpy() in dataframe.groupby('departure_name')}
Finally, you can do the binary search with two np.searchsorted on index[station]: one to know the starting index and one to know the ending index. You can get the length with a simple subtraction of the two.
Note that you may need some tweak since I am not sure the above code will works on your dataset but it is hard to know without an example of code generating the inputs.
You're indexing the dataframe list with a boolean (which will be zero or one, so you're only ever going to get the length of the first or second element) instead of a number. It's going to get evaluated like so:
len(dataframe[(time_point <= dataframe["departure"]) & (dataframe["departure"] < time_point_2) & (dataframe['departure_name'] == station)])
len(dataframe[True & False & True]) # let's just say the variables work out like this
len(dataframe[False])
len(dataframe[0])
This probably isn't the behavior you're after. (let me know what you're trying to do in a comment and I'll try to help out more.)
In terms of code speed specifically, & is bitwise "AND", in python the boolean operators are written out as and, or, and not. Using and here would speed up your code, since python only evaluates parts of boolean expressions where they're needed, e.g.
from time import sleep
def slow_function():
sleep(3)
return False
# This line doesn't take 3 seconds to run as you may expect.
# Python sees "False and" and is smart enough to realize that whatever comes after is irrelevant.
# No matter what comes after "False and", it's never going to make the first half True.
# So, python doesn't bother evaluating it, and saves 3 seconds in the process.
False and slow_function()
# Some more examples that show python doesn't evaluate the right half unless it needs to
False and print("hi")
False and asdfasdfasdf
False and 42/0
# The same does not happen here. These are bitwise operators, expected to be applied to numbers.
# While it does produce the correct result for boolean inputs, it's going to be slower,
# since it can't rely on the same optimization.
False & slow_function()
# Of course, both of these still take 3 seconds, since the right half has to be evaluated either way.
True and slow_function()
True & slow_function()
Code:
NUM_SQUARES = 9
EMPTY = " "
def new_board():
board = []
for square in range(NUM_SQUARES):
board.append(EMPTY)
return board
Background:
I was reading this piece of code for the game 'Tic-Tac-Toe'. I don't understand the for loop in the function new_board().
My Understanding:
So in the first part of the code, I understand that there are two constants for: the number of squares on the board, an empty square on the board. In the function, an empty list is created where empty strings would be added to represent blank squares.
I understand that in range(NUM_SQUARES): means that it will iterate the code below it 9 times. Therefore, it will add nine empty strings as items in the list.
What I don't get:
1) What is the square variable assigned to?
2) What is the purpose of needing this variable square?
3) Why do we commonly say i in for i in range()? (in general)
1) As you mentioned, in range(NUM_SQUARES) means that the commands within the loop will be executed NUM_SQUARES times. The first time it is executed, square will be equal to 0. Then 1, 2 etc. It's value is not used so you could have written for i in range(NUM_SQUARES). Often when you don't need the loop index you would write for _ in range(NUM_SQUARES).
2) The variable is not explicitly needed, but you just need to specify some variable. That's just how for loops work in python.
3) i, j, k is often used for integers in programming, math, physics etc.. I believe that is just the reason why we choose 'i' instead of something else.
This is an interesting mechanism for creating a list to be used for housing the board state of a tic tac toe game. I'll start off by answering your third question, and build from there.
3) i is short for 'index' when used in the classical for-loop structure. In python, index-based loops are typically used when you need to access the actual index itself (e.g. you have something where you need to perform an action on array_member[i]). You could write the loop as for index in range() if that is more explicit/makes better sense; it doesn't matter to the language what name you choose for the index variable.
2) Building on what we covered in #3, the purpose of the variable square in this context, is to act as the index for the list board over which we are iterating. The naming convention of square is more commonly seen when using a for-each loop, where presumably you would already have a list of squares that you were iterating over:
for square in list_of_squares:
if square == 'X':
# Do something, etc.
1) Based on all of this, the square variable starts off with 0 assigned to it. After the first iteration of the loop, it becomes 1, then 2, etc... until it reaches the limit specified by the range (in this case NUM_SQARES, or 9). It's worth noting that the range function is non-inclusive of the boundary specified, so this loop will execute from 0 to 8, which yields a total of 9 'squares' within the array.
so as homework for a programming class on python we're supposed to multiply to integers (n,m) with each other WITHOUT using the * sign (or another multiplication form). We're supposed to use recursion to solve this problem, so i tried just adding n with itself, m number of times. I think my problem is with using recursion itself. I have searched on the internet for recursion usage, no results. Here is my code. Could someone point me in the right direction?
def mult(n,m):
""" mult outputs the product of two integers n and m
input: any numbers
"""
if m > 0:
return n + n
return m - 1
else:
return 1
I don't want to give you the answer to your homework here so instead hopefully I can provide an example of recursion that may help you along :-).
# Here we define a normal function in python
def count_down(val):
# Next we do some logic, in this case print the value
print(val)
# Now we check for some kind of "exit" condition. In our
# case we want the value to be greater than 1. If our value
# is less than one we do nothing, otherwise we call ourself
# with a new, different value.
if val > 1:
count_down(val-1)
count_down(5)
How can you apply this to what you're currently working on? Maybe, instead of printing something you could have it return something instead...
Thanks guys, i figured it out!!!
i had to return 0 instead of 1, otherwise the answer would always be one higher than what we wanted.
and i understand how you have to call upon the function, which is the main thing i missed.
Here's what i did:
def mult(n,m):
""" mult outputs the product of two integers n and m
input: any numbers
"""
if m == 0:
return 0
else:
return n + mult(n, m - 1)
You have the right mechanics, but you haven't internalized the basics you found in your searches. A recursive function usually breaks down to two cases:
Base Case --
How do you know when you're done? What do you want to do at that point?
Here, you've figured out that your base case is when the multiplier is 0. What do you want to return at this point? Remember, you're doing this as an additive process: I believe you want the additive identity element 0, not the multiplicative 1.
Recursion Case --
Do something trivial to simplify the problem, then recur with this simplified version.
Here, you've figured out that you want to enhance the running sum and reduce the multiplier by 1. However, you haven't called your function again. You haven't properly enhanced any sort of accumulative sum; you've doubled the multiplicand. Also, you're getting confused about recursion: return is to go back to whatever called this function. For recursion, you'll want something like
mult(n, m-1)
Now remember that this is a function: it returns a value. Now, what do you need to do with this value? For instance, if you're trying to compute 4*3, the statement above will give you the value of 4*2, What do you do with that, so that you can return the correct value of 4*3 to whatever called this instance? You'll want something like
result = mult(n, m-1)
return [...] result
... where you have to fill in that [...] spot. If you want, you can combine these into a single line of code; I'm just trying to make it easier for you.
Given that we have two lines on a graph (I just noticed that I inverted the numbers on the Y axis, this was a mistake, it should go from 11-1)
And we only care about whole number X axis intersections
We need to order these points from highest Y value to lowest Y value regardless of their position on the X axis (Note I did these pictures by hand so they may not line up perfectly).
I have a couple of questions:
1) I have to assume this is a known problem, but does it have a particular name?
2) Is there a known optimal solution when dealing with tens of billions (or hundreds of millions) of lines? Our current process of manually calculating each point and then comparing it to a giant list requires hours of processing. Even though we may have a hundred million lines we typically only want the top 100 or 50,000 results some of them are so far "below" other lines that calculating their points is unnecessary.
Your data structure is a set of tuples
lines = {(y0, Δy0), (y1, Δy1), ...}
You need only the ntop points, hence build a set containing only
the top ntop yi values, with a single pass over the data
top_points = choose(lines, ntop)
EDIT --- to choose the ntop we had to keep track of the smallest
one, and this is interesting info, so let's return also this value
from choose, also we need to initialize decremented
top_points, smallest = choose(lines, ntop)
decremented = top_points
and start a loop...
while True:
Generate a set of decremented values
decremented = {(y-Δy, Δy) for y, Δy in top_points}
decremented = {(y-Δy, Δy) for y, Δy in decremented if y>smallest}
if decremented == {}: break
Generate a set of candidates
candidates = top_lines.union(decremented)
generate a new set of top points
new_top_points, smallest = choose(candidates, ntop)
The following is no more necessary
check if new_top_points == top_points
if new_top_points == top_points: break
top_points = new_top_points</strike>
of course we are in a loop...
The difficult part is the choose function, but I think that this
answer to the question
How can I sort 1 million numbers, and only print the top 10 in Python?
could help you.
It's not a really complicated thing, just a "normal" sorting problem.
Usually sorting requires a large amount of computing time. But your case is one where you don't need to use complex sorting techniques.
You on both graphs are growing or falling constantly, there are no "jumps". You can use this to your advantage. The basic algorithm:
identify if a graph is growing or falling.
write a generator, that generates the values; from left to right if raising, form right to left if falling.
get the first value from both graphs
insert the lower on into the result list
get a new value from the graph that had the lower value
repeat the last two steps until one generator is "empty"
append the leftover items from the other generator.
I am trying to solve Euler problem 18 where I am required to find out the maximum total from top to bottom. I am trying to use recursion, but am stuck with this.
I guess I didn't state my problem earlier. What I am trying to achieve by recursion is to find the sum of the maximum number path. I start from the top of the triangle, and then check the condition is 7 + findsum() bigger or 4 + findsum() bigger. findsum() is supposed to find the sum of numbers beneath it. I am storing the sum in variable 'result'
The problem is I don't know the breaking case of this recursion function. I know it should break when it has reached the child elements, but I don't know how to write this logic in the program.
pyramid=[[0,0,0,3,0,0,0,],
[0,0,7,0,4,0,0],
[0,2,0,4,0,6,0],
[8,0,5,0,9,0,3]]
pos=[0,3]
def downleft(pyramid,pos):#returns down left child
try:
return(pyramid[pos[0]+1][pos[1]-1])
except:return(0)
def downright(pyramid,pos):#returns down right child
try:
return(pyramid[pos[0]+1][pos[1]+1])
except:
return(0)
result=0
def find_max(pyramid,pos):
global result
if downleft(pyramid,pos)+find_max(pyramid,[pos[0]+1,pos[1]-1]) > downright(pyramid,pos)+find_max(pyramid,[pos[0]+1,pos[1]+1]):
new_pos=[pos[0]+1,pos[1]-1]
result+=downleft(pyramid,pos)+find_max(pyramid,[pos[0]+1,pos[1]-1])
elif downright(pyramid,pos)+find_max(pyramid,[pos[0]+1,pos[1]+1]) > downleft(pyramid,pos)+find_max(pyramid,[pos[0]+1,pos[1]-1]):
new_pos=[pos[0]+1,pos[1]+1]
result+=downright(pyramid,pos)+find_max(pyramid,[pos[0]+1,pos[1]+1])
else :
return(result)
find_max(pyramid,pos)
A big part of your problem is that you're recursing a lot more than you need to. You should really only ever call find_max twice recursively, and you need some base-case logic to stop after the last row.
Try this code:
def find_max(pyramid, x, y):
if y >= len(pyramid): # base case, we're off the bottom of the pyramid
return 0 # so, return 0 immediately, without recursing
left_value = find_max(pyramid, x - 1, y + 1) # first recursive call
right_value = find_max(pyramid, x + 1, y + 1) # second recursive call
if left_value > right_value:
return left_value + pyramid[y][x]
else:
return right_value + pyramid[y][x]
I changed the call signature to have separate values for the coordinates rather than using a tuple, as this made the indexing much easier to write. Call it with find_max(pyramid, 3, 0), and get rid of the global pos list. I also got rid of the result global (the function returns the result).
This algorithm could benefit greatly from memoization, as on bigger pyramids you'll calculate the values of the lower-middle areas many times. Without memoization, the code may be impractically slow for large pyramid sizes.
Edit: I see that you are having trouble with the logic of the code. So let's have a look at that.
At each position in the tree you want to make a choice of selecting
the path from this point on that has the highest value. So what
you do is, you calculate the score of the left path and the score of
the right path. I see this is something you try in your current code,
only there are some inefficiencies. You calculate everything
twice (first in the if, then in the elif), which is very expensive. You should only calculate the values of the children once.
You ask for the stopping condition. Well, if you reach the bottom of the tree, what is the score of the path starting at this point? It's just the value in the tree. And that is what you should return at that point.
So the structure should look something like this:
function getScoreAt(x, y):
if at the end: return valueInTree(x, y)
valueLeft = getScoreAt(x - 1, y + 1)
valueRight = getScoreAt(x + 1, y + 1)
valueHere = min(valueLeft, valueRight) + valueInTree(x, y)
return valueHere
Extra hint:
Are you aware that in Python negative indices wrap around to the back of the array? So if you do pyramid[pos[0]+1][pos[1]-1] you may actually get to elements like pyramid[1][-1], which is at the other side of the row of the pyramid. What you probably expect is that this raises an error, but it does not.
To fix your problem, you should add explicit bound checks and not rely on try blocks (try blocks for this is also not a nice programming style).