Develop Reference

How to compare specific elements in 2D array in Python

I'm trying to compare these specific elements to find the highest number:
Q_2 = [[5,0,41],[6,3,5],[7,4,3],[8,5,40]]
This is my 2d array in Python I want to compare Q_2[i][2] with each other the example is that number 41 gets compared to 5 and 3 and 40 and the result is the highest number.
I came up with 2 ways:
I store the Q_2[i][2] of the every item to a new list (which I don't know why it wont)
Or I do a loop to compare them
from array import *
#These 2 are used to define columns and rows in for other 2d arrays (All arrays have same column and row)
n = int(3)
m = int(input("Enter number of processes \n")) #I always type 4 for this variable
Q_2 = [[5,0,41],[6,3,5],[7,4,3],[8,5,40]]
for i in range(m):
for j in range(1,3,1):
if(Q_2[i][2]>=Q_2[j][2]:
Max_Timers = Q_2[i]
print(Max_Timers) #to check if the true value is returned or not
The result it returns is 40
This worked when the 0 index was lower then others but once I changed the first one to 41 it no longer works

there is no need of two 'for' loops, as your are after just one element from a 2D array rather than complete 1D array.
This is a working code on 2D array, to get the 1D array that got the highest element on index-2:
max_index = 0
Q_2 = [[5,0,41],[6,3,5],[7,4,3],[8,5,40]]
for i in range(len(Q_2)):
if(Q_2[i][2]>=Q_2[max_index][2]):
max_index = i
print(Q_2[max_index])

The reason your code doesn't work can be easily found out by working out a dry run using the values you are using.
According to your logic,
number 41 gets compared to 5 and 3 and 40 and the result is the highest number
and this is achieved by the two for loops with i and j. But what you overlooked was that the max-value that you calculated was just between the current values and so, you are saving the max value for the current iteration only. So, instead of the Global maximum, only the local/current maximum was being stored.
A quick dry-run of your code (I modified a few lines for the dry-run but with no change in logic) will work like this:
41 is compared to 5 (Max_Timers = 41)
41 is compared to 3 (Max_Timers = 41)
5 is compared to 3 (Max_Timers = 5)
40 is compared to 5 (Max_Timers = 40)
40 is compared to 3 (Max_Timers = 40)
>>> print(Max_timers)
40
So, this is the reason you are getting 40 as a result.
The solution to this is perfectly mentioned by #Rajani B's post, which depicts a global comparison by storing the max value and using that itself for comparison.
Note: I didn't mention this above but even when you used a 2nd for loop, which was already not required, there was an even less reason for you to use a range(1,3,1) in the 2nd loop. As you can see in the dry-run, this resulted in skipping a few of the checks that you probably intended.

You can use numpy to significantly simplify this task and for performance as well. Convert your list into an np.array(), then select the column of interest, which is Q_2[:,2] and apply numpy's .max() method.
import numpy as np
Q_2 = [[5,0,41],[6,3,5],[7,4,3],[8,5,40]]
Q_2 = np.array(Q_2)
mx2 = Q_2[:,2].max()
which gives the desired output:
print(mx2)
41

user indexing for something similar to a matrix

Need to start off by saying that this is essentially a homework question and it’s really a tough one for me.
From the print output of a dynamic matrix (or anything that can be formatted and printed to be visually similar), I need to use indexing from a user’s input to change values in that “matrix”. It doesn't have to be a matrix, something that can be formatted to be similar would also work.
What makes this problem hard for me is that only un-nested lists, strings, or dictionaries can be used, and importing packages is not allowed. So, list comprehension is out of the question.
One thing I’ve tried so far is to print separate lists and index based on separate lists but I got stuck.

You can use a 1D list, take x, y coordinates in user input and convert the y coordinate in terms of x offsets.
For example, say you want to represent a 5x3 array and the user wants the second column (x=2) and third row (y=3). Let's assume our matrix displays with 1,1 being top left corner.
Multiply the y coordinate minus 1 by the width of the matrix to obtain the number of cell offsets in your 1D-list, then further offset by x - 1 (remember, Python lists are 0-based) to position correctly on the x-axis.
Example of matrix with 1D-based indices:
0 | 1 | 2 | 3 | 4
5 | 6 | 7 |  8 | 9
10 | 11 | 12 | 13 | 14
Taking the algorithm above:
index = (y - 1) * 5 + x - 1 # ((3 - 1) * 5 + 2 - 1) = 11
As you can see, 11 is indeed in our matrix the second column and third row, as per the example user inputs.
You can then display the matrix by way of a simple for loop, knowing the size of the matrix and inserting a new line as appropriate.
You may simplify the above a bit if the user is requested to input 0-based indices as well. You will not need to substract 1 from x and y.

Random Sudoku Generator

I'm trying to build a python script that generates a 9x9 block with numbers 1-9 that are unique along the rows, columns and within the 3x3 blocks - you know, Sudoku!
So, I thought I would start simple and get more complicated as I went. First I made it so it randomly populated each array value with a number 1-9. Then made sure numbers along rows weren't replicated. Next, I wanted to the same for rows & columns. I think my code is OK - it's certainly not fast but I don't know why it jams up..
import numpy as np
import random
#import pdb
#pdb.set_trace()
#Soduku solver!
#Number input
soduku = np.zeros(shape=(9,9))
for i in range(0,9,1):
for j in range(0,9,1):
while True:
x = random.randint(1,9)
if x not in soduku[i,:] and x not in soduku[:,j]:
soduku[i,j] = x
if j == 8: print(soduku[i,:])
break
So it moves across the columns populating with random ints, drops a row and repeats. The most the code should really need to do is generate 9 numbers for each square if it's really unlucky - I think if we worked it out it would be less than 9*9*9 values needing generating. Something is breaking it!
Any ideas?!

I think what's happening is that your code is getting stuck in your while-loop. You test for the condition if x not in soduku[i,:] and x not in soduku[:,j], but what happens if this condition is not met? It's very likely that your code is running into a dead-end sudoku board (can't be solved with any values), and it's getting stuck inside the while-loop because the condition to break can never be met.

Generating it like this is very unlikely to work. There are many ways where you can generate 8 of the 9 3*3 squares making it impossible to fill in the last square at all, makign it hang forever.
Another approach would be to fill in all the numbers on at the time (so, all the 1s first, then all the 2s, etc.). It would be like the Eight queens puzzle, but with 9 queens. And when you get to a position where it is impossible to place a number, restart.
Another approach would be to start all the squares at 9 and strategically decrement them somehow, e.g. first decrement all the ones that cannot be 9, excluding the 9s in the current row/column/square, then if they are all impossible or all possible, randomly decrement one.
You can also try to enumerate all sudoku boards, then reverse the enumaration function with a random integer, but I don't know how successful this may be, but this is the only method where they could be chosen with uniform randomness.

You are coming at the problem from a difficult direction. It is much easier to start with a valid Sudoku board and play with it to make a different valid Sudoku board.
An easy valid board is:
1 2 3 | 4 5 6 | 7 8 9
4 5 6 | 7 8 9 | 1 2 3
7 8 9 | 1 2 3 | 4 5 6
---------------------
2 3 4 | 5 6 7 | 8 9 1
5 6 7 | 8 9 1 | 2 3 4
8 9 1 | 2 3 4 | 5 6 7
---------------------
3 4 5 | 6 7 8 | 9 1 2
6 7 8 | 9 1 2 | 3 4 5
9 1 2 | 3 4 5 | 6 7 8
Having found a valid board you can make a new valid board by playing with your original.
You can swap any row of three 3x3 blocks with any other block row. You can swap any column of three 3x3 blocks with another block column. Within each block row you can swap single cell rows; within each block column you can swap single cell columns. Finally you can permute the digits so there are different digits in the cells as long as the permutation is consistent across the whole board.
None of these changes will make a valid board invalid.

I use permutations(range(1,10)) from itertools to create a list of all possible rows. Then I put each row into a sudoku from top to bottom one by one. If contradicts occurs, use another row from the list. In this approach, I can find out some valid completed sudoku board in a short time. It continue generate completed board within a minute.
And then I remove numbers from the valid completed sudoku board one by one in random positions. After removing each number, check if it still has unique solution. If not, resume the original number and change to next random position. Usually I can remove 55~60 numbers from the board. It take time within a minute, too. It is workable.
However, the first few generated the completed sudoku board has number 1,2,3,4,5,6,7,8,9 in the first row. So I shuffle the whole list. After shuffling the list, it becomes difficult to generate a completed sudoku board. Mission fails.
A better approach may be in this ways. You collect some sudoku from the internet. You complete them so that they are used as seeds. You remove numbers from them as mention above in paragraph 2. You can get some sudoku. You can use these sudokus to further generate more by any of the following methods
swap row 1 and row 3, or row 4 and row 6, or row 7 and row 9
similar method for columns
swap 3x3 blocks 1,4,7 with 3,6,9 or 1,2,3 with 7,8,9 correspondingly.
mirror the sudoku vertical or horizontal
rotate 90, 180, 270 the sudoku
random permute the numbers on the board. For example, 1->2, 2->3, 3->4, .... 8->9, 9->1. Or you can just swap only 2 of them. eg. 1->2, 2->1. This also works.

Best way to hash ordered permutation of [1,9]

I'm trying to implement a method to keep the visited states of the 8 puzzle from generating again.
My initial approach was to save each visited pattern in a list and do a linear check each time the algorithm wants to generate a child.
Now I want to do this in O(1) time through list access. Each pattern in 8 puzzle is an ordered permutation of numbers between 1 to 9 (9 being the blank block), for example 125346987 is:
1 2 5
3 4 6
_ 8 7
The number of all of the possible permutation of this kind is around 363,000 (9!). what is the best way to hash these numbers to indexes of a list of that size?

You can map a permutation of N items to its index in the list of all permutations of N items (ordered lexicographically).
Here's some code that does this, and a demonstration that it produces indexes 0 to 23 once each for all permutations of a 4-letter sequence.
import itertools
def fact(n):
r = 1
for i in xrange(n):
r *= i + 1
return r
def I(perm):
if len(perm) == 1:
return 0
return sum(p < perm[0] for p in perm) * fact(len(perm) - 1) + I(perm[1:])
for p in itertools.permutations('abcd'):
print p, I(p)
The best way to understand the code is to prove its correctness. For an array of length n, there's (n-1)! permutations with the smallest element of the array appearing first, (n-1)! permutations with the second smallest element appearing first, and so on.
So, to find the index of a given permutation, see count how many items are smaller than the first thing in the permutation and multiply that by (n-1)!. Then recursively add the index of the remainder of the permutation, considered as a permutation of (n-1) elements. The base case is when you have a permutation of length 1. Obviously there's only one such permutation, so its index is 0.
A worked example: [1324].
[1324]: 1 appears first, and that's the smallest element in the array, so that gives 0 * (3!)
Removing 1 gives us [324]. The first element is 3. There's one element that's smaller, so that gives us 1 * (2!).
Removing 3 gives us [24]. The first element is 2. That's the smallest element remaining, so that gives us 0 * (1!).
Removing 2 gives us [4]. There's only one element, so we use the base case and get 0.
Adding up, we get 0*3! + 1*2! + 0*1! + 0 = 1*2! = 2. So [1324] is at index 2 in the sorted list of 4 permutations. That's correct, because at index 0 is [1234], index 1 is [1243], and the lexicographically next permutation is our [1324].

I believe you're asking for a function to map permutations to array indices. This dictionary maps all permutations of numbers 1-9 to values from 0 to 9!-1.
import itertools
index = itertools.count(0)
permutations = itertools.permutations(range(1, 10))
hashes = {h:next(index) for h in permutations}
For example, hashes[(1,2,5,3,4,6,9,8,7)] gives a value of 1445.
If you need them in strings instead of tuples, use:
permutations = [''.join(x) for x in itertools.permutations('123456789')]
or as integers:
permutations = [int(''.join(x)) for x in itertools.permutations('123456789')]

It looks like you are only interested in whether or not you have already visited the permutation.
You should use a set. It grants the O(1) look-up you are interested in.

A space as well lookup efficient structure for this problem is a trie type structure, as it will use common space for lexicographical matches in any
permutation.
i.e. the space used for "123" in 1234, and in 1235 will be the same.
Lets assume 0 as replacement for '_' in your example for simplicity.
Storing
Your trie will be a tree of booleans, the root node will be an empty node, and then each node will contain 9 children with a boolean flag set to false, the 9 children specify digits 0 to 8 and _ .
You can create the trie on the go, as you encounter a permutation, and store the encountered digits as boolean in the trie by setting the bool as true.
Lookup
The trie is traversed from root to children based on digits of the permutation, and if the nodes have been marked as true, that means the permutation has occured before. The complexity of lookup is just 9 node hops.
Here is how the trie would look for a 4 digit example :
Python trie
This trie can be easily stored in a list of booleans, say myList.
Where myList[0] is the root, as explained in the concept here :
https://webdocs.cs.ualberta.ca/~holte/T26/tree-as-array.html
The final trie in a list would be around 9+9^2+9^3....9^8 bits i.e. less than 10 MB for all lookups.

Use
I've developed a heuristic function for this specific case. It is not a perfect hashing, as the mapping is not between [0,9!-1] but between [1,767359], but it is O(1).
Let's assume we already have a file / reserved memory / whatever with 767359 bits set to 0 (e.g., mem = [False] * 767359). Let a 8puzzle pattern be mapped to a python string (e.g., '125346987'). Then, the hash function is determined by:
def getPosition( input_str ):
data = []
opts = range(1,10)
n = int(input_str[0])
opts.pop(opts.index(n))
for c in input_str[1:len(input_str)-1]:
k = opts.index(int(c))
opts.pop(k)
data.append(k)
ind = data[3]<<14 | data[5]<<12 | data[2]<<9 | data[1]<<6 | data[0]<<3 | data[4]<<1 | data[6]<<0
output_str = str(ind)+str(n)
output = int(output_str)
return output
I.e., in order to check if a 8puzzle pattern = 125346987 has already been used, we need to:
pattern = '125346987'
pos = getPosition(pattern)
used = mem[pos-1] #mem starts in 0, getPosition in 1.
With a perfect hashing we would have needed 9! bits to store the booleans. In this case we need 2x more (767359/9! = 2.11), but recall that it is not even 1Mb (barely 100KB).
Note that the function is easily invertible.
Check
I could prove you mathematically why this works and why there won't be any collision, but since this is a programming forum let's just run it for every possible permutation and check that all the hash values (positions) are indeed different:
def getPosition( input_str ):
data = []
opts = range(1,10)
n = int(input_str[0])
opts.pop(opts.index(n))
for c in input_str[1:len(input_str)-1]:
k = opts.index(int(c))
opts.pop(k)
data.append(k)
ind = data[3]<<14 | data[5]<<12 | data[2]<<9 | data[1]<<6 | data[0]<<3 | data[4]<<1 | data[6]<<0
output_str = str(ind)+str(n)
output = int(output_str)
return output
#CHECKING PURPOSES
def addperm(x,l):
return [ l[0:i] + [x] + l[i:] for i in range(len(l)+1) ]
def perm(l):
if len(l) == 0:
return [[]]
return [x for y in perm(l[1:]) for x in addperm(l[0],y) ]
#We generate all the permutations
all_perms = perm([ i for i in range(1,10)])
print "Number of all possible perms.: "+str(len(all_perms)) #indeed 9! = 362880
#We execute our hash function over all the perms and store the output.
all_positions = [];
for permutation in all_perms:
perm_string = ''.join(map(str,permutation))
all_positions.append(getPosition(perm_string))
#We wan't to check if there has been any collision, i.e., if there
#is one position that is repeated at least twice.
print "Number of different hashes: "+str(len(set(all_positions)))
#also 9!, so the hash works properly.
How does it work?
The idea behind this has to do with a tree: at the beginning it has 9 branches going to 9 nodes, each corresponding to a digit. From each of these nodes we have 8 branches going to 8 nodes, each corresponding to a digit except its parent, then 7, and so on.
We first store the first digit of our input string in a separate variable and pop it out from our 'node' list, because we have already taken the branch corresponding to the first digit.
Then we have 8 branches, we choose the one corresponding with our second digit. Note that, since there are 8 branches, we need 3 bits to store the index of our chosen branch and the maximum value it can take is 111 for the 8th branch (we map branch 1-8 to binary 000-111). Once we have chosen and store the branch index, we pop that value out, so that the next node list doesn't include again this digit.
We proceed in the same way for branches 7, 6 and 5. Note that when we have 7 branches we still need 3 bits, though the maximum value will be 110. When we have 5 branches, the index will be at most binary 100.
Then we get to 4 branches and we notice that this can be stored just with 2 bits, same for 3 branches. For 2 branches we will just need 1bit, and for the last branch we don't need any bit: there will be just one branch pointing to the last digit, which will be the remaining from our 1-9 original list.
So, what we have so far: the first digit stored in a separated variable and a list of 7 indexes representing branches. The first 4 indexes can be represented with 3bits, the following 2 indexes can be represented with 2bits and the last index with 1bit.
The idea is to concatenate all this indexes in their bit form to create a larger number. Since we have 17bits, this number will be at most 2^17=131072. Now we just add the first digit we had stored to the end of that number (at most this digit will be 9) and we have that the biggest number we can create is 1310729.
But we can do better: recall that when we had 5 branches we needed 3 bits, though the maximum value was binary 100. What if we arrange our bits so that those with more 0s come first? If so, in the worst case scenario our final bit number will be the concatenation of:
100 10 101 110 111 11 1
Which in decimal is 76735. Then we proceed as before (adding the 9 at the end) and we get that our biggest possible generated number is 767359, which is the ammount of bits we need and corresponds to input string 987654321, while the lowest possible number is 1 which corresponds to input string 123456789.
Just to finish: one might wonder why have we stored the first digit in a separate variable and added it at the end. The reason is that if we had kept it then the number of branches at the beginning would have been 9, so for storing the first index (1-9) we would have needed 4 bits (0000 to 1000). which would have make our mapping much less efficient, as in that case the biggest possible number (and therefore the amount of memory needed) would have been
1000 100 10 101 110 111 11 1
which is 1125311 in decimal (1.13Mb vs 768Kb). It is quite interesting to see that the ratio 1.13M/0.768K = 1.47 has something to do with the ratio of the four bits compared to just adding a decimal value (2^4/10 = 1.6) which makes a lot of sense (the difference is due to the fact that with the first approach we are not fully using the 4 bits).

First. There is nothing faster than a list of booleans. There's a total of 9! == 362880 possible permutations for your task, which is a reasonably small amount of data to store in memory:
visited_states = [False] * math.factorial(9)
Alternatively, you can use array of bytes which is slightly slower (not by much though) and has a much lower memory footprint (by a power of magnitude at least). However any memory savings from using an array will probably be of little value considering the next step.
Second. You need to convert your specific permutation to it's index. There are algorithms which do this, one of the best StackOverflow questions on this topic is probably this one:
Finding the index of a given permutation
You have fixed permutation size n == 9, so whatever complexity an algorithm has, it will be equivalent to O(1) in your situation.
However to produce even faster results, you can pre-populate a mapping dictionary which will give you an O(1) lookup:
all_permutations = map(lambda p: ''.join(p), itertools.permutations('123456789'))
permutation_index = dict((perm, index) for index, perm in enumerate(all_permutations))
This dictionary will consume about 50 Mb of memory, which is... not that much actually. Especially since you only need to create it once.
After all this is done, checking your specific combination is done with:
visited = visited_states[permutation_index['168249357']]
Marking it to visited is done in the same manner:
visited_states[permutation_index['168249357']] = True
Note that using any of permutation index algorithms will be much slower than mapping dictionary. Most of those algorithms are of O(n2) complexity and in your case it results 81 times worse performance even discounting the extra python code itself. So unless you have heavy memory constraints, using mapping dictionary is probably the best solution speed-wise.
Addendum. As has been pointed out by Palec, visited_states list is actually not needed at all - it's perfectly possible to store True/False values directly in the permutation_index dictionary, which saves some memory and an extra list lookup.

Notice if you type hash(125346987) it returns 125346987. That is for a reason, because there is no point in hashing an integer to anything other than an integer.
What you should do, is when you find a pattern add it to a dictionary rather than a list. This will provide the fast lookup you need rather than traversing the list like you are doing now.
So say you find the pattern 125346987 you can do:
foundPatterns = {}
#some code to find the pattern
foundPatterns[1] = 125346987
#more code
#test if there?
125346987 in foundPatterns.values()
True

If you must always have O(1), then seems like a bit array would do the job. You'd only need to store 363,000 elements, which seems doable. Though note that in practice it's not always faster. Simplest implementation looks like:
Create data structure
visited_bitset = [False for _ in xrange(373000)]
Test current state and add if not visited yet
if !visited[current_state]:
visited_bitset[current_state] = True

Paul's answer might work.
Elisha's answer is perfectly valid hash function that would guarantee that no collision happen in the hash function. The 9! would be a pure minimum for a guaranteed no collision hash function, but (unless someone corrects me, Paul probably has) I don't believe there exists a function to map each board to a value in the domain [0, 9!], let alone a hash function that is nothing more that O(1).
If you have a 1GB of memory to support a Boolean array of 864197532 (aka 987654321-12346789) indices. You guarantee (computationally) the O(1) requirement.
Practically (meaning when you run in a real system) speaking this isn't going to be cache friendly but on paper this solution will definitely work. Even if an perfect function did exist, doubt it too would be cache friendly either.
Using prebuilts like set or hashmap (sorry I haven't programmed Python in a while, so don't remember the datatype) must have an amortized 0(1). But using one of these with a suboptimal hash function like n % RANDOM_PRIME_NUM_GREATER_THAN_100000 might give the best solution.

Generation of values above diagonal numpy array

Suppose you have a 10x10 numpy array of intensity values extracted from an image. The exact numbers do not matter right now. I would like to take this matrix, and generate vertices for a graph using only the vertex locations of the upper half of the matrix. More specifically, if our matrix dimensions are defined as (MxN), we could possibly write something like this:
for x in range(M1,M2,M3...M10):
for y in range(N1,N2,N3...N10):
if (x-y) >=0:
graph.addVertex(x,y)
The graph class definition and addVertex definition are NOT important to me as I already have them written. I am only concerned about a method in which I can only consider vertices which are above the diagonal. Open to any and all suggestions, my suggestion above is merely a starting point that may possibly be helpful. Thanks!
EDIT: SOLUTION
Sorry if my clarity issues were atrocious, as I'm somewhat new to coding in Python, but this is the solution to my issue:
g=Graph()
L=4
outerindex=np.arange(L**2*L**2).reshape((L**2,L**2))
outerindex=np.triu(outerindex,k=0)
for i in range(len(outerindex)):
if outerindex.any()>0:
g.addVertex(i)
In this manner, when adding vertices to our newly formed graph, the only new vertices formed will be those that reside in locations above the main diagonal.

I think what you want is something like this:
import numpy as np
a = np.arange(16).reshape((4,4))
print a
for i in range(4):
for j in range(i, 4):
print a[i,j],
# [[ 0 1 2 3]
# [ 4 5 6 7]
# [ 8 9 10 11]
# [12 13 14 15]]
# 0 1 2 3 5 6 7 10 11 15
That is, the key point here is to make the index of the inner loop dependent on the outer loop.
If you don't want to include the diagonal, use the inner loop with range(i+1,4).