Develop Reference

Python - video processing. How to: 1) change pixels value in videos (ie pixelated effect), and then 2) retrieve every single pixel's information

This question is related to an actual science project.
1. The video will be done whit the medium of a white moving object against a black background surface. The final results will be close to a black/white output but I also need to render the video into an actual dual-color channel video (only displaying black and white pixels). I guess it should also be useful to delete/reduce noise (unrelated white pixels caused by light interference)
2. After that, I need to modify the pixels values in order to get a pixelated effect
3. at this very point, I need to obtain all the pixel values from all the frames, maybe arranged in a matrix fashion because I will need to perform some computational math on these matrices
let me show you an example:
frame 1. (0 for black pixel values, 1 for white pixel values)
01110000
01111001
01100011
11101111
01100000
10000000
frame 2. 00111000
01011100
11100000
10010100
10000101
01010101
and so on...

# You could store it in a list like this:
frame1 = [.01110000, .01111001] # and so on. Because an int can't start on zero, I made these decilams
# To change the first item in this list do the following
frame1[0] = .01110011 # This will be the new value and the current list looks like this:
# frame1 = [.01110011, .01111001]
# To retrieve information about every element in the list simply use a for loop
n = -1
for element in frame1:
n += 1
element = frame1[n]
print(element)

How do I fit rectangles to an image in python and obtain their coordinates

I'm looking for a way to split a number of images into proper rectangles. These rectangles are ideally shaped such that each of them take on the largest possible size without containing a lot of white.
So let's say that we have the following image
I would like to get an output such as this:
Note the overlapping rectangles, the hole and the non axis aligned rectangle, all of these are likely scenario's I have to deal with.
I'm aiming to get the coordinates describing the corner pieces of the rectangles so something like
[[(73,13),(269,13),(269,47)(73,47)],
[(73,13),(73,210),(109,210),(109,13)]
...]
In order to do this I have already looked at the cv2.findContours but I couldn't get it to work with overlapping rectangles (though I could use the hierarchy model to deal with holes as that causes the contours to be merged into one.
Note that although not shown holes can be nested.

A algorithm that works roughly as follow should be able to give you the result you seek.
Get all the corner points in the image.
Randomly select 3 points to create a rectangle
Count the ratio of yellow pixels within the rectangle, accept if the ratio satisfy a threshold.
Repeat 2 to 4 until :
a) every single combination of point is complete or
b) all yellow pixel are accounted for or
c) after n number of iteration
The difficult part of this algorithm lies in step 2, creating rectangle from 3 points.
If all the rectangles were right angle, you can simply find the minimum x and y to correspond for topLeft corner and maximum x and y to correspond for bottomRight corner of your new rectangle.
But since you have off axis rectangle, you will need to check if the two vector created from the 3 points have a 90 degree angle between them before generating the rectangle.

find closest clusters of colors in a numpy array from an image file

Current state
I have a numpy array of shape (900, 1800, 3) that has been made from an image file.
That's one array element per pixel: 900 px high, 1800 px wide, and 3 channels (R, G, B) per pixel represented in the array.
There are only a small number (3-20) unique RGB colors in the images being parsed, so there are only very few different RGB value combinations represented in the array.
Goal
Identify the smallest circular areas in the image that contains n number of unique colors, where n will always be less than or equal to the number of unique colors in the image.
Return top y (by count or pct) of the smallest areas.
A 'result' could simply be the x,y value of the center pixel of an identified circular area and its radius.
I do plan to draw a circle around each area, but this question is about the best approach for first identifying the top smallest areas.
The Catch/Caveat
The images are actually flattened projections of spheres. That means that a pixel at the right edge of the image is actually adjacent to a pixel on the left edge, and similarly for top and bottom pixels. The solution must account for this as it is parsing pixels to identify closest pixels with other colors. EDIT: this part may be answered in comments below
The Question
My initial approach is to simply parse pixel by pixel and brute force the problem with handrolled x/y coordinate math: take a pixel, work outwards until we hit n colors, score that pixel for how many steps outward it took, next pixel. Keep a top y dict that gets re-evaluated after each pixel, adding any pixels that make top y, and dumping any that get pushed out. Return that dict as the output.
I know that many python libs like scipy, scikit-image, and maybe others like to work with images as numpy arrays. I'm sure there is a method/approach that is smarter and leverages a library or some kind of clustering algo instead of brute forcing it, but I'm not familiar enough with the space to know intuitively what methods and libs to consider. The question: What is the pseudocode for a good method/lib to do this the right way?

Split image horizontally with python based on dark center line

Goal is to horizontally split an image (double newspaper page) in python based on a vertical centerline that is darker than other areas around.
Example image:
Had some luck using opencv (cv2) for the initial crop and rotation of the double page from a black background using cv2.Canny, and then sorting the contours based on cv2.contourArea.
But, now I'm just interested in finding a center line and then splitting the image into two separate images. Using cv2.Canny again I see that it's able to identify that centerline, but not sure how to identify that long, vertical line and use that to split the image:
End goal would be two images like the following:
Any suggestions would be most welcome.

First, run a horizontal gradient so you only accentuate vertical edges. You can calculate a horizontal gradient with these coefficients:
-1 0 1
-2 0 2
-1 0 1
Then compute the sum of the vertical columns, you can use np.sum(array,axis=0) and you will get this:
I have re-shaped it for ease of viewing - it is actually only 1 pixel tall. Hopefully you can see the bright white line in the middle which you can find with Numpy argmax(). It will also be better when you just do a horizontal gradient because at the moment I am using the purple and yellow image with vertical and horizontal edges enhanced.
Note that the inspiration for this approach is that you said you "want to identify that long, vertical centerline" and the rationale is that a long line of white pixels will add up to a large sum. Note that I have assumed your image is de-skewed (since you said the line is vertical) and this method may not work so well on skew images where "vertical" line will be spread across several columns.

How to measure image coincidence in an optical rangefinder

I have a couple of USB webcams (fixed focal length) setup as a simple stereoscopic rangefinder, spaced N mm apart with each rotated by M degrees towards the centerline, and I've calibrated the cameras to ensure alignment.
When adjusting the angle, how would I measure the coincidence between the images (preferably in Python/PIL/OpenCV) to know when the cameras are focused on an object? Is it as simple as choosing a section of pixels in each image (A rows by B columns) and calculating the sum of the difference between the pixels?

the problem is that you can not assume pixel perfect align of cameras
so let assume x-axis is the parallax shifted axis and y- axis is aligned. You need to identify the x-axis image distortion/shift to detect parallax align even if you are aligned as much as possible. The result of abs difference is not guaranteed to be in min/max so instead of substracting individual pixels substract average color of nearby area of that pixel with radius/size bigger then the align error in y-axis. Let call this radius or size r this way the resulting difference should be minimal when aligned.
Approximation search
You can even speed up the process by r
select big r
scan whole x-range with step for example 0.25*r
choose the lowest difference x-position (x0)
change r to half
go to bullet 2 (but this time whole x range is just between <x0-2.0*r,x0+2.0r>
stops if r is smaller then few pixels
This way you can search in O(log2(n)) instead of O(n)
computer vision approach
this should be even faster:
detect points of interest (in booth images)
specific change in gradient,etc ...
cross match points of interest between images
compute average x-distance between cross-matched points
change parallax align by found distance of points
goto bullet 1 until x-distance is small enough
This way you can avoid checking whole x-range because the align distance is obtained directly ... You just need to convert it to angle or what ever you use to align parallax
[notes]
You do not need to do this on whole image area just select few horizontal lines along the images and scan their nearby area.
There are also another ways to detect align for example for short distances the skew is significant marker of align so compare the height of object on its left and right side between cameras ... If near the same you are aligned if bigger/smaller you are not aligned and know which way to turn ...