Develop Reference

How to crop images using OpenCV without knowing the exact coordinates?

I am trying to crop an image of a piece of card/paper or such so that the card/paper is in focus. I tried the below code but the problem is that it works only when the object in question is alone in the picture. If it is a blank background with nothing else in it- the cropping is flawless, otherwise it does not work as expected.
I am attempting create a system which crops different kinds of images and puts them through a classifier and then extracts text from them.
import cv2
import numpy as np
filenames = "img.jpg"
img = cv2.imread(filenames)
blurred = cv2.blur(img, (3,3))
canny = cv2.Canny(blurred, 50, 200)
## find the non-zero min-max coords of canny
pts = np.argwhere(canny>0)
y1,x1 = pts.min(axis=0)
y2,x2 = pts.max(axis=0)
## crop the region
cropped = img[y1:y2, x1:x2]
filename_cropped = filenames.split('.')
filename_cropped[0] = filename_cropped[0] + '_cropped'
filename_cropped = '.'.join(filename_cropped)
cv2.imwrite(filename_cropped, cropped)
An sample image that works is
Something that does not work is
Can anyone help with this?

The first image works because the entire images besides your target is empty. Canny will also give other results when there is more in the image.
If you are looking for those specific cards I suggest you try to use some colour filtering first. You can try to filer for the blue/purple hue of the card.
Increasing the canny threshold could also work, but you will always still be finding the hand as well in this image unless you add some colour filtering.
You can also try Sobel edge detection . This will probably highlight the instant edges of the card pretty well. But then again, it will also show the hand, so you can't just take all the Sobel/Canny outputs. You need to add processing before it that isolates the card, or after it that can find the rectangular shape of the card in the sobel/canny.

How does cv2.findContours() edit the image?

I have two questions.
I am working with openCv and python and I am trying to have an image's contours. I am succesfull at that but when I try to se what is the difference between when I use cv2.drawContorus() functions and directly edit image with cv2.findContours() without sending a copy of original image as the source parameter. I have tried on some images but I couldnt see anything even happenning.
I am trying to get the contours of a square I created with paint square tool. But when I try with cv2.CHAIN_APPROX_SIMPLE method, it gives me coordinates of 6 points which none of the combinations from them is suitable for my square. Why does it do like that?
Can someone explain?
Here is my code for both problems:
import cv2
import numpy as np
image = cv2.imread(r"C:\Users\fazil\Desktop\12.png")
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
gray = cv2.Canny(gray,75,200)
gray = cv2.threshold(gray,127,255,cv2.THRESH_BINARY_INV)[1]
cv2.imshow("s",gray)
contours, hiearchy = cv2.findContours(gray,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
print(contours[1])
cv2.drawContours(image,contours,1,(45,67,89),5)
cv2.imshow("k",gray)
cv2.imshow("j",image)
cv2.waitKey(0)
cv2.destroyAllWindows()

Extract car images without Mask RCNN

I want to extract car images without using Mask RCNN. I tried a couple of methods but couldn't decide on how to proceed with any of them. I need recommendation on which method would be best and how to go through with it.
Method 1 - Using XML files and haar cascade classifier
I was thinking of using xml files to detection and crop car images. The problems I faced were:
They only detect car in square shapes. I needed car images cropped. So ultimately I ended up with better images of cropped cars. This didn't solve my problem.
The cropped image didn't detect car as a whole but small parts of it. Maybe due to XML file's config.
My code:
!wget https://raw.githubusercontent.com/shaanhk/New-GithubTest/master/cars.xml
import numpy as py
import cv2
car_cascade=cv2.CascadeClassifier('cars.xml')
img = cv2.imread('im1.jpg')
cars = car_cascade.detectMultiScale(img, 1.1, 1)
for (x,y,w,h) in cars:
cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),2)
cv2.imshow('image',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Resulting image:
Method 2 - Using Canny Edge Detection
I tried to perform canny edge detection for car. It worked to some extent that I managed to reduce edges to mostly car object. But I don't know how to proceed from there.
My code:
import cv2
import numpy as np
image= cv2.imread('im1.jpg')
imagecopy= np.copy(image)
grayimage= cv2.cvtColor(imagecopy, cv2.COLOR_RGB2GRAY)
canny= cv2.Canny(grayimage, 300,150)
cv2.imshow('Highway Edge Detection Image', canny)
cv2.waitKey(0)
cv2.destroyAllWindows()
Resulting Image:
Method 3 - Extract car image using color gradients
On googling I found a method using HSV transformation and then creating a custom mask to extract cars. But I don't know much about this method and have no idea how to go about it. I used the code provided and am posting it below.
Code:
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import cv2
%matplotlib inline
image = mpimg.imread('im1.jpg')
hsv = cv2.cvtColor(image, cv2.COLOR_RGB2HSV)
# HSV channels
h = hsv[:,:,0]
s = hsv[:,:,1]
v = hsv[:,:,2]
background_hue = h[10,10]
lower_hue = np.array([background_hue-10,0,0])
upper_hue = np.array([background_hue+10,255,255])
mask = cv2.inRange(hsv, lower_hue, upper_hue)
# Mask the image to let the car show through
masked_image = np.copy(image)
masked_image[mask != 0] = [0, 0, 0]
cv2.imwrite('mask.jpg',masked_image)
# Display it!
plt.imshow(masked_image)
Image:
I'd like to mention, I'm a complete beginner in Computer Vision and am trying to learn by doing some small stuff like these. My code is probably very flawed and hopefully I can work on it on the way. Please feel absolutely free to mention any other method (except Mask RCNN) or any problems with code.

Why does pytesseract fail to recognise digits from image with darker background?

I've this python code which I use to convert a text written in a picture to a string, it does work for certain images which have large characters, but not for the one I'm trying right now which contains only digits.
This is the picture:
This is my code:
import pytesseract
from PIL import Image
img = Image.open('img.png')
pytesseract.pytesseract.tesseract_cmd = 'C:/Program Files (x86)/Tesseract-OCR/tesseract'
result = pytesseract.image_to_string(img)
print (result)
Why is it failing at recognising this specific image and how can I solve this problem?

I have two suggestions.
First, and this is by far the most important, in OCR preprocessing images is key to obtaining good results. In your case I suggest binarization. Your images look extremely good so you shouldn't have any problem but if you do, then maybe you should try to binarize your images:
import cv2
from PIL import Image
img = cv2.imread('gradient.png')
# If your image is not already grayscale :
# img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
threshold = 180 # to be determined
_, img_binarized = cv2.threshold(img, threshold, 255, cv2.THRESH_BINARY)
pil_img = Image.fromarray(img_binarized)
And then try the ocr again with the binarized image.
Check if your image is in grayscale and uncomment if needed.
This is simple thresholding. Adaptive thresholding also exists but it is noisy and does not bring anything in your case.
Binarized images will be much easier for Tesseract to handle. This is already done internally (https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuality) but sometimes things can be messed up and very often it's useful to do your own preprocessing.
You can check if the threshold value is right by looking at the images :
import matplotlib.pyplot as plt
plt.imshow(img, cmap='gray')
plt.imshow(img_binarized, cmap='gray')
Second, if what I said above still doesn't work, I know this doesn't answer "why doesn't pytesseract work here" but I suggest you try out tesserocr. It is a maintained python wrapper for Tesseract.
You could try:
import tesserocr
text_from_ocr = tesserocr.image_to_text(pil_img)
Here is the doc for tesserocr from pypi : https://pypi.org/project/tesserocr/
And for opencv : https://pypi.org/project/opencv-python/
As a side-note, black and white is treated symetrically in Tesseract so having white digits on a black background is not a problem.

OpenCV - RETR_EXTERNAL not working after Otsu's binarization

I have applied the Otsu's binarization to one image and got this result
After that, I use this code to get boxes around the four main shapes:
img = cv.imread('test_bin.jpg', 0)
_, cnts, _ = cv.findContours(img.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_NONE)
for cnt in cnts:
x,y,w,h = cv.boundingRect(cnt)
cv.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv.imwrite('test_cnt.jpg', img)
However, I'm not getting anything. It returns just one contour which I imagine it could be the full image itself. I saw it works for RETR_TREE, but I need it to work with RETR_EXTERNAL for the next operations. What is failing here?

As per the OpenCV contours documentation:
In OpenCV, finding contours is like finding white object from black
background. So remember, object to be found should be white and
background should be black.
But in your case, it is clearly the opposite of the requirements, so you just need to invert your image and it can be simply done as:
img = cv2.bitwise_not(img)
Also, note that:
For better accuracy, use binary images. So before finding contours,
apply threshold or canny edge detection.
I used your image and got following results, after inverting the image. If you want to remove the small boxes, then simply use cv2.threshold to get a binary image.