So I'm building an image search engine where a user can upload an image (query image) , and find other images stored in a directory ( target images) that have the query image in it.
Thats being done using SIFT by extracting the feature maps from both the query and source images and matching them.
Now to make this a feasible application, we'll need to store the feature maps somewhere so that when an image is queried we dont re-extract the feature maps of all images
The problem is that each feature map is around 2k rows with 128 columns per image, totalling about 6mb of storage.
For 1,000,000 image, that would be around 6TB of storage, i dont think this would be scalable and would fit in a database engine properly.
I'm kinda lost from here, how should i store and retrieve such feature maps
Your help is much appreciated!
Edit: the target images are always evolving and not just a static dataset
I'm trying to process some images and obtain numerical output. The skimage library only works with jpg format images. I only have tiff images on hand. Most converting functions work by loading a tiff image and saving it in jpg format. I do agree that the easiest way is
PIL.Image.open('pic.tiff').save('pic.jpg','jpeg')
I'm, on the other hand, trying to abstain from using hard drive for several reasons, but mainly due to the complexity file handling on heroku. Hence the question.
I want to reduce the file size of a PNG file using Python. I have gone through a lot of material on the internet, but I could not find anything where I would reduce the file size for an image without changing the dimensions i.e. height/width. What I have found is how to change the dimensions of the image, using PIL or some other library of Python.
How can I reduce image file size while keeping it's dimensions constant?
PNG is a lossless format but it still can be compressed in a lossless fashion. Pillow's Image class comes with a png writer accepting a compress_level parameter.
It can easily be demonstrated that it works:
Step 1: Load generic rgb lena image in .png format:
import os
import requests
from PIL import Image
from io import BytesIO
img_url = 'http://pngnq.sourceforge.net/testimages/lena.png'
response = requests.get(img_url)
img = Image.open(BytesIO(response.content))
Step 2: Write png with compress_level=0:
path_uncompressed = './lena_uncompressed.png'
img.save(path_uncompressed,compress_level=0)
print(os.path.getsize(path_uncompressed))
> 691968
Step 3: Write png with compress_level=9:
path_compressed = './lena_compressed.png'
img.save(path_compressed,compress_level=9)
print(os.path.getsize(path_compressed))
> 406889
which in this case gives us a respectable 30% compression without any obvious image quality degradation (which should be expected for a lossless compression algorithm).
PNG is lossless format and obviously it will consume more space.
If you are only concerned on resolution, then you can convert to any of the lossy form like JPG.
https://whatis.techtarget.com/definition/lossless-and-lossy-compression
The dimension after conversion would be the same, but quality depends on the compression needed
Snippet to convert PNG to JPG
from PIL import Image
im = Image.open("Picture2.png")
rgb_im = im.convert('RGB')
rgb_im.save('Picture2.jpg')
By default, most PNG writers will use the maximum compression setting, so trying to change that will not help much. Uncompressed PNGs exist, but they make no sense, I don't think you'll run into many of those.
Thus, the only way of making PNGs significantly smaller is to reduce the number of pixels it stores, or to reduce the number of colors it stores. You specifically said not to want to reduce the number of pixels, so the other option is to reduce the number of colors.
Most PNG files will be in "true color" mode (typically 24 bits per pixel, with one byte each for the red, green and blue components of each pixel). However, it is also possible to make indexed PNG files. These store a color map (a.k.a. palette), and a single value per pixel, the index into a color map. If you, for example, pick a color map of 64 entries, then each pixel will need 6 bits to encode the index. You'd store 64*3 bytes + 3/4 bytes per pixel (which of course compress as well). I found this web site comparing a few example images, what they'd look like and how big the file ends up being when reducing colors.
This other question shows how to use PIL to convert an RGB image to an indexed image:
img.convert("P", palette=Image.ADAPTIVE)
This seems to generate an 8-bit color map though, PIL has no support for smaller color maps. The PyPNG module would allow you to write PNG files with any number of colors in the color map.
So, i'm trying to optimize my images from Cloud Store because of using their thumbnails on website, images are tranforming in GAE. And one of the possibilities is converting JPEG images with progressive mode.
Is this possible in google images api to convert simple JPEG image to progressive mode? Or are there another library that i can use in GAE for this task?
The only solution that i can think now is upload images already in progressive mode using jpegtran lib.
I am working on a Image detection problem in opencv for which I need to save many images.
These images are required to be accessed frequently so I wanted to store these images in IPL image format . All these images are grayscale images .
What I wanted to ask is what is the best method to handle all these Images ? Should I store them in a database or a file system ?
Any help would be highly appreciated.