Develop Reference

Compression PNG with PIL and cv2

Im trying to make image compressor in my django project. I did well with jpg, but got a lot of problems with png. For compression i using PIL and cv2, but cant get result better then 16% of compression for big PNG files (>1 mb). Ive tryed to combine both libraries, and its still not innove. Here simple code of my view:
(the above code for jpg compression)
elif picture.mode == ('RGBA'):
if photo.image.size < 1000000:
colorsloss = picture.convert(mode="P", palette=Image.ADAPTIVE)
colorsloss.save('media/new/'+name,"PNG",quality=75, optimize=True, bits=8)
else:
originalImage = cv.imread(str('/home/andrey/sjimalka'+ photo.image.url))
cv.imwrite('media/new/'+name, originalImage,[cv.IMWRITE_PNG_COMPRESSION, 9])
cvcompressed = Image.open('media/new/'+name)
cvcompressed.convert(mode="RGB")
cvcompressed.save('media/new/'+name,"PNG",quality=75, optimize=True)
So here ive got 2 big problems:
1) If ive got low size image (< 1 mb), i using P mode in Pillow. It works great, but if i compressing image with gradient, i can see some distortions in places where i got gradient.
I have good compression (something like 85%), but no idea yet how to fix it.
2) I cant get good compression of big png files. My best goal yet is 16%, with really good quality, but it still not innove. Mb i do something wrong, or i shold use any other library or technology to make it better. I want to get a list 50% of compression with big png files.
I already tryed to use pngquant, but their docs wasnt too clear for me, and i cant find good code examples.

PNG is lossless. You cannot choose to discard information when writing in order to make files smaller like you can with JPEG.
If you go for a palettised version, you only need one byte per pixel instead of three, but then you only get 256 colours and gradients will look rubbish.
Also, the quality setting is not the same as for JPEG - it is more like the --fast or --best parameter to gzip.
One thing you can do, if you have large areas of transparency like you do, is make black all pixels that are 100% transparent. That will help them compress better. See example here.

How to save jpeg data that is identical to the original jpeg file using OpenCV

I'm using OpenCV and Python. I have loaded a jpeg image into a numpy array. Now i want to save it back into jpeg format, but since the image was not modified, I don't want to compress it again. Is it possible to create a jpeg from the numpy array that is identical with the jpeg that it was loaded from?
I know this workflow (decode-encode without doing anything) sounds a bit stupid, but keeping the original jpeg data is not an option. I'm interested if it is possible to recreate the original jpeg just using the data at hand.
The question is different from Reading a .JPG Image and Saving it without file size change, as I don't modify anything in the picture. I really want to restore the original jpeg file based on the data at hand. I assume one could bypass the compression steps (the compression artifacts are already in the data) and just write the file in jpeg format. The question is, if this is possible with OpenCV.

Clarified answer, following comment below:
What you say makes no sense at all; You say that you have the raw, unmodified, RGB data. No you don't. You have the uncompressed data that has been reconstructed from the compressed jpeg file.
The JPEG standards specify how to un-compress an image / video. There is nothing in the standard about how to actually do this compression, so your original image data could have been compressed any one of a zillion different ways. You have no way of knowing the decoding steps that were required to recreate your data, so you cannot reverse them.
Image this.
"I have a number, 44, please tell me how I can get the original
numbers that this came from"
This is, essentially, what you are asking.
The only way you can do what you want (other than just copy the original file) is to read the image into an array before loading into openCV. Then if you want to save it, then just write the raw array to a file, something like this:
fi = 'C:\\Path\\to\\Image.jpg'
fo = 'C:\\Path\\to\\Copy_Image.jpg'
with open(fi,'rb') as myfile:
im_array = np.array(myfile.read())
# Do stuff here
image = cv2.imdecode(im_array)
# Do more stuff here
with open(fo,'wb') as myfile:
myfile.write(im_array)
Of course, it means you will have the data stored twice, effectively, in memory, but this seems to me to be your only option.
Sometimes, no matter how hard you want to do something, you have to accept that it just cannot be done.

Why the frames of a VideoClip change when it is written to a video file?

I wrote the following code:
from moviepy.editor import *
from PIL import Image
clip= VideoFileClip("video.mp4")
video= CompositeVideoClip([clip])
video.write_videofile("video_new.mp4",fps=clip.fps)
then to check whether the frames have changed or not and if changed, which function changed them, i retrieved the first frame of 'clip', 'video' and 'video_new.mp4' and compared them:
clip1= VideoFileClip("video_new.mp4")
img1= clip.get_frame(0)
img2= video.get_frame(0)
img3= clip1.get_frame(0)
a=img1[0,0,0]
b=img2[0,0,0]
c=img3[0,0,0]
I found that a=24, b=24, but c=26....infact on running a array compare loop i found that 'img1' and 'img2' were identical but 'img3' was different.
I suspect that the function video.write_videofile is responsible for the change in array. But i dont know why...Can anybody explain this to me and also suggest a way to write clips without changing their frames?
PS: i read the docs of 'VideoFileClip', 'FFMPEG_VideoWriter', 'FFMPEG_VideoReader' but could not find anything useful...I need to read the exact frame as it was before writing in a code I'm working on. Please, suggest me a way.

Like JPEG, MPEG-4 uses lossy compression, so it's not surprising that the frames read from "video_new.mp4" are not perfectly identical to those in "video.mp4". And as well as the variations caused purely by the lossy compression there are also variations that arise due to the wide variety of encoding options that can be used by programs that write MPEG data.
If you really need to be able to read back the exact same frame data that you write then you will have to use a different file format, but be warned: your files will be huge!
The choice of video format partly depends on what the image data is like and on what you want to do with it. If the data uses 256 colours or less, and you don't intend to perform transformations on it that will modify the colours, a simple GIF anim is a good choice. But bear in mind that even something like non-integer scaling modifies colours.
If you want to analyze the image data and transform it in various ways, it makes sense to use a format with better colour support than GIF, eg a stream of PNG images, which I assume is what Zulko mentions in his answer. FWIW, there's an anim format related to PNG called MNG, but it is not well supported or widely known.
Another option is to use a stream of PPM images, or maybe even a stream of YUV data, which is useful for certain kinds of analysis and convenient if you do intend to encode as MPEG for final consumption. The PPM format is very simple and easy to work with; YUV is slightly messy since it's a raw format with no header data, so you have to keep track of the image size and resolution data yourself.
The file size of PPM or YUV streams is large, since they incorporate no compression at all, but of course they can be compressed using standard compression techniques, if you want to save a little space when saving them to disk. OTOH, typical video processing workflows that use such streams often don't bother writing them to disk: they are sent in pipelines (perhaps using named pipes), so the file size is (mostly) irrelevant.
Although such formats take up a lot of space compared to MPEG-based files, they are far superior for use as intermediate formats while performing image data analysis and transformation, since every time you write & read back MPEG you are losing a little bit of quality.
I assume that you intend to do your image data analysis and transformations using PIL/Pillow. But you can also work with PPM & YUV streams using the ffmpeg / avconv command line programs; and the ffmpeg family happily work with sets of individual image files and GIF anims, too.

You can have lossless compression with the 'png' codec:
clip.write_videoclip('clip_new.avi', codec='png')
EDIT #PM 2Ring: when you write the line above, it makes a video that is compressed using the png algortihm (I'm not sure whether each frame is a png or if it's more subtle).

Best way of Getting Swf File Dimensions with Python

Edit:
I'm considering hexagonit, and swfTools. Does anyone have any other solutions, or insight?
Edit:
New Question - how to solve this error:
I tried using hexagonit.swfheader however I receive the error:
f = 'out/'+w+"/"+s+"/"+"theSwf/"+s
data = hexagonit.swfheader.parse(f)
File "/Library/Python/2.7/site-packages/hexagonit/swfheader/__init__.py", line 26, in parse
signature = ''.join(struct.unpack('<3c', input.read(3)))
struct.error: unpack requires a string argument of length 3
After tracing through this I found that the error occurs here:
def parse(input):
"""Parses the header information from an SWF file."""
need_close=False
if hasattr(input, 'read'):
input.seek(0)
else:
input = open(input, 'rb')*
need_close=True
* being where the error occurs.
I read: Get dimensions from a flash file (.swf) in pure-Python
I have also tried using PIL and Pillow however I am under the impression that they compare images not swf files. I decompiled the swf file's I'm looking at along but also have the swf file itself.
I would like to know what size the file displays as (dimensions).
My first thought was to try using image size comparison.
My issue with this is that some images that are used as assets in the swf are actually larger than the swf itself, otherwise I would use PIL to simply get the dimensions of the largest image asset (ex the background).
Secondly my other issue is that can equally compare svg and png files.. and Pillow and Pil to my knowledge do not handle svg files.
My second idea was to search the actionscript code for the dimensions.
Some files have in their action script something like 300x300 which denotes the size. Unfortunately after looking at most of the files I am working with do not which means this is largely unhelpful.
My 3rd idea was to ignore the decompiled swf data and rather focus on the swf itself.
I could in theory either try to find the dimensions in the byte code (or use a library that does this (which I need to find one as pip and pillow do not appear to work)) or I need to run the ad and then screenshot it and try to find where the ad starts and stops and calculate the pixels based on that. My problem with screens shotting it is that the image may blend into the background and make it hard if not impossible to get the correct dimensions, but more importantly many swfs cannot be played due to security if they are not played in the right url, etc.
Thus I'm left with a dilemma. I think the best way to go about this would be to use the swf file itself.

Take a look at the official SWF file format spec. The dimension info you are looking for should be right near the beginning of the file. Take a look at the section "The SWF header"
The FrameSize field defines the width and height of the on-screen display. This field is stored as a RECT
structure, meaning that its size may vary according to the number of bits needed to encode the coordinates. The
FrameSize RECT always has Xmin and Ymin value of 0; the Xmax and Ymax members define the width and height
(see Using bit values).

Extracting bitmap from a file

given a somewhat complex file of unknown specification that among other things contains an uncompressed bitmap file (.BMP), how would you extract it in Python?
Scan for the "BM" tag and see if the following bytes "resemble" a BMP header?

I'd use the Python Imaging Library PIL and have it a go at the data. If it can parse it, then it's a valid image. When it throws an exception, then it isn't.
You need to search for the begining of the image; if you're lucky, the image reader will ignore garbage after the image data. When it doesn't, use a binary search to locate the end of the image.

Yes, about the only thing you can do is search through the file for the 'BM' marker, pull out the following data into a BITMAPFILEHEADER and corresponding BITMAPINFO, and see if the values in it look valid (i.e. that the dimensions are sensible, colour depth is reasonable, etc).
Once you have found something that looks reasonable, pull that data out and pass it to the library mentioned in another answer.