Develop Reference

ImageMagick is splitting the NASA's [.IMG] file by a black line upon converting to JPG

I have some raw .IMG format files which I'm converting to .jpg using ImageMagick to apply a CNN Classifier. The converted images, however have a black vertical line splitting the image into two. The part on the left side of the line should have actually been on the right side of the right part of the image. I've posted a sample image:
I used the command magick convert input_filename.IMG output_filename.jpg
Raw .IMG File
Here is how the image is supposed to look (converted manually using numpy):
How the image is actually looking (with the vertical black line using ImageMagick):
Version Details:
harshitjindal#Harshits-MacBook-Pro ~ % magick identify -version
Version: ImageMagick 7.0.10-0 Q16 x86_64 2020-03-08
https://imagemagick.org Copyright: © 1999-2020 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php Features: Cipher DPC
HDRI Modules OpenMP(3.1) Delegates (built-in): bzlib freetype heic jng
jp2 jpeg lcms ltdl lzma openexr png tiff webp xml zlib

I don't know why ImageMagick is failing to interpret the file correctly, but I can show you how to make it work.
You need to search in your file for the height, width and data type of your image, you can do that like this:
grep -E "LINES|LINE_SAMPLES|BITS" EW0220149939G.IMG
LINES = 1024
LINE_SAMPLES = 1024
SAMPLE_BITS = 8
That means your image is 1024x1024 and 8 bits/sample (1 byte). Then you need to take that number of bytes from the tail end of the file and feed them into ImageMagick. So, you need the final 1024x1024 bytes which you can get with tail or gtail (GNU tail) as you are on a Mac.
gtail -c $((1024*1024*1)) EW0220149939G.IMG | convert -depth 8 -size 1024x1024 gray:- result.jpg
If your image is 16-bit, like in your other question, you need to use:
gtail -c $((1024*1024*2)) 16-BIT-IMAGE.IMG | convert -depth 16 -size 1024x1024 gray:- result.jpg
If you dislike using gtail to get the last megabyte, you can alternatively specify an offset from the start of the file that tells ImageMagick where the pixel data starts. So, first you need the size of the header:
grep -E "RECORD_BYTES|LABEL_RECORDS" EW*IMG
RECORD_BYTES = 1024
LABEL_RECORDS = 0007
That means we need to skip 1024*7 bytes to get to the image, so the command is:
convert -size 1024x1024+$((1024*7)) -depth 8 gray:EW0220149939G.IMG result.jpg

Replace transparency with whitebackground in BigTiff

I am working with python to convert an RGBA tiff to an RGB tiff with a white background.
I am using the library ImageMagick and GDAL.
My code looks like that:
def add_background_to_rgba_geotiff(source, destination):
convert_rgba_to_rgb_tif(source, destination)
add_metadata_to_new_geotiff_file(source, destination)
def convert_rgba_to_rgb_tif(source, destination):
# work also with BigTiff
command = ' '.join(['convert', quote(source),
'-background', 'white',
'-alpha', 'background',
'-alpha', 'off', quote(destination)])
shell_command.execute_and_log_outputs(command, shell=True)
def add_metadata_to_new_geotiff_file(source, destination):
RGBA_tif = gdal.Open(source, gdalconst.GA_ReadOnly)
RGB_tif = gdal.Open(destination, gdalconst.GA_Update)
RGB_tif.SetMetadata(RGBA_tif.GetMetadata())
RGB_tif.SetGeoTransform(RGBA_tif.GetGeoTransform())
RGB_tif.SetProjection(RGBA_tif.GetProjection())
del (RGBA_tif)
del (RGB_tif)
def execute_and_log_outputs(command, silence_errors=False, **kwargs):
shell_process = execute_async(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
**kwargs)
(out, err) = shell_process.communicate()
When I try to run my code with a tiff everything happens well but when I want to apply that on a BigTIFF (>4GB) it fails with this error:
TIFFWriteDirectoryTagData: Maximum TIFF file size exceeded.
Does someone know how to use BigTiff with ImageMagick?
Or could it be possible to do that with GDAL? For the moment when I try with GDAL, I only achieved to have a black background.
Thanks for your help.

If you check the available formats that ImageMagick knows, you will see a TIFF64:
identify -list format | grep -i tiff
EPT EPT rw- Encapsulated PostScript with TIFF preview
EPT2 EPT rw- Encapsulated PostScript Level II with TIFF preview
EPT3 EPT rw+ Encapsulated PostScript Level III with TIFF preview
GROUP4* TIFF rw- Raw CCITT Group4
PTIF* TIFF rw+ Pyramid encoded TIFF
TIFF* TIFF rw+ Tagged Image File Format (LIBTIFF, Version 4.0.9)
TIFF64* TIFF rw- Tagged Image File Format (64-bit) (LIBTIFF, Version 4.0.9)
If you try to make a TIFF like this it fails:
convert -size 20000x50000 xc:red +noise random a.tif
convert: Maximum TIFF file size exceeded. `TIFFAppendToStrip' # error/tiff.c/TIFFErrors/652.
So, force the TIFF64 delegate by prepending it to the output filename to get a BigTIFF like this:
convert -size 20000x50000 xc:red +noise random TIFF64:a.tif
and it works.

How to convert raw Y video buffer to image using python?

I have a raw file of Y video plane (y.raw) extracted from raw video frame format YUV (YUV422_SEMI_PLANAR).How to convert this y.raw as image png or jpg ?

You have essentially got a single channel greyscale image with no packing or interleaving. So you have a few options:
ImageMagick at the command line,
via NetPBM PGM format and Photoshop, or GIMP
Python
ImageMagick at command line in Terminal
If you have ImageMagick installed like most Linux distros already do (or you can install it for free on Mac and Windows) you could convert it to like this in Terminal.
Say it is 640x480 pixels and 8-bit and you want a PNG:
convert -depth 8 -size 640x480 gray:y.raw result.png
Say it is 1024x768 pixels and 16 bit and you want a contrast-stretched JPG:
convert -depth 16 -size 1024x768 gray:y.raw -auto-level result.jpg
Via NetPBM PGM format and Photoshop or GIMP
Say you don't have ImageMagick, you could make the file into a NetPBM PGM (Portable Grey Map) that you can view/edit/save in GIMP, Paint, feh, Adobe Photoshop:
{ printf "P5\n640 480\n255\n"; cat y.raw; } > result.pgm
If you are unfortunate enough to be on Windows and want to do it this way, I think it would look something like this:
echo "P5" > header.txt
echo "640 480" >> header.txt
echo "255" >> header.txt
copy /b header.txt+y.raw result.ppm
Python
If you really, really want to write a bunch of Python, it might look like this:
#!/usr/local/bin/python3
from PIL import Image
file=open("y.raw",'rb')
rawdata=file.read()
file.close()
imgsize = (640,480)
# Use the PIL raw decoder
img = Image.frombytes('L',imgsize,rawdata)
img.save('result.png')
If anyone else is reading this and wishes they had a frame of Y data to play around with, you can easily create a simulated one containing a black-white gradient with ImageMagick like this:
convert -size 640x480 gradient: -depth 8 gray:y.raw
That will look like this with ls:
-rw-r--r-- 1 mark staff 307200 23 Mar 10:05 y.raw

Generate PDF from JPEGs in Separate Directories Grouped by Date/Name

I am looking to generate a PDF report from JPEGs on a server. The JPEGs are in folders named after the location they were taken and the JPEGs are named based on the date they were taken (...\Location 1\15 08 03 description.jpg). Basically I need to grab all pictures taken at each site last month, group them evenly on a page (4 max/page), label the pages with location and date, export PDF.
I have written projects in Powershell and Python so it would be a lot easier for me to operate in these languages but I will consider all suggestions.
So far, my idea is to use switch/case to select the various folder names, for loop through to select all cases, and select all files with .jpg extension within a month range (maybe user prompted?). Where I fall flat is arranging the JPEGs into a PDF as I described.
Edit: So if you follow Mark Setchell's advice below, create the images he suggested and place them in C:\New folder. So suppose you had in this sub directory 3 folders (New folder, New folder (2), etc.) and 2 of these contain the nine colored JPEGS and the third is empty:
clear
$path="C:\New folder\"
$array=#()
$name="file*.jpg"
foreach ($i in Get-ChildItem -path $path -Filter "New*")
{$i0=$path+$i; Get-ChildItem -path $i0 -Filter $name | ForEach-Object {$array+= $i0+"\"+$_.name}
montage $array -tile 2x2 -geometry +5+5 -title $i -page letter montage.pdf}`
My code overwrites the title on all pages with that of the 3rd empty folder. Also, it begins adding JPEGS from the next folder into the previous page, which should be titled as the previous folder and only contain those JPEGS.

Imagine you have 9 JPEG files in a directory, called file1.jpg...file9.jpg and they were created like this as lumps of red, green, blue, cyan, magenta, yellow, black and gray.
convert -size 300x400 xc:red file1.jpg
convert -size 300x400 xc:lime file2.jpg
convert -size 300x400 xc:blue file3.jpg
convert -size 300x400 xc:cyan file4.jpg
convert -size 300x400 xc:magenta file5.jpg
convert -size 300x400 xc:yellow file6.jpg
convert -size 300x400 xc:black file7.jpg
convert -size 300x400 xc:gray40 file8.jpg
convert -size 300x400 xc:gray80 file9.jpg
If you now go into that directory and run the following bash script, it will montage the files into pages of A4 with 4 images on each page.
#!/bin/bash
for f in file*jpg; do
convert -label "$f" "$f" -depth 8 miff:-
done | montage -tile 2x2 -geometry +5+5 miff:- -page A4 montage.pdf
The crux of the matter is firstly adding a label to each image based on the filename and secondly sending the label and the image to a MIFF file which is capable of holding many images. The combined group of images are then fed into montage whch arranges them four to a page because of the -tile 2x2. The geometry sets the spacing between the pictures - bigger numbers mean bigger spaces. Finally we tell montage that the paper size is A4 and we want a PDF of all the input images - please!
Of course you can diddle with the background, the sizing, the spacing and the labelling till you are happy - but this should give you the basic idea.
You will get out a PDF called montage.pdf with these three pages:
Page1
Page 2
Page 3
Its shoud be fairly trivial to convert the loop to an ugly Windows-y FOR loop - for loop help.

for converting your jpg's into pdf files you can use ImageMagick.
There are also different types of python APIs for imagemagick, but in your case is the best to write a simple powershell script and execute imagemagick directly.
Just use the following imagemagick command:
convert <yourfile.jpg> <newfile.pdf>
Note: You must use file extentions in your command. Otherwise imagemagick don't know what todo.

You could use glob to get the files (and then sort them if needed). I think the switch could get way too complicated.
As for the convert part: Shibumi's answer.

Stitch images together from html

This website has a large image combrised of 132 images, I want to find a way to stitch them together into a single image. I know some python if anyone has a clue where to start.
http://www.mapytatr.net/PRODUKTY/MAPY_TAT/WYSOKIE/SLICES/wys_ii.htm
Thanks!
Matt

Forget Python - use ImageMagic (http://www.imagemagick.org/)
+append to create row
convert tpn_1.jpg tpn_2.jpg tpn_3.jpg +append row_1.jpg
-append to create column
convert row_1.jpg row_2.jpg row_3.jpg -append final.jpg
You can try also montage (from ImageMagic too) to get all in one command
montage -mode concatenate -tile 3x3 tpn_* final.jpg
BTW: on Linux you can download all images using wget and for in bash
for i in $(seq 132); do echo "http://www.mapytatr.net/PRODUKTY/MAPY_TAT/WYSOKIE/SLICES/tpn_$i.jpg"; done | wget -i -
kochane tatry :)