Develop Reference

Python Exiftool get metadata from livestream

I am trying to read geotagging data from a live stream online, here is my code:
import exiftool
def getVideo(url):
with exiftool.ExifToolHelper() as et:
metadata = et.getmetadata(url)
print(metadata)
getVideo("url/to/stream")
however, I got this error:
Traceback (most recent call last):
File "C:\Users\alexa\Documents\vtest2.py", line 9, in <module>
getVideo("url/to/stream")
File "C:\Users\alexa\Documents\vtest2.py", line 4, in getVideo
with exiftool.ExifToolHelper() as et:
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\exiftool\helper.py", line 101, in __init__
super().__init__(**kwargs)
File "C:\Python311\Lib\site-packages\exiftool\exiftool.py", line 300, in __init__
self.executable = executable or constants.DEFAULT_EXECUTABLE
^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\exiftool\exiftool.py", line 374, in executable
raise FileNotFoundError(f'"{new_executable}" is not found, on path or as absolute path')
FileNotFoundError: "exiftool.exe" is not found, on path or as absolute path
is there a better way to read metadata from a live stream?

Python tempfile.TemporaryDirectory() cleanup crashes with PermissionError and NotADirectoryError

Premise
I'm trying to convert some PDF to images via pdf2image and poppler, to then run some computervision tasks on.
The conversion itsself works fine.
However, the conversion creates some artifacts for each page in the pdf as it is being converted, which I would like to be deleted at the end of the function. To facilitate this, I am using tempfile.TemporaryDirectory(). The function looks as follow:
with tempfile.TemporaryDirectory() as path:
images_from_path: [Image] = convert_from_path(
os.path.join(path_superfolder, "calibration_target.pdf"),
size=(2480, 3508),
output_folder=path, poppler_path=r'E:\poppler-22.04.0\Library\bin')
if len(images_from_path) >= page:
images_from_path[page - 1].save(os.path.join(path_superfolder, "result.jpg"))
Problem
The trouble is, that the program always crashes with the following errors, after transforming the PDF and writing the required image to a file.
Traceback (most recent call last):
File "C:\Python310\lib\shutil.py", line 617, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file, because it is being used by another process: 'C:\\Users\\tobia\\AppData\\Local\\Temp\\tmp24c4bmzv\\bd76d834-672e-49fc-ac30-7751b7b660d0-01.ppm'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python310\lib\tempfile.py", line 843, in onerror
_os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file, because it is being used by another process: 'C:\\Users\\tobia\\AppData\\Local\\Temp\\tmp24c4bmzv\\bd76d834-672e-49fc-ac30-7751b7b660d0-01.ppm'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python310\lib\code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 1, in <module>
File "E:\PyCharm 2022.2.3\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 198, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "E:\PyCharm 2022.2.3\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "D:\Dokumente\Uni\Informatik\BA_Thesis\tumexam-scheduling-codebase\generate_data.py", line 393, in <module>
extract_calibration_page_as_image_from_pdf()
File "D:\Dokumente\Uni\Informatik\BA_Thesis\tumexam-scheduling-codebase\generate_data.py", line 190, in extract_calibration_page_as_image_from_pdf
tmp_dir.cleanup()
File "C:\Python310\lib\tempfile.py", line 873, in cleanup
self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
File "C:\Python310\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Python310\lib\shutil.py", line 749, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Python310\lib\shutil.py", line 619, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Python310\lib\tempfile.py", line 846, in onerror
cls._rmtree(path, ignore_errors=ignore_errors)
File "C:\Python310\lib\tempfile.py", line 855, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Python310\lib\shutil.py", line 749, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Python310\lib\shutil.py", line 600, in _rmtree_unsafe
onerror(os.scandir, path, sys.exc_info())
File "C:\Python310\lib\shutil.py", line 597, in _rmtree_unsafe
with os.scandir(path) as scandir_it:
NotADirectoryError: [WinError 267] Directory name invalid: 'C:\\Users\\tobia\\AppData\\Local\\Temp\\tmp24c4bmzv\\bd76d834-672e-49fc-ac30-7751b7b660d0-01.ppm'
When stepping through the cleanup routine, everything seems fine, the path is correct and it starts deleting files, until at some point the internal path variable gets jumbled up and the routine crashes, because obviously a file is not a directory. To me it seems like a race condition is causing problems here.
What I have already tried
Rewriting the function to not use with and instead explicitly call the routine with tmp_dir.cleanup()
Just creating the directory without populating it with the conversion artifacts. The cleanup works in this case.
The documentation for tempfile mentions Permission errors occuring when files are open. The files are however only used in this function and if this is what is causing the error, I am unsure where the files are still opened or which function is causing this. My suspicion of course would be the conversion function.

While experimenting some more and writing this question, I found a working solution:
with tempfile.TemporaryDirectory() as path:
images_from_path: [Image] = convert_from_path(
os.path.join(path_superfolder, f"calibration_target_{exam_type}.pdf"),
size=(2480, 3508),
output_folder=path, poppler_path=r'E:\poppler-22.04.0\Library\bin')
if len(images_from_path) >= page:
images_from_path[page - 1].save(os.path.join(path_superfolder, "result.jpg"))
images_from_path = []
It seems that somehow, the routine had trouble cleaning up, because the converted images, are actually the artifacts created by pdf2image and were still being held by my data structure. Resetting the data structure, before implicitly initiating the cleanup fixed the issue.
If there is a better way of tackling this issue, please do not hesitate to inform me.

IO Error python PIL image preprocessing script

I am following this tutorial and specifically going through the "generate own data" section:
https://github.com/surfertas/deep_learning/tree/master/projects/imdbwiki-challenge
https://github.com/surfertas/deep_learning/blob/master/projects/imdbwiki-challenge/imdb_preprocess.py
and i am facing this issue running the imdb_preprocess.py script;
Dictionary created...
Converting 1000 samples. (0=all samples)
Traceback (most recent call last):
File "imdb_preprocess.py", line 137, in <module>
main()
File "imdb_preprocess.py", line 131, in main
create_and_dump(imdb_dict, args.partial)
File "imdb_preprocess.py", line 106, in create_and_dump
for img_path in imgs
File "/usr/lib64/python2.7/site-packages/scipy/misc/pilutil.py", line 156, in imread
im = Image.open(name)
File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 2477, in open
fp = builtins.open(filename, "rb")
IOError: [Errno 2] No such file or directory: u'/path/48/10000548_1925-04-04_1964.jpg'
Now i manually checked folder 48 and checked that the image is complaining about is indeed there.
Any hints on where the fault is?
path was replaced

Python : Copying directory tree with symbolic links in it

In Python I am trying to copy a directory (actually, its the Jenkins jobs directory), but it contains symbolic links in it, when I copy I get:
Traceback (most recent call last):
File "BackupJenkinsJobs.py", line 272, in <module>
main()
File "BackupJenkinsJobs.py", line 208, in main
distutils.dir_util.copy_tree(JenkinsJobSrc, cleanJobsDir, preserve_symlinks=False)
File "c:\Python27\lib\distutils\dir_util.py", line 163, in copy_tree
verbose=verbose, dry_run=dry_run))
File "c:\Python27\lib\distutils\dir_util.py", line 163, in copy_tree
verbose=verbose, dry_run=dry_run))
File "c:\Python27\lib\distutils\dir_util.py", line 167, in copy_tree
dry_run=dry_run)
File "c:\Python27\lib\distutils\file_util.py", line 148, in copy_file
_copy_file_contents(src, dst)
File "c:\Python27\lib\distutils\file_util.py", line 32, in _copy_file_contents
fsrc = open(src, 'rb')
IOError: [Errno 22] invalid mode ('rb') or filename: 'C:\\Program Files (x86)\\Jenkins\\jobs\\AutoRunTemplate\\builds\\lastFailedBuild'
I am using the following code:
try:
distutils.dir_util.copy_tree(JenkinsJobSrc, cleanJobsDir, preserve_symlinks=False)
except distutils.errors.DistutilsFileError as e:
print("Unable to copy Jenkins jobs. Error: %s".format(e))
return
Any assistance would be appreciated on how to copy, ignoring the links, as preserve_symlinks doesn't appear to work.

cx_freeze compile with pygame and data folder

I'd wish to compile my program wrote in Python 3.2 with Pygame 1.9.5. But I'm a beginner with cx_Freeze and I've no idea how to manage with the complex structure... I mean :
My folder is like that :
gamefolder:
lib:
Menu.py (lauch the program)
level.py
othermodules.py
...
data:
Level1.png
otherspictures.png
So I tried to compile but I always get an error when I lauch Main.exe, indeed I think pygame is not load and also the path are not really clear.
I hope there's a solution, and that with such a structure it will not be impossible because I'd rather not rewrote each path in the scripts ...
Further informations :
the setup.py script is in lib folder and looks like that:
from cx_Freeze import setup, Executable
setup(
name = "Resets",
version = "1.0",
description = "Jeu de plateforme - Projet ISN 2014",
executables = [Executable("Menu.py")]
)
And i get this error when I launch the Menu.exe :
Traceback (most recent call last):
File "C:\Python32\lib\site-packages\pygame\__init__.py", line 117, in <module>
try: import pygame.display
File "ExtensionLoader_pygame_display.py", line 22, in <module>
File "ExtensionLoader_pygame_display.py", line 14, in __bootstrap__
File "ExtensionLoader_pygame_surface.py", line 22, in <module>
File "ExtensionLoader_pygame_surface.py", line 14, in __bootstrap__
ImportError: No module named _view
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python32\lib\site-packages\cx_Freeze\initscripts\Console3.py", line 2
7, in <module>
exec(code, m.__dict__)
File "Menu.py", line 8, in <module>
File "C:\Python32\lib\site-packages\pygame\__init__.py", line 118, in <module>
except (ImportError,IOError):display=MissingModule("display", geterror(), 1)
File "C:\Python32\lib\site-packages\pygame\__init__.py", line 61, in __init__
self.warn()
File "C:\Python32\lib\site-packages\pygame\__init__.py", line 85, in warn
warnings.warn(message, RuntimeWarning, level)
File "C:\Python\32-bit\3.2\lib\warnings.py", line 18, in showwarning
File "C:\Python\32-bit\3.2\lib\warnings.py", line 25, in formatwarning
File "C:\Python\32-bit\3.2\lib\linecache.py", line 15, in getline
File "C:\Python\32-bit\3.2\lib\linecache.py", line 41, in getlines
File "C:\Python\32-bit\3.2\lib\linecache.py", line 127, in updatecache
File "C:\Python\32-bit\3.2\lib\codecs.py", line 300, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x90 in position 2: invalid
start byte
I already try with import pygame._view and import re but it doesn't work too...

Finally I succeeded, it was very simple !
I added import pygame._view in each script. And I copied the data folder into the folder (after being compiled).