Develop Reference

fitz.open() not working when in a for loop (FITZ,PYTHON,PYMUPDF)

I am getting stuck when trying to iterate through files in a directory ('PDFS') with fitz from PyMuPDF.
The thing is, the code works when I am just doing document = "somepdf.pdf", but as soon as I insert a for loop and try to access files that way this error shows up:
filename, stream, filetype, rect, width, height, fontsize
RuntimeError: cannot open sample.pdf: No such file or directory
Here is the code:
for file in os.listdir('PDFS'):
if fnmatch.fnmatch(file, '*.pdf'):
document = file
doc = fitz.open(document)
Thank you for the help!

Your pdf files to open is under sub-directory PDFS, e.g. PDFS/sample.pdf, while your code fitz.open(document) is to open file under current working directory. So, a fix should be:
import fitz
import os
import fnmatch
for file in os.listdir('PDFS'):
if fnmatch.fnmatch(file, '*.pdf'):
document = os.path.join('PDFS', file)
doc = fitz.open(document)
Furthermore, a relative path PDFS is used, so you have to run the code right under the path where PDFS in, say /your/workspace/:
/your/workspace > python code.py
Otherwise,
/your > python workspace/code.py
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'PDFS'
So, a good practice is to
use full path if PDFS is just a user input path; otherwise,
use relative path to the script path
script_path = os.path.abspath(__file__)
project_path = os.path.dirname(script_path)
pdfs_path = os.path.join(project_path, 'PDFS')

I had the same issue. Its because PyCharm automatically installs the module named fitz whereas you need to write the following in the terminal:
pip install PyMuPDF
This will automatically install fitz and you can use the function fitz.open()

python yaml path after deployment

So this is a question about how to handle settings files and relative paths in python (probably also something about best practice).
So I have coded a smaller project that i want to deploy to a docker image and everything is set up now except when I try to run the python task (Through cron) I get the error: settings/settings.yml not found.
tree .
├───settings
│ └───settings/settings.yml
└───main.py
And am referencing the yml file as
open('settings/settings.yml', 'r') as f:
config = yaml.load(f, Loader=yaml.FullLoader)
I can see this is what is causing the problem but am unsure about how to fix it. I wish to reference the main file basically by using the entry_points from setuptools in the future so my quick fix with cd'ing before python main.py will not be a lasting solution.

Instead of hardcoding a path as a string, you can find the directories and build the file path with os.path. For example:
import os
import yaml
current_dir = os.path.dirname(os.path.abspath(__file__))
settings_dir = os.path.join(current_dir, "settings")
filename = "settings.yml"
settings_path = os.path.join(settings_dir, filename)
with open(settings_path, "r") as infile:
settings_data = yaml.load(infile)
This way it can be run in any file system and the python file can be called from any directory.

Real path to config.ini in python binary *app in MacOS

I have this code:
def _read_config(self):
config = configparser.ConfigParser()
config.sections()
# I tried
path_main = os.path.dirname(os.path.realpath(__file__))
# and this after create exec file with pyinstaller nothing changed
path_main = os.getcwd()
print(path_main)
file = os.path.join(path_main, "config.ini")
print(file)
config.read(file)
return config
When I run the code in MacOS using the terminal with python gui.py, it prints this:
/Users/telos/Desktop/Telos-Monitor-Tool/client
/Users/telos/Desktop/Telos-Monitor-Tool/client/config.ini
But when I do pyinstaller --onefile --windowed gui.py, I receive 1 app file, when I run it I get this:
/Users/telos
/Users/telos/config.ini
But the one file app and ``gui.py` is in the same directory.
So I have an error because the Python parser can't find config.ini.
As in comennt discasion advise me to use print(QtCore.QCoreApplication.applicationDirPath()) after recreating app, i have 2 file 1 gui.app, 2-nd gui.exec. gui.exec find config.ini fine and all work fine, but gui.app can't and send the error.
Any idea what is the problem?

Since you are using PyQt5 if you want to get the executable folder you can use:
QtCore.QCoreApplication.applicationDirPath()

py2exe not recognizing jsonschema

I've been attempting to build a Windows executable with py2exe for a Python program that uses the jsonschema package, but every time I try to run the executable it fails with the following error:
File "jsonschema\__init__.pyc", line 18, in <module>
File "jsonschema\validators.pyc", line 163, in <module>
File "jsonschema\_utils.pyc", line 57, in load_schema
File "pkgutil.pyc", line 591, in get_data
IOError: [Errno 0] Error: 'jsonschema\\schemas\\draft3.json'
I've tried adding json and jsonschema to the package options for py2exe in setup.py and I also tried manually copying the jsonschema directory from its location in Python27\Libs\site-packages into library.zip, but neither of those work. I also attempted to use the solution found here (http://crazedmonkey.com/blog/python/pkg_resources-with-py2exe.html) that suggests extending py2exe to be able to copy files into the zip file, but that did not seem to work either.
I'm assuming this happens because py2exe only includes Python files in the library.zip, but I was wondering if there is any way for this to work without having to convert draft3.json and draft4.json into .py files in their original location.
Thank you in advance

Well after some more googling (I hate ugly) I got it working without patching the build_exe.py file. The key to the whole thing was the recipe at http://crazedmonkey.com/blog/python/pkg_resources-with-py2exe.html. My collector class looks like this:
from py2exe.build_exe import py2exe as build_exe
class JsonSchemaCollector(build_exe):
"""
This class Adds jsonschema files draft3.json and draft4.json to
the list of compiled files so it will be included in the zipfile.
"""
def copy_extensions(self, extensions):
build_exe.copy_extensions(self, extensions)
# Define the data path where the files reside.
data_path = os.path.join(jsonschema.__path__[0], 'schemas')
# Create the subdir where the json files are collected.
media = os.path.join('jsonschema', 'schemas')
full = os.path.join(self.collect_dir, media)
self.mkpath(full)
# Copy the json files to the collection dir. Also add the copied file
# to the list of compiled files so it will be included in the zipfile.
for name in os.listdir(data_path):
file_name = os.path.join(data_path, name)
self.copy_file(file_name, os.path.join(full, name))
self.compiled_files.append(os.path.join(media, name))
What's left is to add it to the core setup like this:
options = {"bundle_files": 1, # Bundle ALL files inside the EXE
"compressed": 2, # compress the library archive
"optimize": 2, # like python -OO
"packages": packages, # Packages needed by lxml.
"excludes": excludes, # COM stuff we don't want
"dll_excludes": skip} # Exclude unused DLLs
distutils.core.setup(
cmdclass={"py2exe": JsonSchemaCollector},
options={"py2exe": options},
zipfile=None,
console=[prog])
Some of the code is omitted since it's not relevant in this context but I think you get the drift.

Include .pyd module files in py2exe compilation

I'm trying to compile a python script. On executing the exe I got:-
C:\Python27\dist>visualn.exe
Traceback (most recent call last):
File "visualn.py", line 19, in <module>
File "MMTK\__init__.pyc", line 39, in <module>
File "Scientific\Geometry\__init__.pyc", line 30, in <module>
File "Scientific\Geometry\VectorModule.pyc", line 9, in <module>
File "Scientific\N.pyc", line 1, in <module>
ImportError: No module named Scientific_numerics_package_id
I can see the file Scientific_numerics_package_id.pyd at the location "C:\Python27\Lib\site-packages\Scientific\win32". I want to include this module file into the compilation. I tried to copy the above file in the "dist" folder but no good. Any idea?
Update:
Here is the script:
from MMTK import *
from MMTK.Proteins import Protein
from Scientific.Visualization import VRML2; visualization_module = VRML2
protein = Protein('3CLN.pdb')
center, inertia = protein.centerAndMomentOfInertia()
distance_away = 8.0
front_cam = visualization_module.Camera(position= [center[0],center[1],center[2]+distance_away],description="Front")
right_cam = visualization_module.Camera(position=[center[0]+distance_away,center[1],center[2]],orientation=(Vector(0, 1, 0),3.14159*0.5),description="Right")
back_cam = visualization_module.Camera(position=[center[0],center[1],center[2]-distance_away],orientation=(Vector(0, 1, 0),3.14159),description="Back")
left_cam = visualization_module.Camera(position=[center[0]-distance_away,center[1],center[2]],orientation=(Vector(0, 1, 0),3.14159*1.5),description="Left")
model_name = 'vdw'
graphics = protein.graphicsObjects(graphics_module = visualization_module,model=model_name)
visualization_module.Scene(graphics, cameras=[front_cam,right_cam,back_cam,left_cam]).view()

Py2exe lets you specify additional Python modules (both .py and .pyd) via the includes option:
setup(
...
options={"py2exe": {"includes": ["Scientific.win32.Scientific_numerics_package_id"]}}
)
EDIT. The above should work if Python is able to
import Scientific.win32.Scientific_numerics_package_id

There is a way to work around this types of issues that I have used a number of times. In order to add extra files to the py2exe result you can extend the media collector in order to have a custom version of it. The following code is an example:
import glob
from py2exe.build_exe import py2exe as build_exe
def get_py2exe_extension():
"""Return an extension class of py2exe."""
class MediaCollector(build_exe):
"""Extension that copies Scientific_numerics_package_id missing data."""
def _add_module_data(self, module_name):
"""Add the data from a given path."""
# Create the media subdir where the
# Python files are collected.
media = module_name.replace('.', os.path.sep)
full = os.path.join(self.collect_dir, media)
if not os.path.exists(full):
self.mkpath(full)
# Copy the media files to the collection dir.
# Also add the copied file to the list of compiled
# files so it will be included in zipfile.
module = __import__(module_name, None, None, [''])
for path in module.__path__:
for f in glob.glob(path + '/*'): # does not like os.path.sep
log.info('Copying file %s', f)
name = os.path.basename(f)
if not os.path.isdir(f):
self.copy_file(f, os.path.join(full, name))
self.compiled_files.append(os.path.join(media, name))
else:
self.copy_tree(f, os.path.join(full, name))
def copy_extensions(self, extensions):
"""Copy the missing extensions."""
build_exe.copy_extensions(self, extensions)
for module in ['Scientific_numerics_package_id',]:
self._add_module_data(module)
return MediaCollector
I'm not sure which is the Scientific_numerics_package_id module so I've assumed that you can import it like that. The copy extensions method will get a the different module names that you are having problems with and will copy all their data into the dir folder for you. Once you have that, in order to use the new Media collector you just have to do something like the following:
cmdclass['py2exe'] = get_py2exe_extension()
So that the correct extension is used. You might need to touch the code a little but this should be a good starting point for what you need.

I encountered similar probelm with py2exe and the only solution I can find ,is to use another tool to convert python to exe - pyinstaller
Its very easy tool to use and more important , it works!
UPDATE
As I understood from your comments below , running your script from command line is not working also , due to import error (My recommendation is to first check your code from command line ,and than try to convert it to EXE)
It looks like PYTHONPATH problem.
PYTHONPATH is list of paths (similar of Windows PATH) that python programs use to find import modules.
If your script run from your IDE , that means the PYTHONPATH is set correctly in the IDE ,so all imported modules are found.
In order to set PYTHONPATH you can use :
import sys|
sys.path.append(pathname)
or use the following code that add the all folders under path parameter to PYTHONPATH:
import os
import sys
def add_tree_to_pythonpath(path):
"""
Function: add_tree_to_pythonpath
Description: Go over each directory in path and add it to PYTHONPATH
Parameters: path - Parent path to start from
Return: None
"""
# Go over each directory and file in path
for f in os.listdir(path):
if f == ".bzr" or f.lower() == "dll":
# Ignore bzr and dll directories (optional to NOT include specific folders)
continue
pathname = os.path.join(path, f)
if os.path.isdir(pathname) == True:
# Add path to PYTHONPATH
sys.path.append(pathname)
# It is a directory, recurse into it
add_tree_to_pythonpath(pathname)
else:
continue
def startup():
"""
Function: startup
Description: Startup actions needed before call to main function
Parameters: None
Return: None
"""
parent_path = os.path.normpath(os.path.join(os.getcwd(), ".."))
parent_path = os.path.normpath(os.path.join(parent_path, ".."))
# Go over each directory in parent_path and add it to PYTHONPATH
add_tree_to_pythonpath(parent_path)
# Start the program
main()
startup()

The ImportError is rectified by using "Gil.I" and "Janne Karila" suggestion by setting pythonpath and by using include function. But before this I had to create __init__.py file in the win32 folder of both the modules.
BTW I still got another error for the above script - link