Develop Reference

Python script to create m3u files inside folders

I'm trying to run this specific python script to create .m3u files inside subfolders but it's not working and give me an error, any help is welcome.
The m3u file is a simple tracklist of the subfolder content with specified extensions and named after the subfolder, like this (extensions .aaa and .bbb are just examples):
Folder 1 (contains 'File 1.aaa', 'File 2.aaa', etc)
Folder 1.m3u generated inside Folder 1 with this list
File 1.aaa
File 2.aaa
Folder 2 (contains 'File 1.bbb', 'File 2.bbb', etc)
Folder 2.m3u generated inside Folder 2 with this list
File 1.bbb
File 2.bbb
Here is the script called makem3u.py (not mine, I don't know much about python):
#!/usr/bin/python
"""This script will create an m3u file with a tracklist of each .aaa or .bbb
found in a subdirectory, then name the m3u after the subdirectory. This helps
with multiple disks for emulation.
"""
import os
import os.path
EXT = ['.aaa', '.bbb']
cwd = os.getcwd()
dirs = os.listdir(cwd)
for d in dirs:
contents = os.listdir(os.path.join(cwd, d))
disks = [
os.path.join(d, f) for f in os.listdir(os.path.join(cwd, d))
if os.path.splitext(f)[-1] in EXT
]
if disks:
with open(os.path.join(cwd, '{d}.m3u'.format(d=d)), 'wb') as m3u:
m3u.writelines(['{disk}\n'.format(disk=disk) for disk in disks])
I get this error when I try to run it:
Traceback (most recent call last):
File "makem3u.py", line 16, in <module>
contents = os.listdir(os.path.join(cwd, d))
NotADirectoryError: [WinError 267] The directory name is invalid: 'path\\to\\file\\makem3u.py'
makem3u.py is inside a folder with the subfolders mentioned
Windows 10, Python 3.8.5, python is installed properly and the PATH is in enviroment variables and I can run other scripts just fine
What I'm doing wrong and how can I fix this? Is there a non-python alternative like create a .bat file to do that? Sorry for so many questions, I'm a noob in these things. Thank you in advance!
Also, is there a way to batch zip all the files in the subfolders (the generated m3u + original files) and name each zip after that subfolder? This is an extra, but would be helpful if possible

setuptools does not distribute my data files

I have the following in setup.py:
from setuptools import setup
# ...
setup(
name='xml-boiler',
version='0.0.1',
url='https://github.com/vporton/xml-boiler',
license='AGPLv3',
author='Victor Porton',
author_email='porton#narod.ru',
description='Automatically transform between XML namespaces',
packages=find_packages(),
package_data={'': ['*.ttl', '*.xml']},
scripts=['bin/boiler'],
data_files = [
('/etc/xmlboiler', ['etc/config-cli.ttl'])
],
test_suite="xmlboiler.tests",
cmdclass={'build_py': MyBuild},
)
But after I run python setup.py build, the build directory does not contain any *.xml or *.ttl files.
What is my error?
I also want to distribute all files from xmlboiler/core/data/assets/ and
xmlboiler/core/data/assets/.
I don't understand how it works:
package_data={'': ['*/.xml', '*/.ttl', '*/.net', 'data/assets/*', 'data/scripts/*.xslt', 'xmlboiler/doc/*.html', 'xmlboiler/doc/*.css']},
included xmlboiler/core/data/scripts/section.xslt but not xmlboiler/tests/core/data/xml/simple.xml. Why?!

package_data is a mapping of package names to files or file globs. This means that
package_data = {'', ['*.xml', '*.ttl']}
will include every file ending with .xml or .ttl located in any package directory, for example xmlboiler/file.xml, xmlboiler/core/file.ttl etc. It will, however, not include file xmlboiler/core/data/interpreters.ttl because it is located in data which is not a package dir (not containing an __init__.py file). To include that, you should use the correct file path:
package_data = {'xmlboiler.core', ['data/interpreters.ttl']}
To include every .ttl file under xmlboiler/core/data:
package_data = {'xmlboiler.core', ['data/*.ttl', 'data/**/*.ttl']}
This will include every .ttl file in data directory (glob data/*.ttl) and every .ttl file in every subdirectory of data (glob data/**/*.ttl).
To include every .ttl and .xml file in every package:
package_data = {'', ['*.xml', '**/*.xml', '*.ttl', '**/*.ttl']}
I also want to distribute all files from xmlboiler/core/data/assets/
Same approach for data/assets, but omit the file extension in globs:
package_data={
'xmlboiler.core': ['data/assets/*', 'data/assets/**/*'],
}

Python's docx module giving AsserionError when .exe is created

I've a Python file titled my_python_file.py that makes, among other things, a .doc file using the python-docx module. The .doc is created perfectly and gives no problem. The problem comes when I build a .exe of my script and I try to make the .doc. An AssertionError problem appears.
This is my exe maker code (exe_maker.py):
from distutils.core import setup
import py2exe, sys, os
sys.argv.append('py2exe')
setup(
options = {'py2exe': {'bundle_files': 3, 'compressed': True, 'includes': ['lxml.etree', 'lxml._elementpath', 'gzip', 'docx']}},
windows = [{'script': "my_python_file.py"}],
zipfile = None,
)
It seems that moving the python script to a different location produces the error.
File "docx.pyc", line 1063, in savedocx
AssertionError
This is the savedocx line:
document = newdocument()
[...]
coreprops = coreproperties(title=title, subject=subject, creator=creator, keywords=keywords)
approps = appproperties()
contenttypes2 = contenttypes()
websettings2 = websettings()
wordrelationships2 = wordrelationships(relationships)
path_save = "C:\output"
savedocx(document, coreprops, approps, contenttypes2, websettings2, wordrelationships2, path_save)
The savedox is well writen as it works when it's not an .exe file.
How can I make the docx module work correctly? Do I've to add any other path/variable more when I make the exe?
Thanks in advance

I solved the problem by edditing the api.py file of docx egg folder which is located in the Python folder of the system.
Changing this:
_thisdir = os.path.split(__file__)[0]
_default_docx_path = os.path.join(_thisdir, 'templates', 'default.docx')
To this:
thisdir = os.getcwd()
_default_docx_path = os.path.join(thisdir, 'templates', 'default.docx')
The first one was taking the actual running program and adding it to the path to locate the templates folder.
C:\myfiles\myprogram.exe\templates\default.docx
The solution takes only the path, not the running program.
C:\myfiles\templates\default.docx

Instead of changing some library file, I find it easier and cleaner to tell python-docx explicitly where to look for the template, i.e.:
document = Document('whatever/path/you/choose/to/some.docx')
This effectively solves the py2exe and docx path problem.

Include .pyd module files in py2exe compilation

I'm trying to compile a python script. On executing the exe I got:-
C:\Python27\dist>visualn.exe
Traceback (most recent call last):
File "visualn.py", line 19, in <module>
File "MMTK\__init__.pyc", line 39, in <module>
File "Scientific\Geometry\__init__.pyc", line 30, in <module>
File "Scientific\Geometry\VectorModule.pyc", line 9, in <module>
File "Scientific\N.pyc", line 1, in <module>
ImportError: No module named Scientific_numerics_package_id
I can see the file Scientific_numerics_package_id.pyd at the location "C:\Python27\Lib\site-packages\Scientific\win32". I want to include this module file into the compilation. I tried to copy the above file in the "dist" folder but no good. Any idea?
Update:
Here is the script:
from MMTK import *
from MMTK.Proteins import Protein
from Scientific.Visualization import VRML2; visualization_module = VRML2
protein = Protein('3CLN.pdb')
center, inertia = protein.centerAndMomentOfInertia()
distance_away = 8.0
front_cam = visualization_module.Camera(position= [center[0],center[1],center[2]+distance_away],description="Front")
right_cam = visualization_module.Camera(position=[center[0]+distance_away,center[1],center[2]],orientation=(Vector(0, 1, 0),3.14159*0.5),description="Right")
back_cam = visualization_module.Camera(position=[center[0],center[1],center[2]-distance_away],orientation=(Vector(0, 1, 0),3.14159),description="Back")
left_cam = visualization_module.Camera(position=[center[0]-distance_away,center[1],center[2]],orientation=(Vector(0, 1, 0),3.14159*1.5),description="Left")
model_name = 'vdw'
graphics = protein.graphicsObjects(graphics_module = visualization_module,model=model_name)
visualization_module.Scene(graphics, cameras=[front_cam,right_cam,back_cam,left_cam]).view()

Py2exe lets you specify additional Python modules (both .py and .pyd) via the includes option:
setup(
...
options={"py2exe": {"includes": ["Scientific.win32.Scientific_numerics_package_id"]}}
)
EDIT. The above should work if Python is able to
import Scientific.win32.Scientific_numerics_package_id

There is a way to work around this types of issues that I have used a number of times. In order to add extra files to the py2exe result you can extend the media collector in order to have a custom version of it. The following code is an example:
import glob
from py2exe.build_exe import py2exe as build_exe
def get_py2exe_extension():
"""Return an extension class of py2exe."""
class MediaCollector(build_exe):
"""Extension that copies Scientific_numerics_package_id missing data."""
def _add_module_data(self, module_name):
"""Add the data from a given path."""
# Create the media subdir where the
# Python files are collected.
media = module_name.replace('.', os.path.sep)
full = os.path.join(self.collect_dir, media)
if not os.path.exists(full):
self.mkpath(full)
# Copy the media files to the collection dir.
# Also add the copied file to the list of compiled
# files so it will be included in zipfile.
module = __import__(module_name, None, None, [''])
for path in module.__path__:
for f in glob.glob(path + '/*'): # does not like os.path.sep
log.info('Copying file %s', f)
name = os.path.basename(f)
if not os.path.isdir(f):
self.copy_file(f, os.path.join(full, name))
self.compiled_files.append(os.path.join(media, name))
else:
self.copy_tree(f, os.path.join(full, name))
def copy_extensions(self, extensions):
"""Copy the missing extensions."""
build_exe.copy_extensions(self, extensions)
for module in ['Scientific_numerics_package_id',]:
self._add_module_data(module)
return MediaCollector
I'm not sure which is the Scientific_numerics_package_id module so I've assumed that you can import it like that. The copy extensions method will get a the different module names that you are having problems with and will copy all their data into the dir folder for you. Once you have that, in order to use the new Media collector you just have to do something like the following:
cmdclass['py2exe'] = get_py2exe_extension()
So that the correct extension is used. You might need to touch the code a little but this should be a good starting point for what you need.

I encountered similar probelm with py2exe and the only solution I can find ,is to use another tool to convert python to exe - pyinstaller
Its very easy tool to use and more important , it works!
UPDATE
As I understood from your comments below , running your script from command line is not working also , due to import error (My recommendation is to first check your code from command line ,and than try to convert it to EXE)
It looks like PYTHONPATH problem.
PYTHONPATH is list of paths (similar of Windows PATH) that python programs use to find import modules.
If your script run from your IDE , that means the PYTHONPATH is set correctly in the IDE ,so all imported modules are found.
In order to set PYTHONPATH you can use :
import sys|
sys.path.append(pathname)
or use the following code that add the all folders under path parameter to PYTHONPATH:
import os
import sys
def add_tree_to_pythonpath(path):
"""
Function: add_tree_to_pythonpath
Description: Go over each directory in path and add it to PYTHONPATH
Parameters: path - Parent path to start from
Return: None
"""
# Go over each directory and file in path
for f in os.listdir(path):
if f == ".bzr" or f.lower() == "dll":
# Ignore bzr and dll directories (optional to NOT include specific folders)
continue
pathname = os.path.join(path, f)
if os.path.isdir(pathname) == True:
# Add path to PYTHONPATH
sys.path.append(pathname)
# It is a directory, recurse into it
add_tree_to_pythonpath(pathname)
else:
continue
def startup():
"""
Function: startup
Description: Startup actions needed before call to main function
Parameters: None
Return: None
"""
parent_path = os.path.normpath(os.path.join(os.getcwd(), ".."))
parent_path = os.path.normpath(os.path.join(parent_path, ".."))
# Go over each directory in parent_path and add it to PYTHONPATH
add_tree_to_pythonpath(parent_path)
# Start the program
main()
startup()

The ImportError is rectified by using "Gil.I" and "Janne Karila" suggestion by setting pythonpath and by using include function. But before this I had to create __init__.py file in the win32 folder of both the modules.
BTW I still got another error for the above script - link

How to update the variable list in the config file dynamically in to python exe file

I have a python file that converted into exe using pyinstaller
but i need to get the data from another file which is to be updated and without another pyinstaller execution it needed to be worked like getting a data from the config file to exe
from configfile import variables
variables that is loaded from the configfile.py file but after converting into exe i cant able to update configfile.py variables
any suggestion will be welcomed

function from the python config file updated and loaded dynamically outside the environment of exe
import os
extDataDir = os.getcwd()
ext_config = os.path.join(extDataDir, '', 'configfile.py')
def importCode(code,name,add_to_sys_modules=0):
import sys,imp
module = imp.new_module(name)
exec(code,module.__dict__)
if add_to_sys_modules:
sys.modules[name] = module
return module
configfile_rd = open(ext_config, "r")
configfile_code = configfile_rd.read()
configfile = importCode(configfile_code,"configfile") #dynamically readed the config file from exe and execute the python file configfile.py and working as well as import configfile
constant = configfile.variables()