I'm using the latest Gooey library on Python 2.7 on Windows as a GUI for a simple argparse, but for some reason, the script keeps giving me the [Errno2] No File Exists.
I think it is because there is a space in the path of the Anaconda installation (i.e. C:\Users\FirstName LastName\Etc.) but I'm stumped.
I have tried str.replace all the \ with \\, but I keep getting the same error message. Any ideas of what to do?
Code:
from __future__ import print_function
import pandas as pd
import numpy as np
import glob
import sys
import os
import json
from argparse import ArgumentParser
from gooey import Gooey, GooeyParser
#Gooey(program_name="CPT Lookup")
def parse_args():
stored_args = {}
parser = GooeyParser(description='CPT Lookup')
#Eventually make into checkboxes
parser.add_argument('year',
action='store',
default=stored_args.get('year'),
widget='FileChooser',
help="CSV file with extracted year")
parser.add_argument('CPT',
action='store',
default=stored_args.get('CPT'),
widget='TextField',
help='CPT Code')
args = parser.parse_args()
return args
def loadCSV(year):
#DO I DO SOMETHING LIKE YEAR.REPLACE('\','\\')?
df = pd.read_csv(year)
return df
if __name__ == '__main__':
conf = parse_args()
print("Opening CSV file")
sales_df = loadCSV(conf.year)
This was an issue with the Gooey library itself (I'm the author). It wasn't quoting incoming file paths correctly.
If you pull down the latest release from PyPi (pip install -U gooey), your example script should run without issue.
Related
I have written a function that will install the Module required to run a script. My problem is that the function runs every time the script is running. I need to run the function only the first time the script is running so that after installing the module the function does not run every time the script is running.
My code is
import importlib
import subprocess
import pkg_resources
import os, time, json, datetime, sys
def import_and_install(package):
try:
importlib.import_module(package)
except (ModuleNotFoundError, pkg_resources.DistributionNotFound) as e:
print("{0} module is not installed.\n Don't worry. will take care\n".format(package))
package = [package]
subprocess.check_call([sys.executable, '-m', 'pip', 'install'] + package)
packages = ['pandas', 'numpy', 'threading', 'xlwings']
for package in packages:
import_and_install(package)
import pandas as pd
import threading
import xlwings as xw
import numpy as np
I am not sure what you mean by running the script only the first time the script is running?.
does that mean you want to run the script only once per computer/VM?.
by the looks of it, it seems that the script that you want to run is for handling dependencies and packaging management, if that is the case I would recommend you use packages manager instead such as Poetry(https://python-poetry.org/).
if you still consist on doing everything on your own.
the simplest solution I could think of Is creating a file that stores a flag that tells you if the script has already ran or not.
but there might be better solutions
We can compare the list of packages required with the list of packages installed.
To get the list of packages installed we can use:
import pkg_resources
pkg_resources.working_set
This will return an iterator from where we can get the modules installed names (key) and versions (version).
import pkg_resources
modules_installed = {pkg.key:pkg.version for pkg in pkg_resources.working_set}
# this will return a dict like this one, for example:
# {'setuptools': '65.3.0',
# 'pip': '22.2.2',
# 'xlwings': '0.28.5'
# ...}
Now we can compare them with a function that will help to verify if a package is install and also verify the version in case that the package has '==' in the value:
# package: for example 'pandas==1.5.0'
# modules_installed: {pkg.key:pkg.version for pkg in pkg_resources.working_set}
def is_pkg_installed(package, modules_installed) -> bool:
pkg_name = package.split('==')[0] if '==' in package else package
pkg_ver = package.split('==')[1] if '==' in package else None
if pkg_name not in modules_installed:
return False
elif pkg_ver:
installed_version = modules_installed[pkg_name]
if pkg_ver != installed_version:
return False
return True
Using list comprehensions we can get the list of packages_to_install:
packages_to_install = [package for package in packages_list
if not is_pkg_installed(package, modules_installed)]
# values example: packages_to_install = ['pandas==1.5.0', 'numpy']
To install the packages_to_install, we can use the next function that installs all the packages that are in the list:
import subprocess
import sys
def pip_install(packages: list) -> None:
if not packages:
return
# this will do a call like this one:
# ../python.exe -m pip install pandas=1.5.0, numpy
subprocess.check_call([sys.executable, '-m', 'pip', 'install'] + packages)
finally the complete example, where pandas has a specific version selected pandas==1.5.0:
import subprocess
import pkg_resources
import sys
def pip_install(packages: list) -> None:
if not packages:
return
subprocess.check_call([sys.executable, '-m', 'pip', 'install'] + packages)
def is_pkg_installed(package, modules_installed) -> bool:
pkg_name = package.split('==')[0] if '==' in package else package
pkg_ver = package.split('==')[1] if '==' in package else None
if pkg_name not in modules_installed:
return False
elif pkg_ver:
installed_version = modules_installed[pkg_name]
if pkg_ver != installed_version:
return False
return True
def install_packages(packages_list: list) -> None:
modules_installed = {pkg.key:pkg.version for pkg in pkg_resources.working_set}
packages_to_install = [package for package in packages_list
if not is_pkg_installed(package, modules_installed)]
if packages_to_install:
print(f"{packages_to_install} modules are not installed.\nDon't worry. will take care\n")
pip_install(packages_to_install)
packages = ['pandas==1.5.0', 'numpy', 'threading', 'xlwings']
install_packages(packages)
import pandas as pd
import threading
import xlwings as xw
import numpy as np
install_packages() will get the list of packages that are not installed or have different version, and if the list is not empty, run pip_install()
I am setting up a package using distutils.
I need to allow access to a module that is built during the set-up process and is located in ./build/temp.linux-x86_64-3.6. I do this by including the
include_dirs=["./build/temp.linux-x86_64-3.6"]
when adding the extension to the distutils Configuration.
My question is there a way of setting this using a wildcard such as:
include_dirs=["./build/temp.linux*"]
as when I try this it fails, citing error:
Nonexistent include directory ‘build/temp.linux*’ [-Wmissing-include-dirs]
The reason I want this is that the build folder will be named differently depending on the system. Alternatively if anyone knows a way of figuring out what this temp build folder will be called that would also work.
The way I have got around this problem is as follows:
def return_major_minor_python():
import sys
return str(sys.version_info[0])+"."+str(sys.version_info[1])
def return_include_dir():
from distutils.util import get_platform
return get_platform()+'-'+return_major_minor_python()
Then when calling config.add_extension() using:
include_dirs=['build/temp.' + return_include_dir()]
So the whole process for adding a f90wrapped, f2py extension to a python package is:
def setup_fort_ext(args,parent_package='',top_path=''):
from numpy.distutils.misc_util import Configuration
from os.path import join
import sys
config = Configuration('',parent_package,top_path)
fort_src = [join('PackageName/','fortran_source.f90')]
config.add_library('_fortran_source', sources=fort_src,
extra_f90_compile_args = [ args["compile_args"]],
extra_link_args=[args["link_args"]])
sources = [join('PackageName','f90wrap_fortran_source.f90')]
config.add_extension(name='_fortran_source',
sources=sources,
extra_f90_compile_args = [ args["compile_args"]],
extra_link_args=[args["link_args"]],
libraries=['_tort'],
include_dirs=['build/temp.' + return_include_dir()])
return config
if __name__ == '__main__':
import sys
import subprocess
import os
install_numpy() #installs numpy
install_dependencies() #calls to pip to install any requirements
from numpy.distutils.core import setup
config = {'name':'PackageName',
'version':__version__,
'project_description':'Some Package description',
'description':'Some package Description',
'long_description': open('README.txt').read(),
'long_description_content_type':'text/markdown',
'author':'Your name here',
'author_email':'your email here',
'url':'link to git repo here',
'python_requires':'>=3.3',
'packages':['PackageName'],
'package_dir':{'PackageName':'PackageName'},
'package_data':{'PackageName':['*so*']},
'name': 'PackageName'
}
config_fort = setup_fort_ext(args,parent_package='PackageName',top_path='')
config2 = dict(config,**config_fort.todict())
setup(**config2)
where the source fortran_source.f90 is wrapped beforehand and the resulting wrapped source file (f90wrap_fortran_source.f90) is included as a library, and subsequently compiled by f2py.
args in the above is just a dict with the any linking or compile args you wish to pass through.
I'm new to Python. What I want to do is after setup esptool package (pip install esptool) call its main method with a bunch of arguments in my application. Something like:
esptool.py -p /dev/ttyUSB0 write_flash -fm qio 0x0000
There is an issue I faced. esptool is not on the packages list in python to import (it is installed with pip already). How am I gonna user import and call the main method?
Resolving import issues
You can't simply invoke import esptool because esptool.py is an executable script, thus not meant to be imported like a plain module. However, there are workarounds for importing code from executable scripts; here are two I know of:
extending sys.path
You can extend the sys.path to include the bindir containing the esptool.py script. Simple check from command line:
$ PYTHONPATH=$(which esptool.py)/.. python -c "import esptool; esptool.main()"
should print you the usage help text.
Extending sys.path in code:
import os
import sys
try:
from shutil import which
except ImportError:
from distutils.spawn import find_executable as which
bindir = os.path.dirname(which('esptool.py'))
sys.path.append(bindir) # after this line, esptool becomes importable
import esptool
if __name__ == '__main__':
esptool.main()
using import machinery
You can avoid extending sys.path by using the mechanisms of importing Python code from arbitrary files. I like this solution more than fiddling with sys.path, but unfortunately, it is not portable between Python 2 and 3.
Python 3.5+
import importlib.machinery
import importlib.util
from shutil import which
if __name__ == '__main__':
loader = importlib.machinery.SourceFileLoader('esptool', which('esptool.py'))
spec = importlib.util.spec_from_loader(loader.name, loader)
esptool = importlib.util.module_from_spec(spec)
loader.exec_module(esptool) # after this line, esptool is imported
esptool.main()
Python 2.7
import imp
from distutils.spawn import find_executable as which
if __name__ == '__main__':
esptool = imp.load_source('esptool', which('esptool.py'))
esptool.main()
Passing command line arguments
The command line arguments are stored in sys.argv list, so you will have to temporarily overwrite it in order to pass the arguments to the main function:
# assuming esptool is imported
import sys
if __name__ == '__main__':
# save the current arguments
argv_original = sys.argv[:]
# overwrite arguments to be passed to esptool argparser
sys.argv[:] = ['', '-p', '/dev/ttyUSB0', 'write_flash', '-fm', 'qio', '0x0000']
try:
esptool.main()
except Exception:
# TODO deal with errors here
pass
finally: # restore original arguments
sys.argv[:] = argv_original
I am trying to save an Excel file encrypted with password. I have tried following the guide on https://help.libreoffice.org/Common/Protecting_Content_in - and works perfectly. However, this is in the GUI, but I am looking for a solution using the command line interface in headless mode.
I have looked at the man libreoffice, but I could not find anything in there.
Likewise I have looked at the documentation of the Python 3 library openpyxl, but I did not find anything useful there either.
Is it possible to save an Excel 2007+ file encrypted with a password on Ubuntu 14.04/16.04 using the command line (or Python library) that do not require any user interaction or X session?
There is solution using Jython and Apache POI. If you want like to use it from CPython/PyPy, you can use subprocess module to call external Jython script.
I assume that you have Java JRE/JDK installed
Create non-encrypted xlsx file with Excel/Calc or use xlsxwriter or openpyxl and save it as test1.xlsx
Download standalone Jython
Download Apache POI
Extract Apache POI in same dir where is standalone Jython jar
Save following Jython script as encrypt.py:
import os
import sys
from java.io import BufferedInputStream
from java.io import FileInputStream
from java.io import FileOutputStream
from java.io import File
from java.io import IOException
from org.apache.poi.poifs.crypt import EncryptionInfo, EncryptionMode
from org.apache.poi.poifs.crypt import CipherAlgorithm, HashAlgorithm
from org.apache.poi.poifs.crypt.agile import AgileEncryptionInfoBuilder
from org.apache.poi.openxml4j.opc import OPCPackage, PackageAccess
from org.apache.poi.poifs.filesystem import POIFSFileSystem
from org.apache.poi.ss.usermodel import WorkbookFactory
def encrypt_xlsx(in_fname, out_fname, password):
# read
in_f = File(in_fname)
in_wb = WorkbookFactory.create(in_f, password)
in_fis = FileInputStream(in_fname)
in_wb.close()
# encryption
out_poi_fs = POIFSFileSystem()
info = EncryptionInfo(EncryptionMode.agile)
enc = info.getEncryptor()
enc.confirmPassword(password)
opc = OPCPackage.open(in_f, PackageAccess.READ_WRITE)
out_os = enc.getDataStream(out_poi_fs)
opc.save(out_os)
opc.close()
# write
out_fos = FileOutputStream(out_fname)
out_poi_fs.writeFilesystem(out_fos)
out_fos.close()
if __name__ == '__main__':
in_fname = sys.argv[1]
out_fname = sys.argv[2]
password = sys.argv[3]
encrypt_xlsx(in_fname, out_fname, password)
Call it from console:
java -cp "jython-standalone-2.7.0.jar:poi-3.15/lib/commons-codec-1.10.jar:poi-3.15/lib/commons-collections4-4.1.jar:poi-3.15/poi-3.15.jar:poi-3.15/poi-ooxml-3.15.jar:poi-3.15/poi-ooxml-schemas-3.15.jar:poi-3.15/ooxml-lib/curvesapi-1.04.jar:poi-3.15/ooxml-lib/xmlbeans-2.6.0.jar" org.python.util.jython -B encrypt.py test1.xlsx test1enc.xlsx 12345678
Where:
encrypt.py - name of script
test1.xlsx - input filename
test1enc.xlsx - output filename
12345678 - password
Final encrypted xslx should be in test1enc.xlsx.
I've been playing around with IPython.parallel and I wanted to use some custom modules of my own, but haven't been able to do it as explained on the cookbook using dview.sync_imports(). The only thing that has worked for me was something like
def my_parallel_func(args):
import sys
sys.path.append('/path/to/my/module')
import my_module
#and all the rest
and then in the main just to
if __name__=='__main__':
#set up dview...
dview.map( my_parallel_func, my_args )
The correct way to do this would in my opinion be something like
with dview.sync_imports():
import sys
sys.path.append('/path/to/my/module')
import my_module
but this throws an error saying there is no module named my_module.
So, what is the right way of doing it using dview.sync_imports()??
The problem is that you're changing the PYTHONPATH just in the local process running the Client, and not in the remote processes running in the ipcluster.
You can observe this behaviour if you run the next piece of code:
from IPython.parallel import Client
rc = Client()
dview = rc[:]
with dview.sync_imports():
import sys
sys.path[:] = ['something']
def parallel(x):
import sys
return sys.path
print 'Local: ', sys.path
print 'Remote: ', dview.map_sync(parallel, range(1))
Basically all the modules that you want to use with sync_imports must already be in the PYTHONPATH.
If it's not in the PYTHONPATH then you must add it to the path in the function that you execute remotely, and then import the module in the function.