I have written a Twisted bin file that is deployed on /usr/bin during application deployement, based on the Axiom example provided elsewhere on StackOverflow (I don't remember where), the project can be found here.
My problem is that, during the python setup.py install process, the installed bin file is different than the one from Axiom package:
/usr/bin/axiomatic
#!/code/venv/bin/python
from axiom.scripts import axiomatic
axiomatic.main()
/usr/bin/myapp
#!/code/venv/bin/python
# EASY-INSTALL-DEV-SCRIPT: 'MyApp==0.0.1','myapp'
__requires__ = 'MyApp==0.0.1'
__import__('pkg_resources').require('MyApp==0.0.1')
exec(compile(open(__file__).read(), __file__, 'exec'))
and the latter doesn't work when invoking it from the bash shell: myapp start
I get the following error: unknow command myapp
If I use python setup.py develop instead of python setup.py install everything works smoothly.
I have setup a little test application that starts a tcp service on port 1234:
the command twistd finger works, the service starts
the command fingerize start (different name on purpose, to be sure not calling the wrong one) doesn't work
Here is the code:
bin/fingerize
#!/usr/bin/python
from finger import tap
tap.main()
twisted/plugins/finger_plugin.py
from twisted.application.service import ServiceMaker
Finger = ServiceMaker('Finger', 'finger.plugins', 'blah', 'finger')
finger/plugins.py
from twisted.application import internet
from twisted.internet import endpoints
from twisted.python import usage
from twisted.internet import protocol
class Options(usage.Options):
""" """
def makeService(options):
from twisted.internet import reactor
endpoint = endpoints.TCP4ServerEndpoint(reactor, 1234)
return internet.StreamServerEndpointService(
endpoint,
protocol.Factory())
finger/tap.py
import sys
from twisted.python import usage
from twisted.scripts import twistd
class Start(twistd.ServerOptions):
run = staticmethod(twistd.run)
def subCommands(self):
raise AttributeError()
subCommands = property(subCommands)
def parseOptions(self, args):
print(sys.argv)
print(args)
a = self.getArguments(args)
print(a)
sys.argv[1:] = a
print(sys.argv)
print('Starting finger service...')
self.run()
def getArguments(self, args):
args.extend(['--pidfile', self.parent.pid()])
args.extend(['finger'])
return args
class Options(usage.Options):
def subCommands():
def get(self):
yield ('start', None, Start, 'Launch finger service')
return get,
subCommands = property(*subCommands())
def pid(self):
return '/tmp/finger.pid'
def main(argv=None):
o = Options()
try:
o.parseOptions(argv)
except usage.UsageError, e:
raise SystemExit(str(e))
setup.py
from setuptools import find_packages
from setuptools import setup
METADATA = dict(
name='Finger',
version='0.0.1',
packages=find_packages(),
scripts=['bin/fingerize'],
install_requires=[
'Twisted >= 15.5.0',
],
include_package_data=True,
zip_safe=False,
)
setup(**METADATA)
And when I call fingerize start I get: /code/test/bin/fingerize: Unknown command: finger (test is a virtualenv)
Since you are using setuptools in your setup.py, you can use the newer entry_points keyword, like so:
entry_points={
'console_scripts': [
'fingerize=finger.tap:main',
],
},
instead of the scripts keyword.
Note if python setup.py install is run in a virtualenv, it will put the script in env/bin/fingerize (assuming your virtualenv folder is env).
A nice way to make such "python applications" then available globally (for some deployment purpose), while their inner mechanics doesn't interfere with your system python applciations, is to use pipsi (pip install pipsi) (made by the guy who developed flask). It's designed to expose just the project's executables to the system and have everything else tucked away in its own virtualenv.
You can see more on how to write/configure/distribute python packages at the packaging.python.org guide.
Related
Is it possible to specify a post-install Python script file as part of the setuptools setup.py file so that a user can run the command:
python setup.py install
on a local project file archive, or
pip install <name>
for a PyPI project and the script will be run at the completion of the standard setuptools install? I am looking to perform post-install tasks that can be coded in a single Python script file (e.g. deliver a custom post-install message to the user, pull additional data files from a different remote source repository).
I came across this SO answer from several years ago that addresses the topic and it sounds as though the consensus at that time was that you need to create an install subcommand. If that is still the case, would it be possible for someone to provide an example of how to do this so that it is not necessary for the user to enter a second command to run the script?
Note: The solution below only works when installing a source distribution zip or tarball, or installing in editable mode from a source tree. It will not work when installing from a binary wheel (.whl)
This solution is more transparent:
You will make a few additions to setup.py and there is no need for an extra file.
Also you need to consider two different post-installations; one for development/editable mode and the other one for install mode.
Add these two classes that includes your post-install script to setup.py:
from setuptools import setup
from setuptools.command.develop import develop
from setuptools.command.install import install
class PostDevelopCommand(develop):
"""Post-installation for development mode."""
def run(self):
develop.run(self)
# PUT YOUR POST-INSTALL SCRIPT HERE or CALL A FUNCTION
class PostInstallCommand(install):
"""Post-installation for installation mode."""
def run(self):
install.run(self)
# PUT YOUR POST-INSTALL SCRIPT HERE or CALL A FUNCTION
and insert cmdclass argument to setup() function in setup.py:
setup(
...
cmdclass={
'develop': PostDevelopCommand,
'install': PostInstallCommand,
},
...
)
You can even call shell commands during installation, like in this example which does pre-installation preparation:
from setuptools import setup
from setuptools.command.develop import develop
from setuptools.command.install import install
from subprocess import check_call
class PreDevelopCommand(develop):
"""Pre-installation for development mode."""
def run(self):
check_call("apt-get install this-package".split())
develop.run(self)
class PreInstallCommand(install):
"""Pre-installation for installation mode."""
def run(self):
check_call("apt-get install this-package".split())
install.run(self)
setup(
...
P.S. there are no any pre-install entry points available on setuptools. Read this discussion if you are wondering why there is none.
Note: The solution below only works when installing a source distribution zip or tarball, or installing in editable mode from a source tree. It will not work when installing from a binary wheel (.whl)
This is the only strategy that has worked for me when the post-install script requires that the package dependencies have already been installed:
import atexit
from setuptools.command.install import install
def _post_install():
print('POST INSTALL')
class new_install(install):
def __init__(self, *args, **kwargs):
super(new_install, self).__init__(*args, **kwargs)
atexit.register(_post_install)
setuptools.setup(
cmdclass={'install': new_install},
Note: The solution below only works when installing a source distribution zip or tarball, or installing in editable mode from a source tree. It will not work when installing from a binary wheel (.whl)
A solution could be to include a post_setup.py in setup.py's directory. post_setup.py will contain a function which does the post-install and setup.py will only import and launch it at the appropriate time.
In setup.py:
from distutils.core import setup
from distutils.command.install_data import install_data
try:
from post_setup import main as post_install
except ImportError:
post_install = lambda: None
class my_install(install_data):
def run(self):
install_data.run(self)
post_install()
if __name__ == '__main__':
setup(
...
cmdclass={'install_data': my_install},
...
)
In post_setup.py:
def main():
"""Do here your post-install"""
pass
if __name__ == '__main__':
main()
With the common idea of launching setup.py from its directory, you will be able to import post_setup.py else it will launch an empty function.
In post_setup.py, the if __name__ == '__main__': statement allows you to manually launch post-install from command line.
Combining the answers from #Apalala, #Zulu and #mertyildiran; this worked for me in a Python 3.5 environment:
import atexit
import os
import sys
from setuptools import setup
from setuptools.command.install import install
class CustomInstall(install):
def run(self):
def _post_install():
def find_module_path():
for p in sys.path:
if os.path.isdir(p) and my_name in os.listdir(p):
return os.path.join(p, my_name)
install_path = find_module_path()
# Add your post install code here
atexit.register(_post_install)
install.run(self)
setup(
cmdclass={'install': CustomInstall},
...
This also gives you access the to the installation path of the package in install_path, to do some shell work on.
I think the easiest way to perform the post-install, and keep the requirements, is to decorate the call to setup(...):
from setup tools import setup
def _post_install(setup):
def _post_actions():
do_things()
_post_actions()
return setup
setup = _post_install(
setup(
name='NAME',
install_requires=['...
)
)
This will run setup() when declaring setup. Once done with the requirements installation, it will run the _post_install() function, which will run the inner function _post_actions().
If using atexit, there is no need to create a new cmdclass. You can simply create your atexit register right before the setup() call. It does the same thing.
Also, if you need dependencies to be installed first, this does not work with pip install since your atexit handler will be called before pip moves the packages into place.
I wasn't able to solve a problem with any presented recommendations, so here is what helped me.
You can call function, that you want to run after installation just after setup() in setup.py, like that:
from setuptools import setup
def _post_install():
<your code>
setup(...)
_post_install()
I'm trying to create a cli tool using python but whenever I try to run the command in terminal i get the error 'bash: command not found'. The python modules installs without any errors. I'm on macos and using python version: 3.9.4. I think this may be a PATH issue but currently unsure.
Here is the setup.py:
from setuptools import setup
setup(
name = 'mycommand',
version = '0.1.0',
packages = ['mycommand'],
entry_points = {
'console_scripts': [
'mycommand = mycommand.__main__:main'
]
}
)
Here is the main.py:
import sys
def main():
print('in main')
args = sys.argv[1:]
print('count of args :: {}'.format(len(args)))
for arg in args:
print('passed argument :: {}'.format(arg))
if __name__ == '__main__':
main()
Hope somebody can help me with this.
This is the tutorial I have followed: https://trstringer.com/easy-and-nice-python-cli/
if your layout is like that:
setup.py
mycommand/main.py
then the entrypoint is simply mycommand.main:main:
mycommand/main.py -> mycommand.main
:main -> def main()
You need to use the scripts keyword argument.
First add a file called mycommand (note that you should not have any extension) in the same directory as main.py
#!/usr/bin/env python
from mycommand.main import main
main()
then edit your setup.py like this:
from setuptools import setup
setup(
name = 'mycommand',
version = '0.1.0',
packages = ['mycommand'],
scripts=['mycommand/mycommand'],
)
Then when you install this package you will be able to call mycommand
I'm trying to provide a bash completion script for my CLI tool that is written in Python. According to the Python Packaging Authority, data_files in setup.py is exactly what I need:
Although configuring package_data is sufficient for most needs, in some cases you may need to place data files outside of your packages. The data_files directive allows you to do that. It is mostly useful if you need to install files which are used by other programs, which may be unaware of Python packages.
So I added the completion file like this:
data_files=[
('/usr/share/bash-completion/completions', ['completion/dotenv']),
],
and try to test it with:
pip install -e .
In my virtual environment. However, the completion script gets not installed. Did I forgot something or is pip broken? The full project can be found here
I had the same issue and I have implemented a workaround.
It seems to me that python setup.py develop or (pip install -e .) does not run the same function than python setup.py install.
In fact, I have noticed by looking in the source code that python setup.py install run build_py :
https://github.com/python/cpython/blob/master/Lib/distutils/command/build_py.py#L134
https://github.com/pypa/setuptools/blob/master/setuptools/command/build_py.py
After a few digging I have opted to override the develop command as follow. The following code is python3.6:
""" SetupTool Entry Point """
import sys
from pathlib import Path
from shutil import copy2
from setuptools import find_packages, setup
from setuptools.command.develop import develop
# create os_data_files that will be used by the default install command
os_data_files = [
(
f"{sys.prefix}/config", # providing absolute path, sys.prefix will be different in venv
[
"src/my_package/config/properties.env",
],
),
]
def build_package_data():
""" implement the necessary function for develop """
for dest_dir, filenames in os_data_files:
for filename in filenames:
print(
"CUSTOM SETUP.PY (build_package_data): copy %s to %s"
% (filename, dest_dir)
)
copy2(filename, dest_dir)
def make_dirstruct():
""" Set the the logging path """
for subdir in ["config"]:
print("CUSTOM SETUP.PY (make_dirstruct): creating %s" % subdir)
(Path(BASE_DIR) / subdir).mkdir(parents=True, exist_ok=True)
class CustomDevelopCommand(develop):
""" Customized setuptools install command """
def run(self):
develop.run(self)
make_dirstruct()
build_package_data()
# provide the relevant information for stackoverflow
setup(
package_dir={"": "src"},
packages=find_packages("src"),
data_files=os_data_files,
cmdclass={"develop": CustomDevelopCommand},
)
I'm trying to create Windows service from my Python code using cx_Freeze. Python code works fine, it's debugged and reliable, but when I create exe file I got problems.
Here is my cx_Freeze setup code:
# Config file for packing python scripts into exe file
# Run with "python.exe create_win_exe.py build"
import sys
from cx_Freeze import setup, Executable
version = '1.00'
# Dependencies are automatically detected, but it might need fine tuning.
build_exe_options = {'packages': ['win32timezone'], 'includes': ['idna.idnadata']}
# Use default base - console application
base = None
setup(name = "service-test",
version = version,
description = "Service test",
options = {"build_exe": build_exe_options},
executables = [Executable("service-test.py", base=base)])
Python code (I've taken example from here How do you run a Python script as a service in Windows?):
# -*- coding: utf-8 -*-
import win32serviceutil
import win32service
import win32event
import servicemanager
import socket
import datetime
import time
class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "TestService5"
_svc_display_name_ = "Test Service 5"
stop_flag = False
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.hWaitStop = win32event.CreateEvent(None,0,0,None)
socket.setdefaulttimeout(60)
def SvcStop(self):
self.stop_flag = True
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,''))
self.main()
def main(self):
fout = open('C:\\Users\\Константин\\service-test.log', 'w')
while not self.stop_flag:
try:
fout.write(str(datetime.datetime.now()) + '\n')
fout.flush()
except BaseException as be:
print('Error while writing to file: {}'.format(str(be)))
time.sleep(1)
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
When I create exe file, I could install it as service:
> service-test.exe install
Installing service TestService5
Service installed
Also I could finely debug it, it works as expected.
But it doesn't start:
> service-test.exe start
Starting service TestService5
Error starting service: The service did not respond timely
Same trouble if I use pyinstaller instead of cx_Freeze.
Could anybody help with it?
Might be permissions issue.
Looking at your code, you do not specify the account the service will be run under, so it is created under "Local System".
https://learn.microsoft.com/en-us/windows/desktop/Services/localsystem-account
Try indicating the user to run your service via the install command line arguments, or you can navigate to the services dialog (services.msc) after you've installed the service and modify the Log On.
I created a folder and placed two files into it: file.py and create_win_exe.py (file names not very important)
And I used the following command: pyinstaller file.py in that same directory and it ran through the install process fine. Once it was finished, I navigated into the dist/file/ directory and opened file.exe, and it worked perfectly fine. I'm not sure why you have an error, but try using pyinstaller exactly how I've shown and see if it works.
I'm trying to write a windows Service in python, but the tricky part is i want to deploy it on a machine that doesn't have python.
I've successfully created a service like this, and it works if i run from my machine. the problem starts when i try to convert it to an exe and then try to install it.
first i tried to use cx_freeze service example, (seen here), the setup.py look like this :
from cx_Freeze import setup, Executable
options = {'build_exe': {'includes': ['ServiceHandler']}}
executables = [Executable('Config.py', base='Win32Service', targetName='gsr.exe')]
setup(name='GSR',
version='0.1',
description='GSR SERVICE',
executables=executables,
options=options
)
and config.py is:
NAME = 'GSR_%s'
DISPLAY_NAME = 'GSR TEST - %s'
MODULE_NAME = 'ServiceHandler'
CLASS_NAME = 'Handler'
DESCRIPTION = 'Sample service description'
AUTO_START = True
SESSION_CHANGES = False
but when i try to build it (python setup.py build) i get an error:
"cx_Freeze.freezer.ConfigError: no base named Win32Service"
Second, i tried using a regular cx_freeze setup, the exe i get installs the service fine but once i try to start it i get an error:
"Error 1053: The service did not respond to the start or control request in a timely fashion"
setup.py - python 3.3 regualr exe, installs the service but when trying to start it sends an error:
from cx_Freeze import setup, Executable
packages = ['win32serviceutil','win32service','win32event','servicemanager','socket','win32timezone','cx_Logging','ServiceHandler']
build_exe_options = {"packages": packages}
executable = [Executable("ServiceHandler.py")]
setup( name = "GSR_test",
version = "1.0",
description = "GSR test service",
options = {"build_exe": build_exe_options},
executables = executable)
finally, I managed to get it to work in python 2.7 using py2exe, but py2exe isn't available for python 3.3 and I my code is in 3.3
i guess the problem is in the configuration of the setup.py im using with cx_freeze.
any ideas ??
my ServiceHandler:
import pythoncom
import win32serviceutil
import win32service
import win32event
import servicemanager
import socket
from test import test
import threading
class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "GSR_test"
_svc_display_name_ = "GSR test Service"
def __init__(self,args):
win32serviceutil.ServiceFramework.__init__(self,args)
self.hWaitStop = win32event.CreateEvent(None,0,0,None)
socket.setdefaulttimeout(60)
self.app = test()
self.flag = threading.Event()
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
self.flag.set()
def SvcDoRun(self):
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_,''))
self.main()
def main(self):
t = threading.Thread(target=self.app.run)
t.start()
self.flag.wait()
raise SystemExit
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)
Setup.py , python 2.7 using py2exe that works: (taken from here)
from distutils.core import setup
import py2exe
setup( service = ["ServiceHandler"],
description = "SERVICE TEST",
modules = ["GSR_test"],
cmdline_style='pywin32', )
Thanks,
Amit
tldr Python 3.5 Windows Service build with pyinstaller : gist
Here a simple Windows Service with python :
WindowsService.py
import servicemanager
import socket
import sys
import win32event
import win32service
import win32serviceutil
class TestService(win32serviceutil.ServiceFramework):
_svc_name_ = "TestService"
_svc_display_name_ = "Test Service"
def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
socket.setdefaulttimeout(60)
def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)
def SvcDoRun(self):
rc = None
while rc != win32event.WAIT_OBJECT_0:
with open('C:\\TestService.log', 'a') as f:
f.write('test service running...\n')
rc = win32event.WaitForSingleObject(self.hWaitStop, 5000)
if __name__ == '__main__':
if len(sys.argv) == 1:
servicemanager.Initialize()
servicemanager.PrepareToHostSingle(TestService)
servicemanager.StartServiceCtrlDispatcher()
else:
win32serviceutil.HandleCommandLine(TestService)
Create an exe with pyinstaller (pip install pyinstaller) and python 3.5 :
(env)$ python -V
Python 3.5.2
(env)$ pip freeze
PyInstaller==3.2
(env)$ pyinstaller -F --hidden-import=win32timezone WindowsService.py
Install and run the service
(env) dist\WindowsService.exe install
Installing service TestService
Service installed
(env) dist\WindowsService.exe start
Starting service TestService
Check C:\\TestService.log file.
Stop and clean
(env) dist\WindowsService.exe stop
(env) dist\WindowsService.exe remove
It appears Win32Service was updated with Python 3.x support within the cx_Freeze project as a result of this thread. This user originally had the same issue you reported, so I'm assuming this will also resolve your issue.
Based on the error reported, it's caused when _GetBaseFileName() within Freezer.py fails to find the compiled Win32Service.exe. This executable should be built when cx_Freeze gets built/installed.
If it's not too much to ask, can search the installed cx_Freeze installation directory for "Win32Service.exe" and confirm that it exists. Hopefully this gets you one step closer.
I've edited the Win32Service.c in the cx_freeze src for python3 support
Edited it with some preprocessor commands, for python2 support too (but not tested for python2 yet)
download sources of cx_freeze from pypi and extract it
download cx_logging and extract it to the cx_freeze-4.3.4 directory
replace the Win32Service.c file in cx_Freeze-4.3.4\source\bases with my edited version
edit line 170 in setup.py from if moduleInfo is not None and sys.version_info[:2] < (3, 0): to if moduleInfo is not None:
run python setup.py build in cx_freeze-4.3.4 directory (you must have MSC installed to compile the C source files)
copy cx_Freeze-4.3.4\build\lib.win32-3.4\cx_Freeze\bases\Win32Service.exe to C:\Python34\Lib\site-packages\cx_Freeze\bases (or the path where your python3 is installed)
now you can create frozen exe with the Win32Service base (no more the cx_Freeze.freezer.ConfigError: no base named Win32Service error)