This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to concatenate multiple Python source files into a single file?
Is there a Python “pre-interpreter” to take as input a .py module containing imports and expand it so it can be run inline in an interpreter session on the command line or Telnet session? Imports of built-ins or installed modules are OK to keep, but I’d like the pre-interpreter to expand my own modules. This way, I can avoid an install while still employing modular programming techniques. For example, if I write these two modules:
myprint.py:
from math import pi
def print_pi():
print "{0:6f}".format(pi)
main.py:
from myprint import print_pi
print_pi()
Running the pre-interpreter on main.py, the output would be:
from math import pi
def print_pi():
print "{0:6f}".format(pi)
print_pi()
Update October 10 2012 22:36 Eastern USA:
Thanks to everyone who responded! My program's runtime host is permanently installed on board a locomotive, from where my program will query and monitor that computer and many other onboard systems. You can correctly predict that this computing environment is safety and mission critical (thus the need for my team’s monitoring software). Installing software in such an environment requires privileges, consumes resources, and imposes a small but real risk to the system. Thus, prudent but weeks-consuming checks have been implemented by the railroad to scrutinize software to be installed, including sign-off by a change-control board. This is the route we had planned for. But if we can eliminate this risk and thereby lessen the formality by running without any installation, that could be an advantage to at least consider. However, we do not want our architecture to be restricted by the non-install requirement, i.e., we don't want to have to write the whole program in one module. From your answers it seems my notion is not possible.
Your example assumes that Python imports are something like C macros. That's not the case. Pythons import is much more powerful and complex. Therefore an import cannot be replaced by copying a few lines of code. The the answer is: No there is no such preprocessor. But if you specify more detailed what problem you really would like to solve, we might be help you to solve it in python.
Related
I have written a Python script which models an academic problem which I wish to publish. I will put the source on Github and some academics that just happen to know Python may get my source and play with it themselves. However there are probably more academics that may be interested in the model but that are not python programmers and I would like them to be able to run my model too. Even though they are not programmers they could at least try out editing the values of some of the parameters to see how that affects the results. So now my question is how could I arrange for a non-python programmer to run a Python program as easily (for them) as possible. I would guess that my options may be...
google colab
an online python compiler like this one
compiling the program into an exe (and letting the user set parameters via a config file)
something else?
So now a couple of complications that makes my problem trickier.
The output of the program is graphical and uses matplotlib. As I understand it, the utilities that turn python scripts into exe files struggle or fail altogether when it comes to matplotlib.
The source is split into two separate files, one small neat file which contains the model and the user might like to have a good look at it and get the gist of it even if they're not really a python programmer. And a separate large ugly file which just handles the graphics - an academic would have no interest in this and I'd like to spare them the gory details.
EDIT: I did ask a related question here - but that was all about programmers that won't mind doing things like installing python and using pip... this question is in relation to non-programmers who would not be comfortable doing things like that.
Colab can handle the 2 problems, but you may need to adapt some code.
Matplotlib interface: Colab can display plots just fine. But you may want user to interact with slider, checkbox, dropdown menu. Then, you need to use Colab's own Form UI, or pywidgets. See an example here
2 separate python files: you can convert one of them to a notebook. Then import the other. Or you can create a new notebook that import both files. Here's an example.
This problem involves the collision of several problems, all of which I understand only somewhat well, but I include them together because they could all be the entry point for a solution. Here is the best description I can give.
I have an app, in python. (I imagine I could theoretically solve all of these problems by learning Cocoa and ObjectiveC, but that seems like QUITE a lift, for this problem -- AND, as noted below, this problem may not actually be related to python, really, at all. I just don't know.) A CORE feature of this app is to trigger a minigame, with a hotkey -- meaning, the hotkey itself is fundamental to the desired functionality. And furthermore, I would really like to package this app, to let other people use it. (Locally, it works great! Hey!)
The problem starts with the fact that adding the hotkey -- which I am doing with
import keyboard
keyboard.add_hotkey('windows+shift+y', trigger_minigame)
-- requires root access. Due to DIRE WARNINGS in another SO post Forcing a GUI application to run as root (which, honestly, I only vaguely understand), I would like to grant that access to ONLY this part of the program. I IMAGINE, such an approach would look something like this:
# needs_root.py
import keyboard
from shouldnt_have_root import trigger_minigame
keyboard.add_hotkey('windows+shift+y', trigger_minigame)
# shouldnt_have_root.py
def minigame():
buncha pygame, GUI stuff (which is dangerous???)
def trigger_minigame():
adds event to minigame's event queue
# bash script
sudo python needs_root.py
HOWEVER -- there are several major challenges!
The biggest is that I don't even know if THAT is safe, since I don't know how security and permissions (especially with imports) works at all! And more generally, how dangerous are the imports? It appears that I may in fact have to import substantially more, to make it clear what event queue the trigger is adding an event TO -- and I don't know how to have that communication happen, while still isolating the GUI parts (or generally dangerous ones) from unnecessary and hazardous access.
There's another layer too though; packaging it through pyinstaller means that I can't target the scripts directly, because they'll have been turned into binaries, but according to THIS answer Packaging multiple scripts in PyInstaller it appears I can just target the binaries instead, i.e. have the first binary call
osascript -e 'do shell script "python needs_root_binary" with admin.'
to get the user to bless only the necessary part, but I don't know if that will put OTHER obstacles, or vulnerabilities (or inter-file communication difficulties), in the way.
LAST, I could try STARTING as root, and then switching away from it, as soon as the hotkey is set (and before anything else happens) -- but would that be safe? I'm still worried about the fact that it involves running sudo on the whole app.
In any event --
is this as big a mess as it feels?
How do I give root access to only a piece of a packaged .app, that I've written in python?
I'd advice You to:
enable the root access,
write the script,
disable the root access
as it's closer described in here.
The Pyinstaller is another chapter. When I was making software requiring usage of hotkeys, I was forced to use another than keyboard, because it wasn't working properly on PC without Python, therefore I made a hotkey with tkinter built-in function canvas.bind() (more info here).
Hopefully I helped.
You can not run a specific Python function as root, only the Python process executing your script can be run with elevated permissions.
So my answer is: your problem as described is unsolvable.
I saw this post on Medium, and wondered how one might go about managing multiple python scripts.
How I Hacked Amazon's Wifi Button
This describes a system where you need to run one or more scripts continuously to catch and react to events in your network.
My question: Let's say I had multiple python scripts that I wanted to do run while I work on other things. What approaches are available to manage these scripts? I have to imagine there is a better way than having a large number of terminal windows running each script individually.
I am coming back to python, and have no formal training in computer programming, so any guidance you can provide will be greatly appreciated.
Let's say I had multiple python scripts that I wanted to do run. What
approaches are available to manage these scripts? I have to imagine
there is a better way than having a large number of terminal windows
running each script individually.
If you have several .py files in a directory that you want to run, without having a specific order, you can do:
import glob
pyFiles = glob.glob('path/*.py')
for pyFile in pyFiles:
execfile(pyFile)
Your system already runs a large number of background processes, with output to the system log or occasionally to a service-specific log file.
A common arrangement for quick and dirty deployments -- where you don't necessarily want to invest in making the scripts robust and well-behaved enough to run as proper services -- is to start the script inside screen or tmux. You can detach when you don't need to be looking at it, and can reattach at any time -- even from a remote login -- to view the output, or to troubleshoot.
Take a look at luigi (I've not used it).
https://github.com/spotify/luigi
These days (five years after the question was asked) a lot of people use docker compose. But that's a little heavy weight depending on what you want to do.
I just saw today the script server of bugy. Maybe it might be a solution for you or somebody else.
(I am just trying to find a tampermonkey script structure for python..)
As a result for my graduation paper I am trying to create a new toolbox for ArcGIS using Python scripting. The problem is I am stuck with my code because none of the existing functions in Arcpy does what I need to do. So my question is, is it possible to create a new function in Arcpy or this is restricted to ESRI developers?
Another way to solve this problem would be implement some changes in the tool Cost Distance from Spatial Analyst. So my other question is, do I have access to the coding of the native tools from ArcGIS? And if I have, can I change it to achieve my goal? Or this is also restricted?
Thanks,
Gabriel
You can create your own functions using Python and the Python arcpy site-package. All of ESRI's tools are proprietary, and therefore, most have restricted access. You can check to see if you can edit the tools in the ArcToolbox. For example, you can see the Cost Distance tool is restricted:
While the Spline with Barriers tool can be edited by right-clicking on the script tool.
You can create your own python toolbox for ArcPy following this help: http://resources.arcgis.com/en/help/main/10.2/index.html#//001500000022000000
also checkout the environment variables for your existing tool, it might have some options that you are looking for.
I know this is a year late, but I would like to add a couple ideas to what has been posted for folks like me who are searching for python toolbox help.
For educational purposes, begin by creating a model in Model Builder. This is one way to use ESRI's proprietary tools in new ways. Decide what you want to do and look at ESRI's presence on GitHub. The developers there have a lot of open-source tools ready to use in ArcMap. Here is one such repository: GeospatialPython
Side note, contributing to a repository is a great resume builder.
After creating your working model builder, right click on it in ArcCatalog and select 'export as Python script'. Open the script in your favorite IDE and begin cleaning it up!
Now that you have a python script, it is ready to become a python toolbox. Use gDexter42's link and get to work on that.
My team has some interesting uses for python toolboxes and I am currently creating my very first one.
We use a runner scripts to debug our software. (hard-coded parameters)
We use inheritance for functions that we use over and over again (class BaseToolboxMixin(object):) Stack Exchange Article on Mixins
Most importantly, we have created our own python module around the tool.
The .pyt file we made simply imports arcpy, the module we created, executes the module from a list we created in our 'toolbox_loader.py' file, and has a class that calls the init file that created the module in the first place. >20 lines of code.
As our team creates more tools for the module/python toolbox, we will add them to the list. They will appear inside our toolbox alongside all the ESRI tools. "Seamless integration" was thrown around a lot at the Dev Summit this year.
ESRI is encouraging creativity and open-source usage (check out esri leaflet). I wouldn't constrain my thinking because ESRI's tools are proprietary.
All of this functionality began as a model in ArcMap. Not everyone is going to need to create their own module - complete overkill for most tasks - but it is good to know that the ceiling for Python functionality is high. I am not an experienced developer, but I was able to go from nothing to a functional python toolbox in about 25 man-hours of work. Someone who knows their stuff could do it in a morning.
I am working on cross platform utility which involves drive scanning and the automagical creation of shell/batch files for the OSs upon which it runs. Unfortunately I cannot find one simple answer.
As stated in these links:
When to use os.name, sys.platform, or platform.system?
Reliably detect Windows in Python
Extract file name from path, no matter what the os/path format
Python: What OS am I running on?
and at http://docs.python.org/library/platform.html#module-platform
The various platform.system(), platform.platform(), sys.platform, os.name() etc. etc. all suffer the problem of not necessarily being future perfect. That is if an OS developer changes things a little, these may not work (at least until patched or revised). So obviously the best solution is to try a small part of each of the above, along with targeting some OS specific executable file with a call().
Which leaves my question:
Since the best way to determine this involves platform.system, sys.platform, and os.name (assuming only generalized recognition is needed), what are the various possible outputs for those programs? The docs.python.org sections on each of these modules only lists a few, and the pages are not exactly current. Specifically I would like to know the possible output on the last three mac OS's, Win XP- Win 8, and just knowing Linux covers my needs there. Any one know what the outputs are or where i can find them?
Thanks in advance.
Clarification:
What I am looking for here is the currently known values so that I can incorporate them into an existing project, with an eye towards future code revision being made easier on my end. So the CURRENT return values are what I am seeking (Last 3 gens of Mac OS* and Win * since beyond that probably isn't much used any more)
Edit: For the specific question of all possible return values:
Related stackoverflow answer for Possible values from sys.platform?
Post with the answers pointed to above:
aix3 aix4 atheos beos5 darwin freebsd2 freebsd3 freebsd4 freebsd5
freebsd6 freebsd7 generic irix5 irix6 linux2 mac netbsd1 next3 os2emx
riscos sunos5 unixware7
Also:
linux3, freebsd8, win32, dos, os2
and others.
(They were asking the same question in Aug 2006.)
Note:
As others have indicated, sys.platform is derived from the name that
the system vendor gives their system.
/Edit.
Not sure it's possible to have something so future-perfect. If you only want to know the OS (mac/win/linux) then see the examples for sys.platform.
Also, keeping things in a separate function like get_os_name lets you control or map the input-output if you see a lot of changes in the future - once a year per OS? Also convenient to combine with the other functions you've mentioned. So you can return a tuple based on (os_name, 32/64bit, variant) (where variant is things like XP, Win8, Darwin, etc.) depending on how it affects your script.
The docs.python.org sections on each of these modules only lists a few, and the pages are not exactly current.
Unfortunately true. But again, it's impossible to account for environments or platforms in the future or even all current ones. The logical thing to do is make sure it works on the current platforms that you have tested/developed for.