I'd like to (PGP/GPG) sign python code. Yes, I have read this and many other sites that talk about protecting and obfuscating python code - this all is not what I want. I DON'T want to obfuscate code.
I want customers and users to see the code, they could modify code, copy it and make derivative work, I'd like to have the software under the GPLv3.
But I want to have plugins that are "signed", so they can be kind of trusted during execution.
Is this possible in Python? Can I import a library after checking its gpg signing?
What would be easy: check the gpg signing of a file, and then load it via import, else raise an exception. But this only would be possible for single-file-imports, not directory python modules.
It is clear that, if the customer changes the GPG key in the program, or deletes some lines himself in the checking algorithm, all is gone - but this is not the problem.
He could do anything he wants - but this would be silly.
What he wants is trustworthiness.
I want to let him add a third party plugin by copying it into a "plugins" directory, and have the program check the plugin for "trustworthiness" - and then import it.
(So he could run plugins that are not signed, but with his own risk.)
Python's import mechanism already provide all the tools necessary to achieve what you want. You can install different kinds of import hooks in order to support what you want.
In particular you'll probably find convenient to install a meta path hook that searches for "signed modules" and returns a Loader that is able to perform the imports from this signed format.
A very simple and convenient format for your signed plug-ins would be a zip archive containing:
The code of the plug-in in the form of modules/packages
A PGP signature of the above code
In this way:
Your loader should unpack the zip, and check the signature. If it matches then you can safely load the plug-in, if it doesn't match you should ask the user to trust the plug-in (or not and abort)
If the user wants to modify the plug-in it can simply unpack the zip archive and modify it as he wishes.
Imports from zip archives are already implemented in the zipimport module. This means that you don't have to rewrite a loader from scratch.
Actually if you want to reduce the code for the hooks to the minimum you'd simply need to verify the signature and then add the path to the zip archive into sys.path, since python already handles imports from zip archive even without explicitly using zipimport.
Using this design you just have to install these hooks and then you can import the plug-in as if they were normal modules and the verification etc. will be done automatically.
I know this is an old post, but we've developed a new solution. We were confronted with the same challenge -- to distribute python source code, but to prevent hackers from tampering with the code. The solution we developed was to create a custom loader for our application using signet http://jamercee.github.io/signet/.
What signet does is scans your script and it's dependencies creating sha1 hashes. It embeds these hashes into a custom loader which you deliver to your customer with your script. Your customers run the loader which re-verifies the hashes before it transfers control to your script for normal execution. If there's been tampering it emits an error message, and refuses to run the tampered code.
Signet is multiplatform and runs on windows, unix, linux, freebsd, etc... If you deploy to windows, the loader building process can even apply your company code certificate for 100% verification of your code. It also does PE verification.
The code is fully open source including the c++ source code to the default loader template. You can extend the loader to do additional verifications and even take actions if it detects code tampering (like undoing the tampering...).
Related
I am executing python scripts using python embedding (python.net in C#), I need to make sure these python scripts aren't going to be tampered with. These python scripts can be in a .pyc (compiled) format.
Is there a way to make these scripts tamper-proof? .pyc files can be easily decompiled, tampered with and recompiled. I looked at signet but I believe it requires the python files to be frozen into an executable.
Any help will be welcome.
What you need is signing these scripts. Signing a file means producing a signature from a private key and that file, the idea being that it's impossible to produce that same signature without the private key. Then, you also have a public key (that can be made public), whose only purpose is to verify that the signature matches the file. IIRC, this is the same mechanism that Windows uses to trust software (ie. there are software developers who are trusted by Windows, and when a software has a signature issued by a trusted source, which Windows can verify, then it is considered as trusted software, I think).
This is quite a common cryptographic pattern, so I think there are many tools that implement it, but one that is particularly good is GPG. It's free and open-source, it has bindings in many languages, it is very well documented, and handles everything, from the creation of your key pair to the signing, and much more. This also mean that using GPG is a little bit complex, but I found this post where someone wanted to verify a file they download in C# using GPG, so maybe it's going to be helpful.
Also, notice that in that in the other post they also use a hash check to ensure that the script was not corrupted during download (ie. there was a download error). You could verify that with the signature, but then you would be unable to tell if the error comes from someone deliberately attempting to tamper with your code, or if you just need to re-download the script.
I'd like to (PGP/GPG) sign python code. Yes, I have read this and many other sites that talk about protecting and obfuscating python code - this all is not what I want. I DON'T want to obfuscate code.
I want customers and users to see the code, they could modify code, copy it and make derivative work, I'd like to have the software under the GPLv3.
But I want to have plugins that are "signed", so they can be kind of trusted during execution.
Is this possible in Python? Can I import a library after checking its gpg signing?
What would be easy: check the gpg signing of a file, and then load it via import, else raise an exception. But this only would be possible for single-file-imports, not directory python modules.
It is clear that, if the customer changes the GPG key in the program, or deletes some lines himself in the checking algorithm, all is gone - but this is not the problem.
He could do anything he wants - but this would be silly.
What he wants is trustworthiness.
I want to let him add a third party plugin by copying it into a "plugins" directory, and have the program check the plugin for "trustworthiness" - and then import it.
(So he could run plugins that are not signed, but with his own risk.)
Python's import mechanism already provide all the tools necessary to achieve what you want. You can install different kinds of import hooks in order to support what you want.
In particular you'll probably find convenient to install a meta path hook that searches for "signed modules" and returns a Loader that is able to perform the imports from this signed format.
A very simple and convenient format for your signed plug-ins would be a zip archive containing:
The code of the plug-in in the form of modules/packages
A PGP signature of the above code
In this way:
Your loader should unpack the zip, and check the signature. If it matches then you can safely load the plug-in, if it doesn't match you should ask the user to trust the plug-in (or not and abort)
If the user wants to modify the plug-in it can simply unpack the zip archive and modify it as he wishes.
Imports from zip archives are already implemented in the zipimport module. This means that you don't have to rewrite a loader from scratch.
Actually if you want to reduce the code for the hooks to the minimum you'd simply need to verify the signature and then add the path to the zip archive into sys.path, since python already handles imports from zip archive even without explicitly using zipimport.
Using this design you just have to install these hooks and then you can import the plug-in as if they were normal modules and the verification etc. will be done automatically.
I know this is an old post, but we've developed a new solution. We were confronted with the same challenge -- to distribute python source code, but to prevent hackers from tampering with the code. The solution we developed was to create a custom loader for our application using signet http://jamercee.github.io/signet/.
What signet does is scans your script and it's dependencies creating sha1 hashes. It embeds these hashes into a custom loader which you deliver to your customer with your script. Your customers run the loader which re-verifies the hashes before it transfers control to your script for normal execution. If there's been tampering it emits an error message, and refuses to run the tampered code.
Signet is multiplatform and runs on windows, unix, linux, freebsd, etc... If you deploy to windows, the loader building process can even apply your company code certificate for 100% verification of your code. It also does PE verification.
The code is fully open source including the c++ source code to the default loader template. You can extend the loader to do additional verifications and even take actions if it detects code tampering (like undoing the tampering...).
Would it be possible to create a python module that lazily downloads and installs submodules as needed? I've worked with "subclassed" modules that mimic real modules, but I've never tried to do so with downloads involved. Is there a guaranteed directory that I can download source code and data to, that the module would then be able to use on subsequent runs?
To make this more concrete, here is the ideal behavior:
User runs pip install magic_module and the lightweight magic_module is installed to their system.
User runs the code import magic_module.alpha
The code goes to a predetermine URL, is told that there is an "alpha" subpackage, and is then given the URLs of alpha.py and alpha.csv files.
The system downloads these files to somewhere that it knows about, and then loads the alpha module.
On subsequent runs, the user is able to take advantage of the downloaded files to skip the server trip.
At some point down the road, the user could run a import magic_module.alpha ; alpha._upgrade() function from the command line to clear the cache and get the latest version.
Is this possible? Is this reasonable? What kinds of problems will I run into with permissions?
Doable, certainly. The core feature will probably be import hooks. The relevant module would be importlib in python 3.
Extending the import mechanism is needed when you want to load modules that are stored in a non-standard way. Examples include [...] modules that are loaded from a database over a network.
Convenient, probably not. The import machinery is one of the parts of python that has seen several changes over releases. It's undergoing a full refactoring right now, with most of the existing things being deprecated.
Reasonable, well it's up to you. Here are some caveats I can think of:
Tricky to get right, especially if you have to support several python versions.
What about error handling? Should application be prepared for import to fail in normal circumstances? Should they degrade gracefully? Or just crash and spew a traceback?
Security? Basically you're downloading code from someplace, how do you ensure the connection is not being hijacked?
How about versionning? If you update some of the remote modules, how can make the application download the correct version?
Dependencies? Pushing of security updates? Permissions management?
Summing it up, you'll have to solve most of the issues of a package manager, along with securing downloads and permissions issues of course. All those issues are tricky to begin with, easy to get wrong with dire consequences.
So with all that in mind, it really comes down to how much resources you deem worth investing into that, and what value that adds over a regular use of readily available tools such as pip.
(the permission question cannot really be answered until you come up with a design for your package manager)
I have a python modules question:
I need to write a script that will take the output from various pieces of hardware and then store them in a database. I know about the current hardware but I do not know about any additional hardware that will be added in the future. Sorry let me be more specific.
The script will retrieve power information from hardware modules connected to power distribution boards. These modules take the raw sensor information and make it available over tcp/ip. I have written functions into the test script that interrogates the hardware modules and gathers the info. The problem is that this will need to be deployed where different hardware modules will be used, not just the ones that I know about. All the hardware modules will make the same info available but they will all do it different ways. Some use telnet or ssh, some provide a web page, some provide and xml output, some will use snmp. I want to create a mechanism that a python module can be written for a new device that pulls out the info and provides it to the script in a standard way so that the script can populate the database and life goes on as normal.
So Instead of adding a new function to the script each time a new hardware module type is encountred and then wrapping it in 'if' statements etc... is there a mechanism to offboard that function into a module that can be dynamically added to the script. A plugin type mechanism comes to mind I suppose. But I don't think that quite fits.
Any suggestions or directions to places where something like this has been used or implemented would be greatly appreciated.
You could use import function in your code. Depending on your Python version, try importlib (Added in Python 2.7) or __import__ function.
You could create plugins for each hardware device you're going to support. Each such plugin could then register itself somewhere (main module path put in Environment variable or known path in Windows registry). Your code would then check that location (periodically, on startup - depending on your needs) and load each plugin found. You could also use distutils and have plugins packed as egg packages.
Please note that while it works quite well when you're running from Python sources, it's getting more complex if you plan to build executables with PyInstaller or Py2exe.
Currently, I'm deploying a full python distribution (the original python 2.7 msi) with my app. Which is an embedded web server made with delphi.
Reading this, I wonder if is possible to embed the necessary python files with my app, to decrease load files and avoid conflict with several python versions.
I have previous experience with python for delphi so I only need to know if only shipping the python dll + zip with the distro + own scripts will work (and if exist any caveats I must know or a sample where I can look)
zipimport should work just fine for you -- I'm not familiar with Python for Delphi, but I doubt it disables that functionality (an embedding application can do that, but it's an unusual choice). Just remember that what you can zip up and import directly are the Python-coded modules (or just their corresponding .pyc or .pyo byte codes) -- DLLs (even if renamed as .pyds;-) need to be on disk to be loaded (so if you have a zipfile with them it will need to be unzipped at the start of the app, e.g. into a temporary directory).
Moreover, you don't even need to zip up all modules, just those you actually need (by transitive closure) -- and you can easily find out exactly which modules those are, with the modulefinder module of the standard Python library. The example on the documentation page I just pointed to should clarify things. Happy zipping!
Yes it is possible.
I'm actually writing automatisation script in Python with the Zipimport library. I actually included every .py files in my zip as well as configuration or xml files needed by those script.
Then, I call a .command file targeting a __main__.py class that redirect towards the desired script according to my sys.argv parameters which is really useful!