Develop Reference

How to prepare PATH when embedding Python in C++?

I am implementing an interface which allows Python to be loaded and used from a Windows C++ application. It should be able to use any Python registered in the Windows registry. This seems to work fine with e.g. Python 3.8 downloaded from www.python.org. However, with current Anaconda/Python 3.7 I am having problems. It looks like it expects a lot of directories to be present at PATH, and this knowledge seems to be hidden in a cryptic conda.bat file. Especially the folder C:\ProgramData\Anaconda\Library\bin seems critical, without this e.g. numpy won't load.
In short, assuming that I start just with the following info piece from registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\3.7\InstallPath]
#="C:\\ProgramData\\Anaconda3"
how should I prepare PATH and potentially other things so that the loaded Python DLL, from Anaconda or not, would be fully functional? For this particular Anaconda version I can add them by hand, but this seems not so sustainable.
I know that one solution is to select a check box during Anaconda install which will put the needed directories to the system PATH. However, this check box is not selected by default and is marked as "not recommended", so there is not much hope that our end users will actually check it.

Jupyter running different version of uninstalled library?

So I was trying to modify an existing library and instead of doing it the smart way and using pip -e I instead just installed the libraries, then swapped the modified files for whatever changes I wanted. For example if I had:
Library A/
---doSomethingA.py
---otherFiles.py
I just deleted doSomethingA.py and replaced it with my version of doSomethingA.py. Theoretically I figured, because im editing the file locally, it should still work as planned for my library with whatever extra functionality I want.
HOWEVER.... its basically going crazy. While I can see my edited changes in the file, when I run the library its obviously not running that file. I did things like:
commenting out the whole file (still runs somehow)
Actually uninstalling the library and part of another script using doSomethingA.py it still runs?? (i.e Something like import libraryA works on JupyerHub, but not on the putty terminal... ?)
I've obviously come to the conclusion that its not running the file that it says that it is (and trust me I've checked the path of the file like 10 times).
My question is:
How is this possible? What are the places that python would store another copy of the file etc?
I've also deleted the __pychache__, but I can't think of anything else to do. Is my best option to just give up and create a new virtual environment, etc?

I understand that you are running on jupyter hub.
This means that your python is running remotly on the server and the framework is taking care to synch your local project (but not the installed libraries).
The python on the server is not aware of your local change.
As a temporary mitigation you can copy the installed library to you project root.

Jenkins shared library: How to run code from a python module contained in a package?

I want to run a python script as part of a jenkins pipline triggered from a github repo. If I store the script directly in the repo itself, I can just do sh 'python path/to/my_package/script.py' which work perfectly. However since I want to use this from multiple pipelines from multiple repos, I want to put this in a jenkins shared library.
I found this question which suggested storing the python file in the resources directory and copying it to a temp file before use. That only works if the script is one standalone file. Unfortunately, mine is a package with multiple python files and imports between them, so thats a no go. I also tried to copy the entire folder containing the python package from the answer to this question, which suggests getting the location of the library with
import groovy.transform.SourceURI
import java.nio.file.Path
import java.nio.file.Paths
class ScriptSourceUri {
#SourceURI
static URI uri
}
but its gives me the following error:
Scripts not permitted to use staticMethod java.net.URI create java.lang.String. Administrators can decide whether to approve or reject this signature.
It seems that some additional permissions are required, which I don't think I'll be able to acquire (its a shared machine).
So? Does anyone know how I can run a python package from jenkins shared library? Right now the only solution I can think of is to manually recreate the directory structure of the python package, which is obviously very messy and non-generic.
PS: There is no particular reason for using the python script over writing the same script in groovy. Its just that the python script is well tested, well understood and well supported. Rewriting the whole thing in groovy just isn't feasible right now.

You can go to http://host:8080/jenkins/scriptApproval/ page of your Jenkins installation and approve the request for your scripts, please see below:-
And follow the link for more information.

Out of home folder .pyc files?

If I place my project in /usr/bin/
will my python interpreter generate bytecode? If so where does it put them as the files do not have write permission in that folder. Does it cache them in a temp file?
If not, is there a performance loss for me putting the project there?
I have packaged this up as a .deb file that is installed from my Ubuntu ppa, so the obvious place to install the project is in /usr/bin/
but if I don't generate byte code by putting it there what should I do? Can I give the project write permission if it installs on another persons machine? that would seem to be a security risk.
There are surely lots of python projects installed in Ubuntu ( and obviously other distros ) how do they deal with this?
Thanks

Regarding the script in /usr/bin, if you execute your script as a user that doesn't have permissions to write in /usr/bin, then the .pyc files won't be created and, as far as I know, there isn't any other caching mechanism.
This means that your file will be byte compiled by the interpreter every time so, yes, there will be a performance loss. However, probably that loss it's not noticeable. Note that when a source file is updated, the compiled file is updated automatically without the user noticing it (at least most of the times).
What I've seen is the common practice in Ubuntu is to use small scripts in /usr/bin without even the .py extension. Those scripts are byte compiled very fast, so you don't need to worry about that. They just import a library and call some kind of library.main.Application().run() method and that's all.
Note that the library is installed in a different path and that all library files are byte compiled for different python versions. If that's not the case in your package, then you have to review you setup.py and your debian files since that's not the way it should be.

.pyc/.pyo files are not generated for scripts that are run directly. Python modules placed where Python modules are normally expected and packaged up have the .pyc/.pyo files generated at either build time or install time, and so aren't the end user's problem.

Trac Plugin Not working

At work we are using Trac on several internal wiki's and an external wiki. REcently we found the need for a new plugin. After we going through a few tutorials we went to install a plugin to make sure it would work. It didn't. We've been going through trying to figure out. Below I will list the steps and various things I did while trying to get it to work.
1) I went to trac-hacks website and downloaded their hellow world plugin, figured I couldn't make a mistake using their code.
2) I compiled and made an egg using python setup.py bdist_egg on the machine where trac is installed, to make sure it's the same Python version being used.
3) I then copied it over to /directory/where/trac/is/plugins/ folder and chmod 755 the file egg file.
4) I then restarted http, unable to find a better way of restaring trac so this may be where my problem is. It didn't work. So I deleted the egg folder in plugins
5) Uploaded it via trac->administration->plugins and restarted httpd again. Nothing.
6) I realized I had to edit the trac.ini file and added helloworld.* = enabled under component and restarted the web server.
It's quite possible it's me but any help would be greatly appreciated!
Its the helloworld plugin from trachack, essentially displays hello world and theres a button. There are no error messages provided, hence why googling was hard.
I'm assuming that it's using root and that's the user I built it with. I will look into seeing if it's anybody else, just taking a quick look though I don't see anything else that could be using it. I only copied the egg file to the plugins folder, I set up another folder elsewhere and built it and cp to the plugins folder. I'm glad to know I was doing that right because looking up documentation on how to restart trac turns up practically nothing, they just say restart trac or restart apache. I will look into the logs later on tomorrow. Thanks for the replies! Also we are using trac .12.1.
So after looking at the log files it seems that it doesn't even load the plugin, can't find anywhere that says it's loading or any errors with it. Now we have a few trac sites for various projects and one of the sites already has plugins installed so I went there and and put the test plugin there and checked logs and it didn't work either. So I'm just going to conclude it's the plugin or something we already have in place and it's not me. I believe I'm going to try and make one and test it. Thanks for the help!

It sounds like you built the egg correctly. After you copy it into your plugins folder, change the file's owner and group (I'm assuming you're on Linux since you mentioned chmod) to match the account that your webserver uses. I'm not sure if that's strictly necessary, but it's what's always worked for me.
I may be misreading your #4, but it sounds like you copied the whole egg folder to your plugins directory. Only the .egg file needs to be copied over, it's a self-contained package. I don't think Trac looks for .egg files in subdirectories.
Restarting your webserver is the easiest way to restart Trac. Actually, I'm not aware of any other way to do it.
When it comes to plugin problems, Trac's log is usually a very good source of information. I recommend setting Trac's log level to DEBUG, then shut down the web server. Clear out the contents of Trac's log file, then start the web server and make a copy of Trac's log file after the server has completely come back online. Do this process twice: once with the plugin installed and once without it installed. The difference in the logfiles should give you a good indication of what the problem is. Once you get accustomed to what your logs normally look like, you'll be able to read the log in place without clearing it out and generating two versions of it.
By the way, what Trac version are you using?

Check the Trac version and downloaded plugin
instead python setup.py bdist_egg try python setup.py install

Quite an old thread, but since I ran into the same problem at one point:
Make sure you build the .egg with the same Python version that you use to run Trac with!Backwards compatibility between Python versions does not matter here, because Trac reads information about the Python version out of the .egg file before it even loads it, to make sure it is compatible.
(Small version numbers should not matter, so you should be able to run a .egg with Python 2.7.10 when it was built with 2.7.3, but not when it was built with 2.6.x. Look at the Version number that is written into the .egg file name.)