I am quite new with configuring Jenkins or Python but I have to set up a unitary test in Jenkins. My program is in Python, but only works on Python 2.6 whereas the Jenkins version I should be using is 2.7, so I'm trying to set up Jenkins to set some environment variables so that it prepares launching the accurate Python for that specific test (it is part of a greater project that will successfully run several other tests that work well).
The idea I had was to set in the command to execute several environment variables like PATH, LD_LIBRARY_PATH and PYTHONPATH such as following in the "Execute shell" command line interpreter:
PYTHONPATH=/path/to/python2.6/lib:$PYTHONPATH
PATH=/path/to/python2.6/bin:$PATH
LD_LIBRARY_PATH=/path/to/python2.6/lib:$LD_LIBRARY_PATH
... however, it was still calling the wrong version of Python. Therefore, I forced these variables to:
PYTHONPATH=/path/to/python2.6/lib
PATH=/path/to/python2.6/bin
LD_LIBRARY_PATH=/path/to/python2.6/lib
... and I still get errors because the old version of Python is called instead, even if it should not appear in the PATH ... It appears Jenkins will remember the location of the old libraries however and will try loading them first.
How would I correctly set the environment in a "subproject" in Jenkins so that I can call a different version of Python?
Thank you and best regards,
~Stéphane
If you want your program to run with a specific version of the python interpreter, you indicate it in the shebang
#!/usr/bin/python2.6
#your code here
What i did in my Jenkins shell script using a specific python version was something like this when calling my unit test:
python3 src/test/unit_test.py
I was using it to use Python 3.X but it should work with 2.6 as well using:
python2.6 src/test/unit_test.py
Stupid me... I was indeed doing things correctly, I just had a part of my code that was overriding the PYTHONPATH value, so the solution I had found previously was good.
FYI, I modified my shebang, if it's of any help to anyone ;)
Related
I have a CodeDeploy which deploys application on Windows instances. I have a Python script which is running as part of ValidateService hooks. Below is the code I have in that script:
print("hello")
So, I have removed everything and just printing hello as part of this script. When this script is called by CodeDeploy I get below error:
My appspec.yml file:
...
ValidateService:
- location: scripts/verify_deployment.py
timeout: 900
I tried getting some help on Google but got nothing. Can someone please help me here.
Thanks
As Marcin already answered in a comment, I don't think you can simply run python scripts in CodeDeploy. At least not natively.
The error you see means that Windows does not know how to execute the script you have provided. AFAIK Windows can't run python natively (like most linux distros can).
I am not very accustomed to CodeDeploy, but given the example at https://github.com/aws-samples/aws-codedeploy-samples/tree/master/applications/SampleApp_Windows, I think you have to install python first.
After so much of investigations, I found my answer. The issue is little misleading, there is nothing to do with Code format or ENOEXEC. The issue was due to Python path. While executing my script, CodeDeploy was unable to find Python (Though I had already added python.exe in Environment variable path).
Also, I found that CodeDeploy is unable to execute .py file due to Python path issue. So, I created a PowerShell script and invoking Python script from there. Like below:
C:\Users\<username>\AppData\Local\Programs\Python\Python37-32\python.exe C:\Users\<username>\Documents\verify_deployment.py
It executed Python script successfully and gave me below output:
hello
I am trying to retrieve the parameters set from the jenkins build into my python script, but am having trouble. I understand the parameters set from here:
Are set as env variables and all I have to do in python is do:
# Env variables
UPDATE_DATA = os.environ.get('update_data')
ALL_BUILDS = os.environ.get('all_builds')
However I am getting None for those values. When I do an echo of those parameters in my jenkins script before my python script runs, I could see them being printed out correctly. However, for some reason python does not see them. If I go manually into a terminal and export a variable and run my python script, it works.. So I'm completely lost here.
Jenkins server is running on linux. Using python 2.7
You can use the boolean variable like this:
Output:
It seems like when I ran the python script in the Jenkins config (not inside a file within my project) like how #souravatta suggested, it found the env variable. So that means the env variable Jenkins is setting, is on a different instance somehow (even though they are on the same computer, same user). I just did a workaround where I wrote the env variables to a file and then just read that file in my python script.
I recently found a fix for Python getpass not working on Windows: Python not working in the command line of git bash
Or at least that was the last thing I remember about changing my python configurations. (This is for Python 3.6.1 on Windows 10)
Now I also use Python to other tasks which simply has subprocess calls to type several commands on terminal:
go build ./folder/
mv ./src/ ./bin/
I get the error: go: GOPATH entry is relative; must be absolute: "/c/Users/OP/work". But I don't get it if I type go build ./src/folder myself.
I have GOPATH set to C:\work in Environment Variables. I have tried with a ;.
Is there a way to reverse the alias python every time? Or what is happening exactly when setting an alias for python to winpty?
I'm thinking that when I call go build directly, it is called by either my user profile or system. And when python's subprocess calls it, it calls the opposite. Therefore, I have two GOPATH variables even though I have only 1 set in environment variable.
Side Note: another recent change on GOPATH was changing it from C:/go because it couldn't be the same as GOROOT. That error popped up randomly for some reason. It worked with that setting for a while and I don't remember changing anything before except adding another import package on top of the many other ones already being used.
Update: with type python I get the result: python is aliased to 'winpty python.exe'. Therefore I tried to undo that with unalias python. The new result I get is: python is hashed (/c/Users/OP/AppData/Local/Programs/Python/Python36/python).
This fixed the go build command within Python's subprocess. However, that alias was a fix for another Python issue with using getpass package.
On top of my unalias python fix, I also discovered something interesting: when I change the environment variables for GOPATH from C:\work; to C:\go, all go commands still spit the error go: GOPATH entry is relative; must be absolute: "". I got this same error (but different path) from updating Windows 10 Fall Creators update. Maybe it is related.
Simply closing MINGW and reopening it fixed the issue. So perhaps it was saying a copy of my environment variables and using that as a reference instead of the actual system properties.
I know this is not a popular question, but someone could benefit from my hours of investigating and debugging.
Under Windows you must use a Windows style GOPATH like eg d:\code and probably you should use cmd shell and nothing else. Unfortunately cygwin paths (and probably others too) do no longer work especially for go get reaching out to git.
Stick to Windows paths and Windows shell.
I'm attempting to run python on a system that doesn't allow me to set environment variables. Is there a commandline flag to python that will set PYTHONHOME? I looked here: http://docs.python.org/release/2.3.5/inst/search-path.html but didn't see anything.
So, hopefully something like this:
python -magical_path_flag /my/python/install test.py
EDIT
Thanks for the responses everyone. I'm embarrassed to say I actually meant PYTHONHOME, not PYTHONPATH. (That's what I deserve for asking a question at 1:30 AM.) I've edited my quesiton.
Here's some more info. I'm trying to get python running on Android. I can run python -V no problem, but if I try and execute a script, I get:
I/ControlActivity(18340): Could not find platform independent libraries <prefix>
I/ControlActivity(18340): Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Unfortunately when using the ProcessBuilder and changing the environment variables on Android, it says that they're not modifiable and throws an exception. I'm able to pass all the command line flags I want, so I was hoping I could set PYTHONHOME that way.
I've tried creating a wrapping shell script which exports PYTHONHOME and then calls python but that didn't work. (Got the same error as before.)
Thanks,
Gabe
You could simply set it in your script -- sys.path is a regular, modifiable list. Something like:
import sys
sys.path.append("/path/to/libraries")
should do the trick
In UNIXy shells, you can set an environment variable just for the duration of one command by prepending the command with the environment variable setting:
$ PYTHONPATH=/my/python/install python test.py
If none of the other answers suit you, you can write a wrapper script that temporarily sets the environment variable then exec's your other script. For example:
python mywrapper.py -magical_path_flag /my/python/install test.py
The story began with a very strange error while I was running my script from PyDev. Running the same script from outside will not encounter the same problem.
Fatal Python error: Py_Initialize: can't initialize sys standard streams
File "C:\Python26\lib\encodings\__init__.py", line 123
raise CodecRegistryError,\
^
SyntaxError: invalid syntax
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
I was able to find why this is happening: In PyDev I use two different Python versions: 3.1 that is the default installation and 2.6 as the alternative one.
My Windows Environment does not contains PYTHONHOME, CLASSPATH, PYTHONPATH but PyDev does add them.
Now the problem is at one stage my python script does execute another python script using os.system(python second.py) and the second script will fail with the above error.
Now I'm looking to find a way to prevent this issue, issue that is happening because it will run the execute the default python using the settings for the non-default one (added by PyDev).
I do not want to change the standard call (python file.py) but I want to be able to run my script from pydev without problem and being able to use default or alternative python environment.
Any ideas?
I found a solution that seams acceptable specially because it will not interfere with running the scripts on other systems, just to run python -E second.py - this will force Python to ignore PYTHON* environment variables.
I may not be understanding this quite right, but I think you're invoking a script from pydev that works okay, but this script executes another script which requires a different version.
While this would unfortunately be installation-specific, you could use os.system("c:\absolute\path\to\proper\version\of\python.exe second.py").
If PyDev is setting up conflicting environmental variables, you may want to look into subprocess over os.system.
http://docs.python.org/library/subprocess.html#using-the-subprocess-module
This will allow you to invoke a process with a handle, so you can optionally wait for it to terminate. It will also allow you to pass environment variables upon execution.
I believe your call should be:
import sys
os.system(sys.executable+ ' second.py')
So that you guarantee you're using the same interpreter you're currently running and not launching the other one (or did you really mean to use the other interpreter?)