Develop Reference

Using Jenkins variables/parameters in Python Script with os.path.join

I'm trying to learn how to use variables from Jenkins in Python scripts. I've already learned that I need to call the variables, but I'm not sure how to implement them in the case of using os.path.join().
I'm not a developer; I'm a technical writer. This code was written by somebody else. I'm just trying to adapt the Jenkins scripts so they are parameterized so we don't have to modify the Python scripts for every release.
I'm using inline Jenkins python scripts inside a Jenkins job. The Jenkins string parameters are "BranchID" and "BranchIDShort". I've looked through many questions that talk about how you have to establish the variables in the Python script, but with the case of os.path.join(),I'm not sure what to do.
Here is the original code. I added the part where we establish the variables from the Jenkins parameters, but I don't know how to use them in the os.path.join() function.
# Delete previous builds.
import os
import shutil
BranchID = os.getenv("BranchID")
BranchIDshort = os.getenv("BranchIDshort")
print "Delete any output from a previous build."
if os.path.exists(os.path.join("C:\\Doc192CS", "Output")):
shutil.rmtree(os.path.join("C:\\Doc192CS", "Output"))
I expect output like: c:\Doc192CS\Output
I am afraid that if I do the following code:
if os.path.exists(os.path.join("C:\\Doc",BranchIDshort,"CS", "Output")):
shutil.rmtree(os.path.join("C:\\Doc",BranchIDshort,"CS", "Output"))
I'll get: c:\Doc\192\CS\Output.
Is there a way to use the BranchIDshort variable in this context to get the output c:\Doc192CS\Output?

User #Adonis gave the correct solution as a comment. Here is what he said:
Indeed you're right. What you would want to do is rather:
os.path.exists(os.path.join("C:\\","Doc{}CS".format(BranchIDshort),"Output"))
(in short use a format string for the 2nd argument)
So the complete corrected code is:
import os
import shutil
BranchID = os.getenv("BranchID")
BranchIDshort = os.getenv("BranchIDshort")
print "Delete any output from a previous build."
if os.path.exists(os.path.join("C:\\Doc{}CS".format(BranchIDshort), "Output")):
shutil.rmtree(os.path.join("C:\\Doc{}CS".format(BranchIDshort), "Output"))
Thank you, #Adonis!

Python script keeps running when using pyRserve

I am trying to learn how to send a list of lists in Python to R -script which runs statistical methods and gives two or three data frames back to Python
I stumbled across the pyRserve package. I was able to follow the manual in their documentation and everything works great in command line (>>> ). When I run a script, it does not stop. I have installed Rserve package and started its service in RStudio. Below is the code:
import pyRserve
print "here1" #prints this line...
conn = pyRserve.connect(host='localhost', port=6311)
print "here2"
a= conn.eval('3+5')
print a
Can anyone please help?

The (docs) suggest:
$ python
>>> import pyRserve
>>> conn = pyRserve.connect()
And then go on with:
To connect to a different location host and port can be specified explicitly:
pyRserve.connect(host='localhost', port=6311)
This is not meant to indicate that both lines should be run. The second line should be viewed as a potential modifier for the first. So if you need an alternate address or port, then it should look like:
$ python
>>> import pyRserve
>>> conn = pyRserve.connect(host='localhost', port=6311)
Also note this caveat for windows users:
Note On some windows versions it might be necessary to always provide ‘localhost’ for connecting to a locally running Rserve instance.

How to debug / correctly setup git-multimail

I would like to use git-multimail as post receive hook in one of my git repositories (no gitolite used). Unfortunately, I cannot get it work, and I have hardly any experience using Python.
What I did so far:
I added the following block to the project.git/config file:
[multimailhook]
mailingList = email#example.com
from = email#example.com
envelopeSender = email#example.com
mailer = smtp
smtpServer = smtp.mydomain.com
smtpUser = myUser
smtpPass = myPassword
Please note that I do not know whether "smtp", which is defined in the mailer variable, is installed on my machine.
I copied the current git_multimail.py file into project.git/hooks.
I created a project.git/hook/post-receive file with the following content. The file is executable, I copied this from https://github.com/git-multimail/git-multimail/blob/master/git-multimail/post-receive.example
#! /usr/bin/env python
import sys
import os
import git_multimail
config = git_multimail.Config('multimailhook')
try:
environment = git_multimail.GenericEnvironment(config=config)
#environment = git_multimail.GitoliteEnvironment(config=config)
except git_multimail.ConfigurationException:
sys.stderr.write('*** %s\n' % sys.exc_info()[1])
sys.exit(1)
mailer = git_multimail.choose_mailer(config, environment)
git_multimail.run_as_post_receive_hook(environment, mailer)
What happens:
When I push a change, a file project.git/hooks/git_multimail.pyc is created, but no email is sent.
Doing a configuration test using GIT_MULTIMAIL_CHECK_SETUP=true python git_multimail.py as described on https://github.com/git-multimail/git-multimail/blob/master/doc/troubleshooting.rst tells me that git-multimail seems properly set up
Is there a way to log something like an output of the script? What can I do to find out what is not working? Are there even errors in my files?
Thanks in advance.

Using post-receive.example is by far not the simplest way to set up git_multimail.py: as the header of post-receive.example script says:
The simplest way to use git-multimail is to use the script
git_multimail.py directly as a post-receive hook, and to configure it
using Git's configuration files and command-line parameters.
In other words, just
cp /path/to/git_multimail.py /path/to/project.git/hooks/post-receive
and you're all set (since you already have project.git/config filled-in). See Invocation in git-multimail's README for a bit more detail.
(note: admitedly, the doc is not so clear for beginners, I'll try to improve that when I get time)

OK guys, the error was probably as little as it could be. I did just one very little mistake in the post receive hook file: The sys.exit(1) command is not indented.
So, the WRONG version from my question:
try:
environment = git_multimail.GenericEnvironment(config=config)
except git_multimail.ConfigurationException:
sys.stderr.write('*** %s\n' % sys.exc_info()[1])
sys.exit(1)
CORRECT is (compare last line):
try:
environment = git_multimail.GenericEnvironment(config=config)
except git_multimail.ConfigurationException:
sys.stderr.write('*** %s\n' % sys.exc_info()[1])
sys.exit(1)
Like I said, I hardly know Python, so I did not pay attention to the indents. After correcting this, the email was sent, so feel free to use the above steps as a little tutorial for setting up git-multimail the "easiest" way. (I did not find a tutorial for this exact solution.)

Python Fabric as a Library, execute and environments?

I've tried really hard to find this but no luck - I'm sure it's possible I just can't find and example or figure out the syntax for myself
I want to use fabric as a library
I want 2 sets of hosts
I want to reuse the same functions for these different sets of hosts (and so cannot us the #roles decorator on said functions)
So I think I need:
from fabric.api import execute, run, env
NODES = ['192.168.56.141','192.168.56.152']
env.roledefs = {'head':['localhost'], 'nodes':NODES}
env.user('r4space')
def testfunc ():
run('touch ./fred.txt')
execute(testfunc(),<somehow specific 'head' or 'nodes' as my hosts list and user >)
I've tried a whole range of syntax // hosts=NODES, -H NODES, user='r4space'....much more but I either get a syntax error or "host_string = raw_input("No hosts found. Please specify (single)""
If it makes a difference, ultimately my function defs would be in a separate file that I import into main where hosts etc are defined and execute is called.
Thanks for any help!

You have some errors in your code.
env.user('r4space') is wrong. Should be env.user = 'r4space'
When you use execute, the first parameter should be a callable. You have used the return value of the function testfunc.
I think if you fix the last line, it will work:
execute(testfunc, hosts = NODES)

Subprocess statement works in python console but not work in Serverdensity plugin?

in the python console the following statement works perfectly fine (i guess using eval that way is not really good, but its just for testing purpose in this case and will be replaced with proper parsing)
$ python
>>> import subprocess
>>> r = subprocess.Popen(['/pathto/plugin1.rb'], stdout=subprocess.PIPE, close_fds=True).communicate()[0]
>>> data = eval(r)
>>> data
{'test': 1}
when i convert this into a Serverdensity plugin however it keeps crashing the agent.py daemon everytime it executes the plugin. i was able to narrow it down to the subprocess line but could not find out why. exception catching did not seem to work also.
here how the plugin looks like:
class plugin1:
def run(self):
r = subprocess.Popen(['/pathto/plugin1.rb'], stdout=subprocess.PIPE, close_fds=True).communicate()[0]
data = eval(r)
return data
I'm quite new to work with python and cant really figure out why this wont work. Thanks a lot for ideas :)

Do you have subprocess imported in the module? Also what error are you getting could you post the error message ?

After switching my dev box (maybe because of the different python version?) i finally was able to get some proper error output.
Then it was rather simple: I really just needed to import the missing subprocess module.
For who is interested in the solution:
http://github.com/maxigs/Serverdensity-Wrapper-Plugin/blob/master/ruby_plugin.py
Not quite production ready yet, but works already for save input