Is it possible to have local and remote tasks execute from within the same task method?
e.g., I want to do something like the following:
#fabric.api.task
def Deploy():
PrepareDeploy()
PushDeploy()
execute(Extract())
execute(Start())
Where PrepareDeploy and PushDeploy are local tasks (executing only locally, via the fabric.api.local() method):
#fabric.api.task
#fabric.decorators.runs_once
def PrepareDeploy():
#fabric.api.task
#fabric.decorators.runs_once
def PushDeploy():
And Extract/Start are methods that should be run on the remote hosts themselves:
#fabric.api.task
def Extract():
#fabric.api.task
def Start():
However, when I try to do fab Deploy, I get something like:
[remote1.serv.com] Executing task 'Deploy'
[localhost] local: find . -name "*.java" > sources.txt
...
The first line seems wrong to me (and in fact, causes errors).
You can spawn new task and defining on what hosts should it run, for example - how to create rabbitmq of all hosts are provisioned with puppet with the same erlang cookie.
See around line 114 - there is an executon of the tasks on specific hosts.
https://gist.github.com/nvtkaszpir/17d2e2180771abd93c46
I hope this helps.
Related
Here is my script :
from fabric2 import Connection
c = Connection('127.0.0.1')
with c.cd('/home/bussiere/'):
c.run('ls -l')
But i have this error :
paramiko.ssh_exception.AuthenticationException: Authentication failed.
So how do I run a command on localhost ?
In Fabric2, the Connection object has got a local() method.
Have a look at this object's documentation here.
As of July 2020, with fabric2 if you don't pass argument to your task decorator by default you are on the local machine.
for example the following will run on your local machine (localhost) :
Example 1 : Only on local
#python3
#fabfile.py
from fabric import task, Connection
c = Connection('remote_user#remote_server.com')
#task
def DetailList(c):
c.run('ls -l') # will run on local server because the decorator #task does not contain the hosts parameter
You then would run this on your machine with
fab DetailList
If you want to mix code that should be running on remote server and on local you should pass the connection to the #task decorator as a parameter.
Example 2: on local and on remote (but different functions)
#python3
#fabfile.py
#imports
from fabric import task, Connection
#variables
list_of_hosts = ['user#yourserver.com'] #you should have already configure the ssh access
c = Connection(list_of_hosts[0])
working_dir = '/var/www/yourproject'
#will run on remote
#task(hosts = list_of_hosts)
def Update(c):
c.run('sudo apt get update') # will run on remote server because hosts are passed to the task decorator
c.run(f'cd {working_dir} && git pull') # will run on remote server because hosts are passed to the task decorator
c.run('sudo service apache2 restart') # will run on remote server because hosts are passed to the task decorator
#will run on local because you do not specify a host
#task
def DetailsList(c):
c.run('ls -l') # # will run on local server because hosts are NOT passed to the task decorator
As mentionned by Ismaïl there also is a 'local' method that can be used when passing the hosts parameter, the 'local' method will run on the localhost although you have specified the host parameter to the task decorator. Be careful though you can not use the 'local' method if you didn't specified any hosts parameters, use run instead as shown in example 1 & 2.
Example 3 : use both on remote and local servers but under the same functions, note we are not decorating functions that are called in the UpdateAndRestart function.
#python3
#fabfile.py
#imports
from fabric import task, Connection
#variables
list_of_hosts = ['www.yourserver.com'] #you should have already configure the ssh access
c = Connection(list_of_hosts[0])
working_dir = '/var/www/yourproject'
def UpdateServer(c):
c.run('sudo apt get update') # will run on remote server because hosts are passed to the task decorator
c.local('echo the remote server is now updated') # will run on local server because you used the local method when hosts are being passed to the decorator
def PullFromGit(c):
c.run(f'cd {working_dir} && git pull') # will run on remote server because hosts are passed to the task decorator
c.local('echo Git repo is now pulled') # will run on local server because you used the local method when hosts are being passed to the decorator
def RestartServer(c):
c.run('sudo service apache2 restart') # will run on remote server because hosts are passed to the task decorator
c.local('echo Apache2 is now restarted') # will run on local server because you used the local method when hosts are being passed to the decorator
#task(hosts = list_of_hosts)
def UpdateAndRestart(c):
UpdateServer(c)
PullFromGit(c)
RestartServer(c)
c.local('echo you have updated, pulled and restarted Apache2') # will run on local server because you used the local method when hosts are being passed to the decorator
You will be able to run the entire stack with :
fab UpdateAndRestart
Im trying to execute main task that needs to execute tasks differently for each host. In the following setup, task 'sometask' will get execute twice for each host. What is the best way to prevent that?
#task
#hosts('host1', 'host2')
def test():
execute(do_something_everywhere) # execute on both hosts
execute(sometask, 'arg1', host='host1') # execute on host1 only
execute(sometask, 'arg2', host='host2') # execute on host2 only
You can user the #runs_once decorator to remedy this but I find that can cause extra work making wrapper functions to get the execution order you want, so here's a quick fix using the env.host_string value to evaluate which server you are deploying to and adjust your script accordingly:
#hosts('host1', 'host2')
#task
def checkout(branch='master'):
execute(_test_task_w_arg, 'all-servers')
execute(_test_task_w_arg, 'arg1' if env.host_string == 'host1' else 'arg2')
def _test_task_w_arg(arg1):
local("touch test-file-" + arg1)
Results in this output which seems to achieve what you want:
[host1] Executing task 'checkout'
[localhost] local: touch test-file-all-servers
[localhost] local: touch test-file-arg1
[host2] Executing task 'checkout'
[localhost] local: touch test-file-all-servers
[localhost] local: touch test-file-arg2
I'm attempting to announce deployment start and end in my fabric script. Feels like this should be easy, but for the life of me I can't figure out how to do it.
env.hosts = ['www1', 'www2', 'www3', 'www4']
def announce_start():
# code to connect to irc server and announce deployment begins
pass
def announce_finish():
# code to connect to irc server and announce deployment finishes
pass
def deploy():
# actual deployment code here
pass
Here's what I've tried:
If I make my deploy task contain 'announce_start' and 'announce_finish'. It will attempt to run all those tasks on each server.
def deploy():
announce_start()
# actual deployment code here
announce_finish()
If I decorate announce_start() and announce_end() with #hosts('localhost'), it runs it on localhost, but still four times. One for each host.
As I was typing this, I finally got it to work by using the decorator #hosts('localhost') on announce_start/end and the fab command:
fab announce_start deploy announce_end
But this seems a bit hacky. I'd like it all wrapped in a single deploy command. Is there a way to do this?
You can use fabric.api.execute, e.g.
def announce_start():
# code to connect to irc server and announce deployment begins
pass
def announce_finish():
# code to connect to irc server and announce deployment finishes
pass
#hosts(...)
def deploy_machine1():
pass
#hosts(...)
def deploy_machine2():
pass
def deploy():
announce_start()
execute(deploy_machine1)
execute(deploy_machine2)
announce_finish()
and then just invoke fab deploy
This question already has answers here:
How can I properly set the `env.hosts` in a function in my Python Fabric `fabfile.py`?
(5 answers)
Closed 9 years ago.
I'm cutting my teeth on Python as I work with Fabric. Looks like I have a basic misunderstanding of how Python and/or Fabric works. Take a look at my 2 scripts
AppDeploy.py
from fabric.api import *
class AppDeploy:
# Environment configuration, all in a dictionary
environments = {
'dev' : {
'hosts' : ['localhost'],
},
}
# Fabric environment
env = None
# Take the fabric environment as a constructor argument
def __init__(self, env):
self.env = env
# Configure the fabric environment
def configure_env(self, environment):
self.env.hosts.extend(self.environments[environment]['hosts'])
fabfile.py
from fabric.api import *
from AppDeploy import AppDeploy
# Instantiate the backend class with
# all the real configuration and logic
deployer = AppDeploy(env)
# Wrapper functions to select an environment
#task
def env_dev():
deployer.configure_env('dev')
#task
def hello():
run('echo hello')
#task
def dev_hello():
deployer.configure_env('dev')
run('echo hello')
Chaining the first 2 tasks works
$ fab env_dev hello
[localhost] Executing task 'hello'
[localhost] run: echo hello
[localhost] out: hello
Done.
Disconnecting from localhost... done.
however, running the last task, which aims to configure the environment and do something in a single task, it appears fabric does not have the environment configured
$ fab dev_hello
No hosts found. Please specify (single) host string for connection:
I'm pretty lost though, because if I tweak that method like so
#task
def dev_hello():
deployer.configure_env('dev')
print(env.hosts)
run('echo hello')
it looks like env.hosts is set, but still, fabric is acting like it isn't:
$ fab dev_hello
['localhost']
No hosts found. Please specify (single) host string for connection:
What's going on here?
I'm not sure what you're trying to do, but...
If you're losing info on the shell/environment -- Fabric runs each command in a separate shell statement, so you need to either manually chain the commands or use the prefix context manager.
See http://docs.fabfile.org/en/1.8/faq.html#my-cd-workon-export-etc-calls-don-t-seem-to-work
If you're losing info within "python", it might be tied into this bug/behavior that I ran into recently [ https://github.com/fabric/fabric/issues/1004 ] where the shell i entered into Fabric with seems to be obliterated.
How do I NOT parallelize local commands inside a task with #parallel decorator:
#parallel
def myTask():
local('initialize localhost')
run('command on remote host')
local('clean up localhost')
I want commands on local host only to execute once, and commands for remote hosts run in parallel. Right now I'm seeing local host commands running for each remote host instance. What is the cleanest way to do this?
Thanks
Group your parallel commands into a decorated function. Then use execute() to call it inbetween the local calls.
Does the following work for you?
def local_init():
local('initialize')
#parallel
def myTask():
run('remote command')
def local_cleanup():
local('clean up')
And then:
fab local_init myTask local_cleanup