I have problem with fabic (2.4). I have no access to environment variables in remote server (I'm using FreeBSD).
In my ~/.profile file i have variable:
export MY_KEY="123456789"
In my fabfile.py i have simple task:
from fabric import task
#task(hosts=['user#myhost.com'])
def deploy(context):
context.run('echo 123')
context.run('echo $MY_KEY')
When I run the fab deploy command, I see only 123 but after connecting via ssh my variable is visible.
And what about using Connection.prefix as a context manager ?
with context.prefix('MY_KEY="123456789"'):
context.run('echo 123')
context.run('echo $MY_KEY')
Related
My environments are based on windows with vagrant or docker as the actual environments. I'd like to set up a quick way of ad hoc deploying stuff directly from windows though, it would be great if I could just run
fab deploySomething
And that would for example locally build an react app, commit and push to the server. However I'm stuck at the local bit.
My setup is:
Windows 10
Fabric 2
Python 3
I've got a fabfile.py set up with a simple test:
from fabric import Connection, task, Config
#task
def deployApp(context):
config = Config(overrides={'user': 'XXX', 'connect_kwargs': {'password': 'YYY'}})
c = Connection('123.123.123.123', config=config)
# c.local('echo ---------- test from local')
with c.cd('../../app/some-app'):
c.local('dir') #this is correct
c.local('yarn install', echo=True)
But I'm just getting:
'yarn' is not recognized as an internal or external command, operable program or batch file.
you can replace 'yarn' with pretty much anything, I can't run a command with local that works fine manually. With debugging on, all i get is:
DEBUG:invoke:Received a possibly-skippable exception: <UnexpectedExit: cmd='cd ../../app/some-app && yarn install' exited=1>
which isn't very helpful...anyone came across this? Any examples of local commands with fabric I can find seem to refer to the old 1.X versions
To run local commands, run them off of your context and not your connection. n.b., this drops you to the invoke level:
from fabric import task
#task
def hello(context):
with context.cd('/path/to/local_dir'):
context.run('ls -la')
That said, the issue is probably that you need the fully qualified path to yarn since your environment's path hasn't been sourced.
I have a shell script which in turn calls a python script. But before the python script runs I am setting up the environment variables in order to have the right P4 configurations.
shell: /home/ag/ump_prod/run.sh
python script: /home/ag/ump_prod/cron.py
Environment conf: /home/ag/ump_prod/env.conf
The python script executes command line P4 commands via the subprocess module.
Here is the code for the shell script
#!/bin/sh
. /home/ag/ump_prod/env.conf
python /home/ag/ump_prod/cron.py
env.conf
export SHELL=/bin/bash
export USER=ag
export MAIL=/var/mail/ag
export HOME=/home/ag
export LOGNAME=ag
export P4CONFIG=/home/ag/ump_prod/.perforce
The perforce config /home/ag/ump_prod/.perforce :
P4CLIENT=ag_ump
P4EDITOR=/usr/bin/vim
P4PORT=rsh:ssh -2 -q -a -x -l p4server p4.****.com /bin/true
P4USER=ag
Manually running the shell script executes it without any issues.
However, when I run it via cronjob it complains that it cannot connect to the server.
Error Message:
['TCP receive failed.\n', 'read: socket stdio: Connection reset by peer\n', 'Perforce client error:\n', '\tTCP receive failed.\n', '\tread: socket stdio: Connection reset by peer\n']
Please let me know where I could be going wrong in setting the environment variables for P4 configs. Thanks in advance!
I found the solution: I needed to start the ssh-agent dynamically for the cronjob as well. It is just like how you login for the first time and start the ssh-agent. We need to tell cronjob too to include eval ssh-agent
I have a Django project hosted on a remote server. This contains a file called tmp_file.py. There's a function called fetch_data() inside that file. Usually I follow the below approach to run that function.
# Inside Django Project
$ python manage.py shell
[Shell] from tmp_file import feth_data
[Shell] fetch_data()
Also the file doesn't contain __name__ section. So can't run as a stand alone script. What's the best way to perform this task using Ansible. I couldn't find anything useful from Ansible docs.
There's --command switch for shell django-admin command.
So you can try in Ansible:
- name: Fetch data
command: "django-admin shell --command='from tmp_file import feth_data; fetch_data()'"
args:
chdir: /path/to/tmp_file
I have set up an environment variable which I execute locally using a .sh file:
.sh file:
#!/bin/sh
echo "environment variables"
export BROKER="amqp://admin:password#11.11.11.11:4672//"
Locally inside a virtual environment I can now read this in Python using:
BROKER = os.environ['BROKER']
However, on my production server (Ubuntu). I run the same file chmod +x name_of_file.sh and source settings.sh and can see the variable using printenv, but Python gives the error KeyError: 'BROKER' Why?
This only happens on my production machine despite the fact I can see the variable using printenv. Note my production machines does not use virtualenv.
If I run the python shell on Ubuntu and do os.environ['BROKER'] it prints out the correct value. So I have not idea what the app file does not find it.
This is the task that gets run which cannot find the variable (supervisor task)
[program:celery]
directory = /srv/app_test/
command=celery -A tasks worker -l info
stdout_logfile = /var/log/celeryd_.log
autostart=true
autorestart=true
startsecs=5
stopwaitsecs = 600
killasgroup=true
priority=998
user=ubuntu
Celery Config (which does not find the variable when executed under supervisor:
from kombu import Exchange, Queue
import os
# Celery Settings
BROKER = os.environ['BROKER']
When I restart supervisor it gives the key error.
The environment variables from your shell will not be visible within supervisor tasks.
You need to use the environment setting in your supervisor config:
[program:celery]
...
environment=BROKER="amqp://admin:password#11.11.11.11:4672//"
This requires supervisor 3.0+.
I have a nat and it has various server
So from my local server I want to go to nat and then from nat i have to ssh to other machines
Local-->NAT(abcuser#publicIP with key 1)-->server1(xyzuser#localIP with key 2)
nat has different ssh key
and each of the server has different ssh key
how can i accomplish this type of multihop ssh using fabric
I tried using env.roledefs feature but it doesnt seems to be working
also I am not sure how to define two ssh keys.I know we can define a list of keys with env.key_filename but issue is will it check each key with each server?How can I be more specific and match a key with one server only
I have tried using command from my local machine
fab deploy -g 'ec2-user#54.251.151.39' -i '/home/aman/Downloads/aws_oms.pem'
and my script is
from __future__ import with_statement
from fabric.api import local, run, cd, env, execute
env.hosts=['ubuntu#10.0.0.77']
env.key_filename=['/home/ec2-user/varnish_cache.pem']
def deploy():
run("uname -a")
It's possible. Double hop to 10.0.0.2 (and list files) via gateway hop 10.0.0.1. Basically, you simply nest the connections with the gateway parameter.
# coding: utf-8
from fabric import Connection
path = '/'
conn1 = Connection(host='user1#10.0.0.1', connect_kwargs={'password': '***'})
conn2 = Connection(host='user2#10.0.0.2', connect_kwargs={'password': '***'}, gateway=conn1)
result = conn2.run(f'''cd {path} && ls -al''', hide=True)
conn2.close()
conn1.close()
msg = "Ran {0.command!r} on {0.connection.host}, got stdout:\n{0.stdout}"
print(msg.format(result))
Please remember to run the SSH connection manually once to introduce the servers to each other!
Install via
pip3 install --upgrade fabric
pip3 install cryptography==2.4.2 # optional to hide some annoying warnings
http://docs.fabfile.org/en/latest/concepts/networking.html
Python 3.6+.
In order to connect to remote hosts via an intermediate server, you can use the --gateway command-line option :
http://docs.fabfile.org/en/latest/usage/fab.html#cmdoption-g
Or, alternatively, set the env.gateway variable inside your fabfile :
http://docs.fabfile.org/en/latest/usage/env.html#gateway
For more detail information, see:
http://docs.fabfile.org/en/stable/concepts/networking.html#ssh-gateways