Develop Reference

Run local commands with Fabric

My environments are based on windows with vagrant or docker as the actual environments. I'd like to set up a quick way of ad hoc deploying stuff directly from windows though, it would be great if I could just run
fab deploySomething
And that would for example locally build an react app, commit and push to the server. However I'm stuck at the local bit.
My setup is:
Windows 10
Fabric 2
Python 3
I've got a fabfile.py set up with a simple test:
from fabric import Connection, task, Config
#task
def deployApp(context):
config = Config(overrides={'user': 'XXX', 'connect_kwargs': {'password': 'YYY'}})
c = Connection('123.123.123.123', config=config)
# c.local('echo ---------- test from local')
with c.cd('../../app/some-app'):
c.local('dir') #this is correct
c.local('yarn install', echo=True)
But I'm just getting:
'yarn' is not recognized as an internal or external command, operable program or batch file.
you can replace 'yarn' with pretty much anything, I can't run a command with local that works fine manually. With debugging on, all i get is:
DEBUG:invoke:Received a possibly-skippable exception: <UnexpectedExit: cmd='cd ../../app/some-app && yarn install' exited=1>
which isn't very helpful...anyone came across this? Any examples of local commands with fabric I can find seem to refer to the old 1.X versions

To run local commands, run them off of your context and not your connection. n.b., this drops you to the invoke level:
from fabric import task
#task
def hello(context):
with context.cd('/path/to/local_dir'):
context.run('ls -la')
That said, the issue is probably that you need the fully qualified path to yarn since your environment's path hasn't been sourced.

Calling Matlab scripts from Django with Python's Popen class

I'm developing a Django app which runs Matlab scripts with Python's Popen class. The python script that calls Matlab scripts lives in the main folder of my Django app (with views.py). When I call the script from command line, it runs like a charm but when I make a request from the client in order to run the corresponding python script, I receive the following warning:
"< M A T L A B (R) > Copyright 1984-2018 The MathWorks, Inc. R2018a (9.4.0.813654) 64-bit (glnxa64) February 23, 2018 To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. >> [Warning: Unable to create preferences folder in /var/www/.matlab/R2018a. Preferences folder location must be writable. Using a temporary preferences folder for this MATLAB session. See the preferences documentation for more details.] >>
My app uses a Python virtual environment and it is being deployed with Apache web server.
Here is my python script that calls Matlab scripts:
import os
import subprocess as sp
import pymat_config
def pymat_run():
pwd = pymat_config.pwd_config['pwd']
cmd1 = "-r \"Arg_in = '/path/to/my/main/folder/input.txt'; Arg_out = '/path/to/my/main/folder/file.txt'; matlab_script1\""
baseCmd1 = ['/usr/local/MATLAB/R2018a/bin/matlab', '-nodesktop', '-nosplash', '-nodisplay', 'nojvm', cmd1]
os.chdir('/path/to/matlab_script1')
sudo_cmd = sp.Popen(['echo', pwd], stdout=sp.PIPE)
exec1 = sp.Popen(['sudo', '-S'] + baseCmd1, stdin=sudo_cmd.stdout, stdout=sp.PIPE, stderr=sp.PIPE)
out, err = exec1.communicate()
return out
Any suggestions ?

Finally I managed to find the solution of that issue by myself. The problem came from the kind of user who called the Matlab's script. When I was running the above script from a Python interpreter or from the shell, it was the user (with the user password) who was running the script while when I was calling the script from the client the user was the web server's user: www-data.
So at first to avoid the above warning I gave permissions to www-data user to the /var/www folder with the following command:
sudo chown -R www-data /var/www/
After that, the "Warning" disappeared but the script still didn't run because it was asking for www-data's password internally and taking user's password from pymat_config file.
To solve this, I edited /etc/sudoers file in order for www-data to be able to call Matlab scripts without asking password. So I added the following line:
www-data ALL=(ALL) NOPASSWD: /usr/local/MATLAB/R2018a/bin/matlab
and now it runs like a charm !

how to execute remote python to open a web on remote machine by using a command on a local machine

I've been searching on net for a long time but I still don't get the answer.
Let me give an examle to describe my question more clearly:
machine A(local) is now conneted with machine B(remote).ALL I WANT TO DO is to :
run a command on A(local),then stop and wait ,and do nothing,and then,a web page is opened on B(remote) automatically.
P.S this python program is stored on machine B.
Here's what I've achived by now:
This is my python program named test.py,and it is stored on B under /home/pi/Documents:
import webbrowser
webbrowser.open('http://www.google.com')
On A,I used command:
ssh <username of B>#<ip of B> python /home/pi/Documents/test.py
After running the above command on A,there is no errors on A but also no action on B.
if I change the command into creating a file on B or sudo reboot,then after running this command there will be a file on B created or B is shut down successfully.
if I change the python program into printing something,like:
print("hello from B")
the content is magically printed on A's terminal.
It seems this command does not work well if I want to open a web on B or print somthing on B.
Can anyone help me with this or is there any other way to accomplish it?
helpless..
Someone has any ideas please???
Thanks in advance!

Assuming B is a Linux or Unix-like system, you have a DISPLAY problem. The webbrowser module locates a browser on the machine, and tries to open it on current display. It works when you launch it localy, but a ssh session has by default no configured display, so any attempt to launch a XWindow (GUI) application will fail.
The rationale behind it is that the -X and -Y flags of the ssh command allow to pass the client display to the server and open the window on the local screen (in your example on A). So if the permissions of the X servers on A and B are compatible, you could try:
A$ ssh -Y <username of B>#<ip of B> # open an interactive shell on B
B$ echo $DISPLAY # control the DISPLAY env variable
-> localhost:10.0 # common value for ssh transported DISPLAY
B$ python /home/pi/Documents/test.py # open the window on A
To force the opening on B, you could set the DISPLAY to localhost:0.0 (primary XWindow)
A$ ssh ssh <username of B>#<ip of B> # open an interactive shell on B
B$ DISPLAY = localhost:0.0 # sets the DISPLAY env variable
B$ export DISPLAY
B$ python /home/pi/Documents/test.py # open the window on B
You might need to tweek authorization of the XWindow servers (or use the awful xhost +) on A and/or B to make the above examples work
Once you have been able to successfully open a window on the proper screen, you will just have to set the DISPLAY environment variable to the correct value in your Python script before opening the browser window.

One of the simplest solutions is to use redirect of stdin
$ ssh pi#B python <<EOF
> print "Hello World from B"
> EOF
Hello World from B
$
However, if the script is quite big, it is better to copy py file to server B and then call ssh with the file name as #Eliethesaiyan suggested.
$ scp X.py pi#B:/home/pi/
X.py 100% 26 0.0KB/s 00:00
$ ssh pi#B python X.py
Hello World from B
$

I've tested this using a VM running Ubuntu, which OS are you running on your remote system? Here's my launch_google.py:
import os
os.environ["DISPLAY"] = ":0"
import webbrowser
webbrowser.open("https://google.com")
Launch this using:
ssh <user>#<IP Address> "python launch_google.py&"
I included the ampersand otherwise the ssh session remains open. The python process doesn't need to keep running.
It is important to set the DISPLAY environment variable before importing the webbrowser module, otherwise the browsers won't be setup correctly. You can verify this running python via SSH:
>>> import os
>>> "DISPLAY" in os.environ
False
>>> import webbrowser
>>> len(webbrowser._browsers)
0
>>> webbrowser.open("https://google.com")
False
>>> os.environ["DISPLAY"] = ":0"
>>> reload(webbrowser)
<module 'webbrowser' from '/usr/lib/python2.7/webbrowser.pyc'>
>>> len(webbrowser._browsers)
3
>>> webbrowser.open("https://google.com")
True
>>>

PuTTY Windows pywinauto background

I did a script on Windows using PuTTY:
from pywinauto.application import Application
app = Application().Start(cmd_line='C:\Program Files (x86)\PuTTY\putty.exe -l user -pw **pwd** -load Proxy_10.153.1.250 '+ ip +' -ssh')
putty = app.PuTTY
putty.Wait('ready')
time.sleep(7)
cmd1 = "show log "+ "{ENTER}"
This script will be executed for many switchs, but when it is executed, I cannot do other tasks on Windows else script will be interrupted? Is it possible to be executed in background?

You need a proper tool for CLI automation. Just run subprocess.call('ssh user#host <the rest of cmd>') or use Paramiko to run remote SSH command.
BTW, pywinauto's code is incomplete, I don't see .type_keys(cmd1). You may try .send_chars(cmd1) instead and use putty.minimize() first. But send_chars is not guaranteed to work with every app (and it's experimental). So you can just try.

Running Python Script at Ubuntu Startup not working

I am trying to run a python script at ubuntu startup.
The script(play.py) plays a youtube video in chrome browser.
import webbrowser
import time
import subprocess as sp
val = 0;
while(val <1):
time.sleep(1)
#webbrowser.open("https://www.youtube.com/watch?v=dO1rMeYnOmM")
child = sp.Popen("google-chrome %s" % "https://www.youtube.com/watch?v=dO1rMeYnOmM", shell=True)
val = val +1
To run this script at ubuntu startup I added the following entry in rc.local:
python /home/user_name/Desktop/python/play.py &
And changes the mode of play.py to 777.
I also tried copying this script file to /etc/init.d.
But still the script does not execute at system startup.

Try sudo /usr/bin/python /home/user_name/Desktop/python/play.py & instead.