Develop Reference

Run script for send in-game Terraria server commands

In the past week I install a Terraria 1.3.5.3 server into an Ubuntu v18.04 OS, for playing online with friends. This server should be powered on 24/7, without any GUI, only been accessed by SSH on internal LAN.
My friends ask me if there is a way for them to control the server, e.g. send a message, via internal in-game chat, so I thought use a special character ($) in front of the desired command ('$say something' or '$save', for instance) and a python program, that read the terminal output via pipe, interpreter the command and send it back with a bash command.
I follow these instructions to install the server:
https://www.linode.com/docs/game-servers/host-a-terraria-server-on-your-linode
And config my router to forward a dedicated port to the terraria server.
All is working fine, but I really struggle to make python send a command via "terrariad" bash script, described in the link above.
Here is a code used to send a command, in python:
import subprocess
subprocess.Popen("terrariad save", shell=True)
This works fine, but if I try to input a string with space:
import subprocess
subprocess.Popen("terrariad \"say something\"", shell=True)
it stop the command in the space char, output this on the terminal:
: say
Instead of the desired:
: say something
<Server>something
What could I do to solve this problem?
I tried so much things but I get the same result.
P.S. If I send the command manually in the ssh putty terminal, it works!
Edit 1:
I abandoned the python solution, by now I'll try it with bash instead, seem to be more logic to do this way.
Edit 2:
I found the "terrariad" script expect just one argument, but the Popen is splitting my argument into two no matter the method I use, as my input string has one space char in the middle. Like this:
Expected:
terrariad "say\ something"
$1 = "say something"
But I get this of python Popen:
subprocess.Popen("terrariad \"say something\"", shell=True)
$1 = "say
$2 = something"
No matter i try to list it:
subprocess.Popen(["terrariad", "say something"])
$1 = "say
$2 = something"
Or use \ quote before the space char, It always split variables if it reach a space char.
Edit 3:
Looking in the bash script I could understand what is going on when I send a command... Basically it use the command "stuff", from the screen program, to send characters to the terraria screen session:
screen -S terraria -X stuff $send
$send is a printf command:
send="`printf \"$*\r\"`"
And it seems to me that if I run the bash file from Python, it has a different result than running from the command line. How this is possible? Is this a bug or bad implementation of the function?
Thanks!

I finally come with a solution to this, using pipes instead of the Popen solution.
It seems to me that Popen isn't the best solution to run bash scripts, as described in How to do multiple arguments with Python Popen?, the link that SiHa send in the comments (Thanks!):
"However, using Python as a wrapper for many system commands is not really a good idea. At the very least, you should be breaking up your commands into separate Popens, so that non-zero exits can be handled adequately. In reality, this script seems like it'd be much better suited as a shell script.".
So I came with the solution, using a fifo file:
First, create a fifo to be use as a pipe, in the desired directory (for instance, /samba/terraria/config):
mkfifo cmdOutput
*/samba/terraria - this is the directory I create in order to easily edit the scripts, save and load maps to the server using another computer, that are shared with samba (https://linuxize.com/post/how-to-install-and-configure-samba-on-ubuntu-18-04/)
Then I create a python script to read from the screen output and then write to a pipe file (I know, probably there is other ways to this):
import shlex, os
outputFile = os.open("/samba/terraria/config/cmdOutput", os.O_WRONLY )
print("python script has started!")
while 1:
line = input()
print(line)
cmdPosition = line.find("&")
if( cmdPosition != -1 ):
cmd = slice(cmdPosition+1,len(line))
cmdText = line[cmd]
os.write(outputFile, bytes( cmdText + "\r\r", 'utf-8'))
os.write(outputFile, bytes("say Command executed!!!\r\r", 'utf-8'))
Then I edit the terraria.service file to call this script, piped from terrariaServer, and redirect the errors to another file:
ExecStart=/usr/bin/screen -dmS terraria /bin/bash -c "/opt/terraria/TerrariaServer.bin.x86_64 -config /samba/terraria/config/serverconfig.txt < /samba/terraria/config/cmdOutput 2>/samba/terraria/config/errorLog.txt | python3 /samba/terraria/scripts/allowCommands.py"
*/samba/terraria/scripts/allowCommands.py - where my script is.
**/samba/terraria/config/errorLog.txt - save Log of errors in a file.
Now I can send commands, like 'noon' or 'dawn' so I can change the in-game time, save world and backup it with samba server before boss fights, do another stuff if I have some time XD, and have the terminal showing what is going on with the server.

busybox ash via paramiko not emitting prompt on stdout

I created Python GUI that takes a list of commands as input and executes the list through a Telnet or SSH session. While opening a SSH session (using Paramiko in Python) I run commands on various devices using this code in a for loop:
stdin.write(command+"\n")
then = time.time()
timeout=10.0
while not stdout.channel.exit_status_ready():
if timeout is not None:
timeTest =time.time() - then
if timeout <= timeTest:
break
# Only print data if there is data to read in the channel
if stdout.channel.recv_ready():
# Write data from stdout
temp=str(stdout.channel.recv(1024))
print temp
if (temp.endswith(delim)) or (temp.endswith("# ")) :
print "successful exit"
break
The code is designed to be used on modems that have BusyBox installed. Thus it's common for a user to enter a command to open busybox and run a sequence of commands in the BusyBox shell. As you can see in this line of code "if (temp.endswith(delim)) or (temp.endswith("# "))" the while loop is suppose to break when the command prompt of busybox is detected which is a "# " (this means the command has finished outputting).
The problem I'm having is that BusyBox isn't printing the command prompt to stdout or in the debug line "print temp". Why is this? The command outputs (an example is ls -l) are successfully printed to stdout but not the command prompt or busybox intro message: when a user enters busybox on these modems a introduction message is printed "BusyBox v1.17.2 (2014-10-02 10:50:35 PDT) built-in shell (ash)
Enter 'help' for a list of built-in commands." which is also not printed to STDOUT. This is forcing the code to utilize the timeout for each command executed in busybox which is undesirable, i.e. it's slow and there could be command outputs that take longer than the desired timeout so it's best to look for the command prompt.
Is this issue due to the implementation of ash in BusyBox? Is there a way to receive the command prompt text?

First: If you're trying to recognize shell prompts when invoking a shell programmatically, you're Doing It Wrong.
If you use exec_command() on a new channel (over the same transport) for each command you want to run, you'll get a separate stdout, stderr, etc. for each command; have the exit status for that command individually reported, and never need to guess whether a piece of output is from the command or from the outer shell.
Second: If you really want to do it that way (and, again, you shouldn't!), you should be using invoke_shell() after a get_pty() call, not exec_command('sh').

See the entry on PS1 (the variable specifying the prompt to emit) in IEEE standard 1003.1 (emphasis added):
PS1
Each time an interactive shell is ready to read a command, the value of this variable shall be subjected to parameter expansion and written to standard error. The default value shall be "$ ". For users who have specific additional implementation-defined privileges, the default may be another, implementation-defined value. The shell shall replace each instance of the character '!' in PS1 with the history file number of the next command to be typed. Escaping the '!' with another '!' (that is, "!!" ) shall place the literal character '!' in the prompt. This volume of POSIX.1-2008 specifies the effects of the variable only for systems supporting the User Portability Utilities option.
Thus, any POSIX-compliant shell -- not just busybox ash, but ksh, dash, bash, and all the rest -- must write its prompt to stderr, not stdout.
Your simple fix, then, is to either read from stderr, or to use redirection to combine the streams (running exec 2>&1, for instance).

why is call to ssh with Python subprocess breaking outer bash loop

I have a bash script myscript.sh:
#!/bin/bash
while read line; do
myprog.py
done
calling a python program myprog.py
#!/usr/bin/env python
import subprocess
output = subprocess.check_output(['ssh', 'user#host', 'cmd'])
The ssh command that is called by subprocess executes without error, the output is correct. But when called like this the loop in myscript.sh only runs through the first line of input and then exits with status 0. If I replace the subprocess.check_output(...) call with a subprocess.Popen(...) and don't subsequently call Popen.wait() then the outer loop works as expected and the output from the ssh command is dumped to standard out some time after any output from the bash script. With the Popen.wait() behavior is the same as with check_output: bash loop only goes through one iteration before exiting without error.
If instead of ssh another command, e.g. ls, is called with check_output then the bash loop works as expected.
Can anyone help me understand why the code as shown isn't working as expected?
Note: this is a simplified version of what I am trying to do, though I do experience the same behavior with this code. In reality I am doing something with "$line" in the bash script and the subprocess call is wrapped in a try/except block.

As #larsmans guessed the ssh call was consuming stdin, breaking the outer bash loop. Adding the -n option to the ssh command resolved the issue:
output = subprocess.check_output(['ssh', '-n', 'user#host', 'cmd'])

The problem is that ssh reads from standard input, therefore it "eats" all the remaining lines in the loop. You can just connect its standard input to nowhere using the -n flag:
output = subprocess.check_output(['ssh', '-n', 'user#host', 'cmd'])
Look for the details on the man pages of ssh here https://linux.die.net/man/1/ssh and https://man.openbsd.org/ssh

how can I silence all of the output from a particular python command?

Autodesk Maya 2012 provides "mayapy" - a modded build of python filled with the necessary packages to load Maya files and act as a headless 3D editor for batch work. I'm calling it from a bash script. If that script opens a scene file in it with cmds.file(filepath, open=True), it spews pages of warnings, errors, and other info I don't want. I want to turn all of that off only while the cmds.file command is running.
I've tried redirecting from inside of the Python commands I'm sending into mayapy inside the shell script, but that doesn't work. I can silence everything by redirecting stdout/err to /dev/null in the call to the bash script. Is there any way to silence it in the call to the shell, but still allow my passed-in command inside the script to print out information?
test.sh:
#!/bin/bash
/usr/autodesk/maya/bin/mayapy -c "
cmds.file('filepath', open=True);
print 'hello'
"
calling it:
$ ./test.sh # spews info, then prints 'hello'
$ ./test.sh > /dev/null 2>&1 # completely silent

Basically, I think the best way to solve this is to implement a wrapper that will execute test.sh and sanitize the output to the shell. To sanitize the output, I would simply prepend some string to notify your wrapper that this text is good for output. My inspiration for the wrapper file came from this: https://stackoverflow.com/a/4760274/2030274
The contents are as follows:
import subprocess
def runProcess(exe):
p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
retcode = p.poll() #returns None while subprocess is running
line = p.stdout.readline()
yield line
if(retcode is not None):
break
for line in runProcess(['./test.sh']):
if line.startswith('GARYFIXLER:'):
print line,
Now you could imagine test.sh being something along the lines of
#!/bin/bash
/usr/autodesk/maya/bin/mayapy -c "
cmds.file('filepath', open=True);
print 'GARYFIXLER:hello'
"
and this will only print the hello line. Since we are wrapping the python call in a subprocess, all output typically displayed to the shell should get captured and you should intercept the lines that you don't want.
Of course, to call test.sh from a python script, you need to make sure you have the correct permissions.

I knew I was just getting twisted around with pipes. Maya is indeed sending all batch output to stderror. This frees stdout entirely once you properly pipe stderr away. Here's an all-bash one-liner that works.
# load file in batch; divert Maya's output to /dev/null
# then print listing of things in file with cmds.ls()
/usr/autodesk/maya/bin/mayapy -c "import maya.standalone;maya.standalone.initialize(name='python');cmds.file('mayafile.ma', open=True);print cmds.ls()" 2>/dev/null

Cannot Launch Interactive Program While Piping to Script in Python

I have a python script that needs to call the defined $EDITOR or $VISUAL. When the Python script is called alone, I am able to launch the $EDITOR without a hitch, but the moment I pipe something to the Python script, the $EDITOR is unable to launch. Right now, I am using nano which shows
Received SIGHUP or SIGTERM
every time. It appears to be the same issue described here.
sinister:Programming [1313]$ echo "import os;os.system('nano')" > "sample.py"
sinister:Programming [1314]$ python sample.py
# nano is successfully launched here.
sinister:Programming [1315]$ echo "It dies here." | python sample.py
Received SIGHUP or SIGTERM
Buffer written to nano.save.1
EDIT: Clarification; inside the program, I am not piping to the editor. The code is as follows:
editorprocess = subprocess.Popen([editor or "vi", temppath])
editorreturncode = os.waitpid(editorprocess.pid, 0)[1]

When you pipe something to a process, the pipe is connected to that process's standard input. This means your terminal input won't be connected to the editor. Most editors also check whether their standard input is a terminal (isatty), which a pipe isn't; and if it isn't a terminal, they'll refuse to start. In the case of nano, this appears to cause it to exit with the message you included:
% echo | nano
Received SIGHUP or SIGTERM
You'll need to provide the input to your Python script in another way, such as via a file, if you want to be able to pass its standard input to a terminal-based editor.
Now you've clarified your question, that you don't want the Python process's stdin attached to the editor, you can modify your code as follows:
editorprocess = subprocess.Popen([editor or "vi", temppath],
stdin=open('/dev/tty', 'r'))

The specific case of find -type f | vidir - is handled here:
foreach my $item (#ARGV) {
if ($item eq "-") {
push #dir, map { chomp; $_ } <STDIN>;
close STDIN;
open(STDIN, "/dev/tty") || die "reopen: $!\n";
}
You can re-create this behavior in Python, as well:
#!/usr/bin/python
import os
import sys
sys.stdin.close()
o = os.open("/dev/tty", os.O_RDONLY)
os.dup2(o, 0)
os.system('vim')
Of course, it closes the standard input file descriptor, so if you intend on reading from it again after starting the editor, you should probably duplicate its file descriptor before closing it.