Develop Reference

Test for a program that receives input from keyboard

I have done a program that receives input from the terminal with the readline function in c (https://eli.thegreenplace.net/2016/basics-of-using-the-readline-library/), readline saves keyboard input from the terminal (basically a read for the stdin) and saves it inside a buffer.
Imagine, when my program is executed, it just prints a message on the screen like: *Enter a command: * and I can write a bash command and it's going to execute it (Just like bash).
I wanted to do a test (In python, c++ or bash) that is going to start the program, and writes into the readline and check is output compared to bash, does anyone knows a way to do it?

Here is an initial bash script that randomizes input with some properties:
for i in {1..5}; do
input=$(echo {A..Z} {a..z} | tr ' ' "\n" | shuf | xargs | tr -d ' ' | cut -b 1-18)
echo ${input}
/path/to/your_program < ${input}
done
You can add characters, make the input shorter etc.

I think what you want is to grab the input in python (I suggest you use the input() function), store it in a variable, then print it out in the terminal using print()

Python: how do I read (not run) a shell script, inserting arguments along the way

I have a simple shell script script.sh:
echo "ubuntu:$1" | sudo chpasswd
I need to open the script, read it, insert the argument, and save it as a string like so: 'echo "ubuntu:arg_passed_when_opening" | sudo chpasswd' using Python.
All the options suggested here actually execute the script, which is not what I want.
Any suggestions?

You would do this the same way that you read any text file, and we can use sys.argv to get the argument passed when running the python script.
Ex:
import sys
with open('script.sh', 'r') as sfile:
modified_file_contents = sfile.read().replace('$1', sys.argv[1])
With this method, modified_file_contents is a string containing the text of the file, but with the specified variable replaced with the argument passed to the python script when it was run.

How to assign variable from a shell command to python script

I am trying to run a batch process using array in slurm. I only know shell command to extract variable from array (text files), but failed to assign it as Python variable.
I have to assign a variable to a Python slurm script. I used a shell command to extract values from the array. but facing errors while assigning it to the variable. I used subprocess, os.system and os.popen.
or is there any way to extract values from text file to be used as a Python variable?
start_date = os.system('$(cat startdate.txt | sed -n ${SLURM_ARRAY_TASK_ID}p)')
start_date = subprocess.check_output("$(cat startdate.txt | sed -n ${SLURM_ARRAY_TASK_ID}p)", shell=True)
start_date = os.popen('$(cat startdate.txt | sed -n ${SLURM_ARRAY_TASK_ID}p)').read()
start_date = '07-24-2004'

Don't use $(...). That will execute the command, and then try to execute the output of the command. You want the output to be sent back to python, not re-executed by the shell.
start_date = subprocess.check_output("cat startdate.txt | sed -n ${SLURM_ARRAY_TASK_ID}p", shell=True)

Barmar is correct, the $(...) part is why you are not getting what you want, but the real question is why when you are using python would you want to use cat and sed as well. Just open the file and pull out the information you want
import os
with open("startdate.txt", "r") as fh:
lines = fh.readlines()
start_date = lines[os.environ['SLURM_ARRAY_TASK_ID']].strip()
the .strip() part gets rid of the newline character.

Python/Unix os.system: Can anyone tell me why this piece of code using awk does not work when executed with os.system?

I have been working with this piece of code for some time in Unix systems and it works just fine while running on a normal command line. However, for the sake of a project and learning how to execute Unix commands through Python, I am trying to run it using the os.system() command in Python.
Overall data structure has 5 columns and 1500 rows and the goal is to replace values greater than 2.706 in column 4 ($4) and then proceed to save them into file2.txt while keeping all other values in the file the same.
os.system("awk '{print $1,$2,$3,$5,($4>=2.706)? -999 : $4}' file1.txt > file2.txt")
From this code, I receive the message Invalid syntax after attempting the execution from a python script.
As I am new to Python, I believe I must just missing something in the syntax from that side of the code, but I cannot for the life of me figure it out. Any help would be greatly appreciated.
A new attempt with the same code, but using the subprocess module instead of os.system:
arg1 = "awk '{print $1,$2,$3,$5,($4>=2.7059553)\? -999 \: $4}' phenotypes.txt > replacetest.txt"
subprocess.run(arg1, shell=TRUE)
This code also gives the Invalid syntax response to the creation of the arg1 command.
(Code is being run in Python 2.7.5 on Linux2.)

Taking the results of a bash command and using it in python

I am trying to write a code in python that will take some information from top and put it into a file.
I want to just write the name of the application and generate the file. The problem i am having is that i can't get the output of the pidof command so i can use it in python. My code looks like this :
import os
a = input('Name of the application')
val=os.system('pidof ' + str(a))
os.system('top -d 30 | grep' + str(val) + '> test.txt')
os.system('awk '{print $10, $11}' test.txt > test2.txt')
The problem is that val always has 0 but the command is returning the pid i want. Any input would be great.

First up, the use of input() is discouraged as it expects the user to type in valid Python expressions. Use raw_input() instead:
app = raw_input('Name of the application: ')
Next up, the return value from system('pidof') isn't the PID, it's the exit code from the pidof command, i.e. zero on success, non-zero on failure. You want to capture the output of pidof.
import subprocess
# Python 2.7 only
pid = int(subprocess.check_output(['pidof', app]))
# Python 2.4+
pid = int(subprocess.Popen(['pidof', app], stdout=subprocess.PIPE).communicate()[0])
# Older (deprecated)
pid = int(os.popen('pidof ' + app).read())
The next line is missing a space after the grep and would have resulted in a command like grep1234. Using the string formatting operator % will make this a little easier to spot:
os.system('top -d 30 | grep %d > test.txt' % (pid))
The third line is badly quoted and should have caused a syntax error. Watch out for the single quotes inside of single quotes.
os.system("awk '{print $10, $11}' test.txt > test2.txt")

Instead of os.system, I recommend you to use the subprocess module: http://docs.python.org/library/subprocess.html#module-subprocess
With that module, you can communicate (input and output) with a shell. The documentation explains the details of how to use it.
Hope this helps!