Develop Reference

Run two different instances of the same script with different configuration

I have a simple script that is responsible of fetching data from an external API, lets call it connector.py.
That script takes some params as an input ,do its job and then write it to a file and return the output.
I want to implement a scheduler that would create and manage two instances of that script, each with his own input(different settings) and make them run in configured intervals with the next constraint:
Input: Pass the parameters of the connector from the settings, to the sub-process via the stdin channel (not as process args)
Output: Pass the connector output from the sub-process to the service via the stdout channel
I have to implement the constant loop cycle by myself (not use a Scheduler for example)
What mechanisem should I use in order to acheive that goal processes?, threads?, sub-process?
Im mainly struggling to understand how to deal with stdin/stdout issue for the different connector instances.
Any advice would be appericiated.

You have two possibilities whith the scheduling of tasks.
Make your script a factory which will run everytime until something stop it. So you will have the possibility to choose either threads or processes (subprocess use porcess). Here a little description of threads and processes. (If I use this method I would use sub-processes)
What is the difference between a process and a thread?
https://www.backblaze.com/blog/whats-the-diff-programs-processes-and-threads/
However I don't see the utility of using threads or subprocesses in your case because you're telling us that you will make them run in configured intervals. You can just integerate the program to your to make them run separatly.
For task scheduling you also have the use of cronjobs. It allows the execution of commands depending of the date, repetition, user, etc. Here some detail on how setting up a cronjob:
https://phoenixnap.com/kb/set-up-cron-job-linux

Is it possible to use Node.js to communicate between a HTML page and Python script?

I currently have a simple HTML with three input text boxes and a button running on Node.js. I am able to send values from the HTML page to the python script as arguments when executing the script (sys.argv) through Node.js as a child process.
The python script keeps generating values. Right now, I simply print those values, but can I grab these values and send them back to the webpage every second until the script stops running after about 3 minutes? If yes, how can I grab them?
I want to use Node.js because I want to use the package pdfmake (https://www.npmjs.com/package/pdfmake) from npm to generate reports of the same.

If you're already executing the Python script as a child process, simply capture the STDOUT stream from the process.
https://nodejs.org/api/child_process.html
https://nodejs.org/api/child_process.html#child_process_subprocess_stdout

I do not have comment privileges yet and this may not be an exact answer but have you looked into websockets? It looks like you should be able to emit the data from the python code to the port that you may be hosting your web page on.

python: running 'nohup': how to get rid of message?

I have a variable list of programs that I want to kick off from a cron job. The solution I have settled on at least for now is to write the actual cron job in python, then run through the list, starting each program with:
outf=open('the_command.log','w')
subprocess.Popen(['nohup','the_command', ...],stdout=outf)
outf.close()
The problem with this is that it creates a nohup.out file - the same one for each process, it seems. If I did this same thing from the command line, it might look like:
$ nohup the_command ... > the_command.log 2>&1
This works fine, except I get a message from nohup when I run it:
nohup: ignoring input and redirecting stderr to stdout
I have tried to redirect stderr to /dev/null, but the result is that the_command.log is empty. How can I solve this?

I solved this by using a different command detach from http://inglorion.net/software/detach/
But I now consider this to be improper. It would be better to use oneshot services started by your cron job script or make your cron entry cause a oneshot service to be started.
With this there would be no need to detach as the processes aren't your scripts children rather they are children of the supervisor. Any init that supports starting a normally down service and does not restart it when it exits can be used.

running commands in an external gnome-terminal using subprocess.Popen

I am trying to execute commands using communicate in the terminal that i spawned.
sitecreate_proc = subprocess.Popen(['gnome-terminal'], stdout=subprocess.PIPE, stdin=subprocess)
out = sitecreate_proc.communicate("pwd")
print out
the "out" variable is always empty.
Displaying the terminal is necessary.

gnome-terminal is a graphical application and as one, likely doesn't use its own standard streams that it got from the parent process.
You need to run console applications instead to communicate with them -
either the commands themselves:
>>> subprocess.check_output("pwd")
'/c/Users/Ivan\n'
or an interactive shell command, then send input to it and receive responses as per Interacting with bash from python
If you just need to output stream data to the same console that python is using, you can simply write out their data as you're getting it - either automatically with tee, or by hand at appropriate moments.
If, instead, you need to launch an independent terminal emulator window on a desktop and interact with it via IPC, that's another matter entirely - namely, UI automation, and has nothing to do with standard console streams.
The most common way for that in Linux is D-Bus (there are other options outlined on the previous link). Ppl report however (as of 2012) that gnome-terminal doesn't support D-bus and you have to jump through hoops to interact with it. There is an article on controlling konsole via D-Bus though.

As I remember communicate return a tuple,
communicate() returns a tuple (stdoutdata, stderrdata)
. so you can't user communicate("pwd"). gnome-terminal returns, then try to get that result, by sitecreate_proc.communicate()[0] for stroutdate, or sitecreate_proc.communicate()[0] for stderrdata

How to check if pdf printing is finished on linux command line

I have a bunch of files that I need to print via PDF printer and after it is printed I need to perform additional tasks, but only when it is finally completed.
So to do this from my python script i call command "lpr path/to/file.doc -P PDF"
But this command immediately returns 0 and I have no way to track when printing process is finished, was it successful or not etc...
There is an option to send email when printing is done, but to wait for email after I start printing looks very hacky to me.
Do you have some ideas how to get this done?
Edit 1
There are a plenty of ways to check if printer is printing something at current moment. Therefore at the moment after I start printing something I run lpq command every 0.5 second to find out if it is still printing. But this looks to m e not the best way to do it. I want to be able get alerted or something when actual printing process is finished. Was it successful or not etc...

If you have CUPS, you can use the System V-compatible lp instead of lpr. This prints, on stdout, a job id, e.g.
request id is PDF-5 (1 file(s))
(this is for the virtual printer cups-pdf). You can then grep for this id in the output of lpstat:
lpstat | grep '^PDF-5 '
If that produces no output, then your job is done. lpstat -l produces more status information, but its output will also be a bit harder to parse.
Obviously, there are cleaner Python solutions then running this actual shell code. Unfortunately, I couldn't find a way to check the status of a single job without plowing through the list of jobs.

You can check the state of the printer using the lpstat command (man lpstat). To wait for a process to finish, get the PID of the process and pass it wait command as argument