Develop Reference

Java ProcessBuilder cant run ".\venv\Scripts\active" in windows

Using Java's ProcessBuilder, I try to run a python script with activating virtual environment but i'm getting error like '.\venv\Scripts\activate" "' is not recognized as an internal or external command.
In command window cmd, I can use ".\venv\Scripts\activate" directly with no problem but in my code at below, error occured.
ProcessBuilder processBuilder = new ProcessBuilder("cmd.exe", "/c","cd C:\\Users\\onurc\\PycharmProjects\\OCR", "& .\\venv\\Scripts\\activate", "& python main.py");

That command doesn't look valid to me. You may be confused about what ProcessBuilder does / how windows (or any OS) works.
When you type a command in a dosbox or terminal, you're sending it to the shell - on linux, probably /bin/bash, or zsh or whatnot. On windows, you're sending it to cmd.exe. That program (so, bash or cmd) then decodes what you typed and does all sorts of fancy things to it, and then it in turns asks the underlying kernel to actually go run that app.
Java's ProcessBuilder does not invoke cmd/bash/zsh, it is a sibling to cmd/bash/zsh - it also does some fancy things, but far fewer fancy things. On windows, even the OS (or rather, the tool) does some fancy things whereas on linux it does not.
In general, trying to even guess whether the tool you run does fancy things, and which fancy things java's ProcessBuilder does, is not a sound way to write dependable software. Especially considering the relative hardship of testing this, given that it involves invoking external stuff.
Hence, you should opt out. Of all the fancy things. Thus:
Always call the command with a complete and absolute path.
Always use the multi-string/list version, never rely on java to split on spaces.
Never use *, ? or any other glob. Write out each and every parameter in full.
Do not try to use > or & or any other 'symbol that means something specific'. An OS doesn't support any of this stuff - you just pass [A] the full path to an executable and [B] a bunch of strings that the kernel will directly pass to the process without doing any processing on it. If you want to do stuff to standard out, error out, or standard in - then use Process's methods to do this.
Those &s in your command sure look like shellisms to me.
To fix:
You can tell the process which directory to run in, so there is no need for a cd command.
You should write the absolute path to CMD.exe. You can ask the environment. You're now relying on a combination of a properly configured PATH + the undefined behaviour of the JVM that does a bare minimum of fancy things (turning relative paths to absolute ones by PATH-scanning is a 'fancy thing' you should not be relying on).
I don't know what that & thing is, but you don't want that.
If activate is a script (shouldn't that end in .bat or .js or .vbs or whatnot?), run that, then run python separately. If activate sets things in the local environment table, make a script file that runs both and run the script instead.
Running python is not a good idea. Run C:\whatever\python.exe.
You have the right idea for activate, more or less (you're in a known dir and then write an actual path. If python.exe is guaranteeed to be in that OCR dir, run .\\python.exe, not python.

Convert command as a single string instead of passing separate arguments to the ProcessBuilder.
ProcessBuilder processBuilder = new ProcessBuilder("cmd.exe", "/c","cd C:\\Users\\onurc\\PycharmProjects\\OCR & .\\venv\\Scripts\\activate & python main.py");
Another way could be, if all command has to run under same folder, then change ProcessBuilder working directory using directory method:
Path rootDir = Paths.get("C:\\Users\\onurc\\PycharmProjects\\OCR");
ProcessBuilder processBuilder = new ProcessBuilder("cmd", "/C", ".\\venv\\Scripts\\activate & python main.py");
processBuilder.directory(rootDir.toFile());
Process process = processBuilder.start();
process.waitFor();

strange problem with running bash-scripts from python in docker

I have python-script, which run bash-scripts via subprocess library. I need to collect stdout and stderr to files, so I have wrapper like:
def execute_chell_script(stage_name, script):
subprocess.check_output('{} &>logs/{}'.format(script, stage_name), shell=True)
And it works correct when I launch my python script on mac. But If I launch it in docker-container (FROM ubuntu:18.04) I cant see any log-files. I can fix it if I use bash -c 'command &>log_file' instead of just command &>log_file inside subprocess.check_output(...). But it looks like too much magic.
I thought about the default shell for user, which launches python-script (its root), but cat /etc/passwd shows root ... /bin/bash.
It would be nice if someone explain me what happened. And maybe I can add some lines to dockerfile to use the same python-script inside and outside docker-container?

As the OP reported in a comment that this fixed their problem, I'm posting it as an answer so they can accept it.
Using check_output when you don't get expect any output is weird; and requiring shell=True here is misdirected. You want
with open(os.path.join('logs', stage_name)) as output:
subprocess.run([script], stdout=ouput, stderr=output)

How to make a command that runs Python a script in Windows command line?

Background:
I'm using Windows. I know some of programming with python. I don't know much about batch, which I think I might need to do what I want.
I will show a example so it becomes more clear what I'm trying to do.
Example:
When using git, after you install it, you can call the git command from anywhere of your computer, you can execute git commands, like git init and this will create a git file in your current folder.
I don't know exactly how git works or what language they use but I want to do the same thing, create my own command that I can execute from anywhere in my computer after I "install" my program.
What I'm trying to do:
I want to make my own command and when I call it, it executes a python script.
e.g.
I install my program and it creates a command called myprogram and when I type myprogram in the command line, it's like if I typed python myprogram.py. And myprogram -someargument would be the same as python myprogram.py -someargument.
What I tried until now:
I'm searched for How to make a environment variable that runs Python script? but I never get exactly what I want, is always something like How do I set environment variable using Python script?.
Maybe I'm making the wrong question and the result I want are not showing?
I'm looking for a explanation on how to do this or at least a tutorial/guide.
Edit 1:
As UnholySheep said in the comments, it's not environment variable, its commands, so I changed the question even does what I want to know is the same thing.

Files you need:
First you need a python script (obviously) so I created a file called myprogram.py that have this simple line:
print("This should be from a command")
After you need to make a batch file, in my case I used a .cmd file called myprogram.cmd that have:
#ECHO OFF
python_directory\python.exe python_script_directory\myprogram.py %*
Configurations to make:
You need to set in PATH environment variable the location of the batch file batch_file_directory\myprogram.cmd
And now if you execute in the command line myprogram it will print This should be from a command.
You can also use .exe or .bat files.

How to execute a system call in python using PyCharm on Debian Linux?

I am not able to run a system call using PyCharm and can't figure out what variables or environment settings to change.
Given this simple script:
import os
cmd = 'ifconfig -a'
os.system(cmd)
...which runs fine at the command line in terminal, yields the following error:
sh: ifconfig: command not found
This is happening with really any process I'm trying to run such as CSVSQL, PSQL, etc.
I have tried: Displaying my python interpreter paths dispayed at the command line, I tried adding them to the PyCharm interpreter paths, to no avail.
There are several other threads out there describing similar problems, but there doesn't seem to be a good solution that I have come across.
I'm running Linux Mint 19, though this works on my Windows installation (PATH output is much different).
My apologies if this is really simple... Thank you!

Run printenv on both Python and terminal, and check the PATH variable. Use os.environ['PATH'] = 'My path' to set it to what you saw on the terminal.
For future issues (That I've run into):
A quick way to check if it's an exported environment variable is to run os.system("/bin/sh -c \"MYCMD\""), and then run the same "/bin/sh -c \"MYCMD\"" string in your terminal. If there's still a problem, then it must be an export (And this is the likely issue).
To resolve this, try printenv in both python and the terminal to see the list of exports. You should see a discrepancy. The format is simple as you can simple copy the output of the terminal's printenv (Which should be a series of declares), and paste it into python so python will get the same variables. Then your "/bin/sh CMD" calls should align.
The wrapped /bin/sh is in case they're running different shells or have different local variables. echo $SHELL can confirm this, at which point you can compare sets and printenvs and copy paste in the same way. Wrapped you only have to compare exports, as that's what get passed to child processes.

Looks like pycharm isn't getting the PATH from your profile or rc. Try giving the absolute path of the command.
import os
cmd = '/sbin/ifconfig -a'
os.system(cmd)
You can also verify your path using following.
print(os.environ['PATH'])
And use following to add your custom path to current env path.
os.environ['PATH'] += ':/sbin'

How do I change directory in python so it remains after running the script?

I'm trying to change the terminal directory through a python script. I've seen this post and others like it so I know about os.chdir, but it's not working the way I'd like. os.chdir appears to change the directory, but only for the python script. For instance I have this code.
#! /usr/bin/env python
import os
os.chdir("/home/chekid/work2/")
print os.getcwd()
Unfortunately after running I'm still in the directory of the python script (e.g. /home/chekid) rather than the directory I want to be in. See below.
gandalf(pts/42):~> pwd
/home/chekid
gandalf(pts/42):~> ./changedirectory.py
/home/chekid/work2
gandalf(pts/42):~> pwd
/home/chekid
Any thoughts on what I should do?
Edit: Looks like what I'm trying to do doesn't exist in 'normal' python. I did find a work around, although it doesn't look so elegant to me.
cd `./changedirectory.py`

You can't. The shell's current directory belongs to the shell, not to you.
(OK, you could ptrace(2) the shell and make it call chdir(2), but that's probably not a great design, won't work on Windows, and I would not begin to know how to do it in pure Python except that you'd probably have to mess around with ctypes or something similar.)
You could launch a subshell with your current working directory. That might be close enough to what you need:
os.chdir('/path/to/somewhere')
shell = os.environ.get('SHELL', '/bin/sh')
os.execl(shell, shell)
# execl() does not return; it replaces the Python process with a new shell process
The original shell will still be there, so make sure you don't leave it hanging around. If you initially call Python with the exec builtin (e.g. exec python /path/to/script.py), then the original shell will be replaced with the Python process and you won't have to worry about this. But if Python exits without launching the shell, you'll be left with no shell open at all.

You can if you cheat: Make a bash script that calls your python script. The python script returns the path you want to change directory to. Then the bash script does the acctual chdir. Of course you would have to run the bash script in your bash shell using "source".

The current working directory is an attribute of a process. It cannot be changed by another program, such as changing the current working directory in your shell by running a separate Python program. This is why cd is always a shell built-in command.

You can make your python print the directory you want to move to, and then call your script with cd "$(./python-script.py)". In condition your script actually does not print anything else.