Develop Reference

How do I get a python script to run a command shell in Windows (10) where it opens in a program specific location?

I need to run commands in command prompt but they only work when the command prompt is set at a particular location in the system. I need the following commands to run in a python script:
import os
os.system("set OMP_NUM_THREADS=2")
os.system("explorer.exe /e,::{20D04FE0-3AEA-1069-A2D8-08002B30309D}"#
os.system("cd C:\CFD\crit_vel_01_02")
os.system("mpiexec -n 9 FDS crit_vel_01_02.fds")
os.system("PAUSE")
the system does not recognise the command
os.system("mpiexec -n 9 FDS crit_vel_01_02.fds")
unless this is run in the command shell which is installed on installation of the program "fds" which is a fire dynamics simulator. I appreciate this seems quite specific to the program but I am assuming there is some generic way that python can run command shell from a different location/with different settings.
The shortcut to the command prompt is called CMDfds and is installed in:
"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\FDS6"
in the properties the target in the shortcut tab is:
"C:\Windows\System32\cmd.exe /k fdsinit"

Not sure it will work but you can give a try at subprocess.run with shell=True.
If shell is True, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of ~ to a user’s home directory.
Also try running the python script from the fds command shell. It seems to be initializing stuff in the shell.

The trouble with running programs with system commands is that they often have a different shell environment. In order to prevent problems arising from this it's a good idea to use absolute paths. In your case:
os.system("mpiexec -n 9 FDS crit_vel_01_02.fds")
should be changed to:
os.system("/absolute/path/to/mpiexec -n 9 FDS crit_vel_01_02.fds")

R: system() cannot use bash function defined in .bashrc

Question
My default Python is 2.7, but I have a script that requires Python 3.4. I am trying to create a function in R that will:
Switch to Python 3.4
Run this script
Switch back to Python 2.7
Import results into R
To switch between Python versions, I use my cluster's "dotkit" system, like this:
use Python-2.7
use Python-3.4
"use" is a bash function that is imported in my .bashrc file. It sets all of my path variables (PATH, LIBRARY_PATH, LD_LIBRARY_PATH, CPATH, C_INCLUDE_PATH, etc). The problem is that when I try to call this function in R, I get the following error:
system('use Python-3.4')
sh: use: command not found
It seems like this is a problem with my PATH. I am using the correct shell:
system('echo $SHELL')
/bin/bash
My $PATH variable also looks good. However, when I create a script that essentially does the same thing:
load_py34.sh:
#!/bin/bash
source ~/.bashrc
use Python-3.4
and call this script through R, then it actually runs, but for some reason, it doesn't change my python version within R. (I have verified that this script works from the command line.)
> R
> system('python --version')
Python 2.7.1
> system('sh load_py34.sh')
Prepending: R-3.4 (ok)
> system('python --version')
Python 2.7.1
So I'm a little confused, but if anyone can help, I would really appreciate it.
Suggested fixes
When I combine them into a single command, I still have the same problem:
> system("sh load_py34.sh; python --version")
Prepending: Python-3.4 (already loaded)
Python 2.7.1
When I try calling bash directly, I still have a problem with the PATH:
> system("bash -c 'use Python-3.4; python --version'")
bash: use: command not found
Python 2.7.1

.bashrc is only loaded for interactive bash sessions.
"use" is a bash function that is imported in my .bashrc file. It sets
all of my path variables.
If set via export, the environment of the calling process will not be altered.
export [-fn] [name[=word]] ... The supplied names are marked for automatic export to the environment of subsequently executed commands. (https://man7.org/linux/man-pages/man1/bash.1.html)
Child processes do not normally have access to the parent process' environment. (This poses a problem because system() creates a sub-process.)
The source and . built-ins execute the commands in the current shell environment, hence why your script works.
Other commands (executables, non-shell-builtins) are executed by the fork-and-exec mechanism, whereby the executing shell process forks, creating a child process with an identical environment and state. This new child process is the process in which the command is executed. Changes to the environment of that process are not replicated to the parent's environment.
This means that you will not be able to rely on system('...') to modify the environment of the R process, or that of processes spawned by subsequent system() invocations.
In a single invocation to system(), you can construct a command-line that changes the environment of the spawned shell like so:
bash -c 'source ~/.bashrc; use Python-3.4; python --version'
Mind you, ~/.bashrc is not really the best place to put this functionality (might be subjective).

When you call system() it uses /bin/sh, not /bin/bash. sh doesn't read your .bashrc file when it starts up, so it does not know any of the functions you've defined there.
To use the function from your .bashrc, you must get bash to run it instead:
system("bash -c 'use Python-3.4; python --version'")
Edit: placement of closing single quote.

I have edited .profile , but os.environ doesn't see my variables [duplicate]

To test its effects immediately instead of having to reload the terminal

use source /etc/profile
for details, man source or you can checkout this link http://bash.cyberciti.biz/guide/Source_command

If this is bash you can do . /etc/profile, it's the same as doing source /etc/profile.

As sjr says, you can approximate the effect of the change by re-reading /etc/profile using the dot . (or, in Bash, source) command.
However, you need to be aware that /etc/profile gets to work with a more minimal starting environment, so the effect you get by rereading the profile is not necessarily identical to the effect you get on login. You can simulate the original environment more accurately using the env command to unset the environment. With bash, you can use the -l option to make the shell behave more like a login shell - in conjunction with env, you can simulate the login environment accurately:
env -i HOME=$HOME PATH=/bin:/usr/bin USER=$USER ... bash -l

See stdout when running bash script in PyCharm

I use a bash script to call several python scripts. I installed the bash plugin for PyCharm. I can run the script, but I don't see stdout during runtime, even though I see it after everything finished. How can I make that visible during runtime?

Without all of the required information, my guess would be that this is due to Python buffering its output, which is its default behavior. You can easily disable this by passing python the -u flag or by setting the PYTHONUNBUFFERED environment variable.
This is described in this SO answer.

Python mode in Emacs: No such file or directory, pdb

I have a python script that I want to debug with python-mode. I read in this thread that I can debug my python script with M-x pdb, however I get the following error:
Searching for program: no such file or directory, pdb
I can provide python -m pdb my_source_file.py in the prompt in the minibuffer, but it would be nice if Emacs could infer this command directly from the file on which I run M-x pdb
Update:
Running on:
Red Hat Enterprise Linux Server release 5.1 (Tikanga)
Emacs 23.3.1
Differences between paths
I get different paths when I run M-: exec-path and when I run M-: (getenv "PATH") (the one returned by M-: (getenv "PATH") is longer).
With this:
Where is pdb located? How can I add it to the Emacs path?
Is there a way to ask Emacs to also look into the paths held by the environment variable PATH?

Further to my comment earlier, and your subsequent update to the question:
First figure out a value for $PATH that works in your terminal. Use which pdb to find where the pdb executable is located.
Then, set the $PATH environment variable explicitly in Emacs, and sync it to exec-path as follows:
(setenv "PATH" "/usr/local/bin:/usr/bin:/bin:/some/other/dir")
(setq exec-path (split-string (getenv "PATH") path-separator))
It's possible you would need to also explicitly set PYTHONPATH or similar environment variables; you can do that using lines like the "setenv" line above, or just use the exec-path-from-shell elisp package.
Update
Okay, so it turns out Emacs' pdb command isn't provided by python-mode, and it expects to find an executable called "pdb". The easy way to fix this, then is to create a shell wrapper called "pdb", in a directory on your $PATH:
#!/bin/sh
exec python -m pdb "$#"
(I found a note here suggesting this technique.)
The equivalent under Windows would be a file called pdb.bat, containing:
python -u -m pdb %1
(The -u prevents Python from buffering its output.)

To run the Python Debugger, M-x pdb expects to find an executable named pdb. While the pdb executable may exist in some Python distributions, it doesn't exist in all of them.
A proposal to fix this is in GNU bug report #21521: pdb default suggested command.
Until the bug is fixed, you can set the variable gud-pdb-command-name to define the command used to launch pdb. In .emacs, add...
(setq gud-pdb-command-name "python -m pdb")

At a shell prompt type
which pdb
In Emacs, type M-x customize. Select Programming > Tools > Gud. Set the value of gud-pdb-command-name to the path returned by which pdb.
If your version of Emacs presents a different organization for the customize menu, you could also try
C-h v gud-pdb-command-name
Then click on the customize link, and set the path to pdb there.
Though the instructions above are different, I found this out by reading "Running pdb under emacs" .

You can create a custom command like this:
;; PDB command line
(defun user-python-debug-buffer ()
"Run python debugger on current buffer."
(interactive)
(setq command (format "python -u -m pdb %s " (file-name-nondirectory buffer-file-name)))
(let ((command-with-args (read-string "Debug command: " command nil nil nil)))
(pdb command-with-args)))

In Emacs 23.3.1 and presumably higher, yet another variation is to use the Emacs shell, Eshell (M-x eshell). Under Eshell, there's a pre-existing, Lisp-based definition of pdb. These Lisp functions work in Eshell just like ordinary shell commands.
So pdb "./manage.py runserver" will start a Django server, for instance.

Everyone is going wild saying you gotta make a pdb file and make it an executable and then type ./pdb your_code.py. It is easier than that.
Be where you want to run the debugger from. Probably in your python file, maybe use M-x cd to get somewhere.
Then type: M-x pdb
It will prompt you with:
Run pdb (like this):
You want to make that look like:
Run pdb (like this): python -m pdb your_code.py
Sometimes if I want to run my code as a module.
Run pdb (like this): python -m pdb -m some_package.my_code
Then type help and go read this https://docs.python.org/3/library/pdb.html

My answer builds on what #Chad Nouis mentioned.
a link
However, I've added this to python-mode, everytime python-mode loads, it will set gud-pdb-command-name to "python -m pdb"
;; Set the PDB command
(add-hook 'python-mode-hook
(lambda () (setq gud-pdb-command-name "python -m pdb")))