Observed an interesting bash behavior that I don't understand.
Basically, when chaining commands in bash, whether there's a semi-colon between the two commands seem to make a difference in bash behavior. In particular the two examples below, one can read the env variable in Python, the other cannot
yuli#yuli-desktop:~$ A=1; python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ['A']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'A'
>>> exit()
In a new bash process:
yuli#yuli-desktop:~$ A=1 python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>>
>>>
>>> os.environ['A']
'1'
You're not chaining two commands in the second example.
When you run a command in bash, you can set per-command environment variables by prefixing the command line with variable assignments, as in:
A=1 B=2 COLOR=red python3
In this case, those variables (A, B, COLOR) will be available in the environment of the command:
>>> os.getenv('COLOR')
'red'
In the first example, the semicolon splits the command line into two independent commands. Since the python3 command has no variable assignments associated with it, you're not setting any environment variables. Instead, you've set a shell-local variable A that would be visible in the shell but not in any child processes.
You can see documentation about this in the "SHELL GRAMMAR" section of the bash man page.
In comments, ErikMD mentions the export shell built-in, which is used to set persistent environment variables:
$ export COLOR=red
Or:
$ COLOR=red
$ export COLOR
An environment variable set this way will persist until explicitly unset and will be available in all subsequent commands. Contrast that with a variable set as part of a command invocation (as discussed earlier in this answer), which is only available for the duration of that command.
Related
I just downloaded the latest Git Bash (2.36.1 64-bit) and installed Python 10. I'm running Windows 11. Among a couple of other unexpected changes from my earlier setup using a previous version of Git Bash and mostly running Python 3.9 (i.e. I now have to run python -i for the interactive python interpreter, for all 2.x and 3.x versions, instead of just running python), the most frustrating is that ctrl+z followed by Enter no longer quits from the interpreter. Instead I must call exit(). Ex:
user#User MINGW64 ~
$ python -i
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> # ctrl+z, Enter pressed here
File "<stdin>", line 1
^
SyntaxError: invalid syntax
>>> exit()
user#User MINGW64 ~
$
The same is true for my other installations of python 3 (3.7, 3.8, 3.9), but python 2.7 still has the expected behavior:
user#User MINGW64 ~
$ $PYTHON\\Python27\\python -i
Python 2.7.18 (v2.7.18:8d21aa21f2, Apr 20 2020, 13:25:05) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> # ctrl+z, Enter pressed here
user#User MINGW64 ~
$
What I've tried
I've done quite a bit of research but can't figure out what may have changed. In the earlier version of Git Bash, I remember that ^Z, ^X, ^C, and ^V characters (and maybe more) would display explicitly in the console when typed. Curiously, pressing ctrl+c followed by Enter comes with the error message below. Did some sort of character encoding change in the newer versions of Git Bash?
user#User MINGW64 ~
$ python
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> # ctrl+c, Enter pressed here
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 14, in decode
def decode(self,input,errors='strict'):
KeyboardInterrupt
The above exception was the direct cause of the following exception:
KeyboardInterrupt: decoding with 'cp1252' codec failed (KeyboardInterrupt: )
>>>
I am too lazy to constantly type exit(). How can I restore the functionality where pressing ctrl+z, then Enter quits the interpreter for my python 3 versions?
After stumbling upon this answer, I've found a solution that restores the functionality I wanted. Instead of:
alias python3='.../Python310/python
in which case you must get to the interactive interpreter using python3 -i, and ctrl+Z keystrokes are not recognized, use:
alias python3='winpty .../Python310/python
If it's helpful to anyone, here's how I've set up my ~/.bash_aliases:
alias python2='winpty C:\Users\<user>\AppData\Local\Programs\Python/Python27/python'
alias python3.7='winpty C:\Users\<user>\AppData\Local\Programs\Python/Python37/python'
alias python3.8='winpty C:\Users\<user>\AppData\Local\Programs\Python/Python38-32/python'
alias python3.9='winpty C:\Users\<user>\AppData\Local\Programs\Python/Python39/python'
alias python3.10='winpty C:\Users\<user>\AppData\Local\Programs\Python/Python310/python'
alias python3='python3.10'
alias python='python2'
and these lines must be in ~/bashrc:
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
Now you can open the python interpreter without requiring -i (just run python), and ctrl+Z, Enter exits it.
Edit: After reading this answer, I reinstalled Git Bash with "Enable experimental support for pseudo consoles" checked and functionality is restored without needing to run Python through winpty.
I'm a beginner in Python. I tried to resolve this error but I couldn't. This code worked before but not anymore. I run the code in PyCharm and getting this error:
Traceback (most recent call last):
File "C:/Users/MJavad/Desktop/test.py", line 3, in <module>
b = float(sys.argv[1])
IndexError: list index out of range
I ran CMD and had also an error:
File "<stdin>", line 1
python test.py 1 2
^
SyntaxError: invalid syntax
Can anyone help, please? This is my code:
import sys
import math
b = float(sys.argv[1])
c = float(sys.argv[2])
f = b * b - 4.0 * c
d = math.sqrt(f)
print((-b + d) / 2.0)
print((-b - d) / 2.0)
and this is the code and error in PyCharm:
It seems that there is confusion about how Python code can be executed and processed.
On the one hand, there is the Python interpreter in the interactive mode. This is usually started with the command python (without arguments) and then you have the possibility to execute Python code directly in an interactive Python specific shell. This distinguishes Python from other languages that need to be compiled first to execute code. Further information are available in the official Python tutorial.
On the other hand, Python can also be executed in such a way that not the interpreter with an interactive shell is started, but a file is read and processed. This is usually done with the command python together with the path to the Python file as argument, e.g. python test.py. See also the documentation about using Python.
With this knowledge the problems that have happened to you can now be explained and solved:
If you are simply starting the Python interpreter in interactive mode (without any further arguments), you don't have access to the command line arguments any more, for example:
$ python3.8 # or whatever your command is, maybe only python or python3
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.argv
['']
As you can see, there isn't really a usable information in argv. And that is your problem: The arguments aren't successfully loaded into sys.argv. So an index error happened, because the arguments are simply missing:
$ python3.8
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.argv[1]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
The only difference to your example is that you have already provided the path to the script, because it's File "C:/Users/MJavad/Desktop/test.py", line 3, in <module> instead of File "<stdin>", line 1, in <module>. So you have started the program via python test.py, but also without any further arguments which would be loaded into sys.argv in the program, see:
$ python3.8 test.py
Traceback (most recent call last):
File "test.py", line 3, in <module>
b = float(sys.argv[1])
IndexError: list index out of range
Your sys.argv now looks like this: ['test.py'], but still no index positions 1 and 2 available. So you have to invoke python also with additional arguments which will be passed into sys.argv:
$ python3.8 test.py 1 2
Traceback (most recent call last):
File "test.py", line 6, in <module>
d = math.sqrt(f)
ValueError: math domain error
And it worked! Ok, you have another exception, but it's in line 6 and every line before was successfully processed, also the command line arguments. Now you can proceed to debug your program, i.e. start programming or trying other parameters than 1 and 2 etc.
So that's the theory. Since you're also using PyCharm and have the "Terminal", the "Run configuration" and a "Python Console" available, things get a bit more complicated:
The "Terminal" should have a prompt available if you start one. This prompt shouldn't be a prompt from the Python interpreter (normally prefixed by >>>). It should be a terminal prompt (normally prefixed by an $ at the end), where you can also start a python interpreter in interactive mode or start a python program as described above. This "Terminal" is a terminal emulator given you by PyCharm and can also do other things for you, not only starting python programs. See the PyCharm documentation for more information.
The "Python Console" is a similar Python interpreter you also can get if starting python from the "Terminal". But then, you already started the interactive mode of the interpreter and there are no possibilities to pass command line arguments (maybe somewhere, but not as default). See the PyCharm documentation for more information.
If you're using an IDE like PyCharm, you should normally start the program as recommended by the IDE. In this case, you're writing a file and start the file neither by running the "Terminal", nor going into an interactive Python shell. Instead of this, you have to configure the IDE "Run" as described in the PyCharm documentation or as you can see here:
This is just the GUI way of calling python C:/Users/MJavad/Desktop/test.py 1 2 directly inside PyCharm.
So I would recommend that you're only starting your programs via option 3 (via "Run" configuration or "DEBUG" configuration). You only have to pay attention, running the right configuration (see if the path is the correct one and the parameters are right).
It is not normal to have a Python prompt (>>>) directly after starting a "Terminal", though. And inside interactive mode of Python's interpreter, you simply cannot start a python script, because you're already in a python interpreter, for example:
$ python3.8
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> python test.py 1 2
File "<stdin>", line 1
python test.py 1 2
^
SyntaxError: invalid syntax
I should also mention that you can pass arguments into python interactive mode, for example (the - stands for <stdin>):
$ python3.8 - hello world
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.argv
['-', 'hello', 'world']
>>>
I am using virtualenv to run a script that uses subprocess popen to run another program that requires the system wide python and original environment variables. How do I prevent virtualenv from affecting it?
You can pass in a modified PATH for the subprocess with env=.
from subprocess import Popen
from os import environ
from os.path import join as path_join
myenv = environ.copy()
if 'VIRTUAL_ENV' in environ:
myenv['PATH'] = ':'.join(
[x for x in environ['PATH'].split(':')
if x != path_join(environ['VIRTUAL_ENV'], 'bin')])
Popen(['command', '--with', 'arguments'], env=myenv)
virtualenv creates a copy of python executable and you can activate it to your current shell:
This will change your $PATH so its first entry is the virtualenv’s
bin/ directory. (You have to use source because it changes your shell
environment in-place.) This is all it does; it’s purely a convenience.
If you directly run a script or the python interpreter from the
virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or
/path/to/ENV/bin/python-script.py) there’s no need for activation.
So when I've activated python in a virtualenv for my project it's the one that will be used:
gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ source ../venv/bin/activate
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ which python
/home/gonczor/Projects/papukurier/venv/bin/python
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ python
Python 2.7.14 (default, Sep 23 2017, 22:06:14)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/home/gonczor/Projects/papukurier/venv/bin/python'
>>>
But at the same time you can give full path to execute any other python instance on your computer:
(venv) gonczor#wiktor-papu:~/Projects/papukurier/papukurier$ /usr/bin/python
Python 2.7.14 (default, Sep 23 2017, 22:06:14)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/usr/bin/python'
>>>
I believe subprocess doesn't care about you venv
subprocess.run('/path/to/system/python program.py', stdout=PIPE, stderr=PIPE)
I have a script that calls os.execvp into another Python instance. After doing this, I appear to be attached to a cmd.exe instance, not the Python instance I just created. The Python instance responds to Ctrl+C however.
H:\bin>Python 3.2.1 (default, Jul 10 2011, 21:51:15) [MSC v.1500
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print('hi')
Can't find file ('hi')
H:\bin>
H:\bin>
KeyboardInterrupt
>>> echo hi
hi
The call to exec:
from sys import argv
os.execvp('python', argv)
How do I replace the original Python instance with the new one, as per the behaviour one might see on Linux?
On Unix executing binaries is split into two stages - fork(3) to clone current process and exec(3) to load executable into address space. On windows there is only CreateProcess which does the same thing as fork+exec.
For portability your best bet is to use subprocess.Popen (which also does proper filename quoting on Windows unlike os.* counterparts) as in http://docs.python.org/library/subprocess.html#replacing-the-os-spawn-family
So today I upgraded to bazaar 2.0.2, and I started receiving this message (I'm on snow leopard, btw):
bzr: warning: unknown locale: UTF-8
Could not determine what text encoding to use.
This error usually means your Python interpreter
doesn't support the locale set by $LANG (en_US.UTF-8)
Continuing with ascii encoding.
very strange, since my LANG is actually empty. Similar thing happen when I try to tinker with the locale module
Python 2.5.4 (r254:67916, Nov 30 2009, 14:09:22)
[GCC 4.3.4] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/sbo/runtimes/lib/python2.5/locale.py", line 443, in getdefaultlocale
return _parse_localename(localename)
File "/Users/sbo/runtimes/lib/python2.5/locale.py", line 375, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
exporting LANG does not help
sbo#dhcp-045:~ $ export LANG=en_US.UTF-8
sbo#dhcp-045:~ $ bzr
bzr: warning: unknown locale: UTF-8
Could not determine what text encoding to use.
This error usually means your Python interpreter
doesn't support the locale set by $LANG (en_US.UTF-8)
Continuing with ascii encoding.
However, this solved the problem
sbo#dhcp-045:~ $ export LANG=en_US.UTF-8
sbo#dhcp-045:~ $ export LC_ALL=en_US.UTF-8
Python 2.5.4 (r254:67916, Nov 30 2009, 14:09:22)
[GCC 4.3.4] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('en_US', 'UTF8')
Could you please explain what's going on here, for better googlability ?
2016 UPDATE: Turns out that this is a Python bug since at least 2013, very probably earlier too, consisting in Python not reacting well to non-GNU locales - like those found in Mac OS X and the BSDs. The bug is still open as of September 2016, and affects every Python version.
If there was no LANG environment variable set, chances are you had either an LC_CTYPE (the key variable) or LC_ALL (which overrides if set) environment variable set to UTF-8, which is not a valid OS X locale. It's easy enough to reproduce with the Apple-supplied /usr/bin/python or with a custom python, as in your case, that was built with the 10.6 SDK (probably also the 10.5 SDK). You won't be able to reproduce it that way with a python.org python; they are currently built with the 10.4 SDK where the locale APIs behave differently.
$ unset LANG
$ env | grep LC_
$ export LC_CTYPE="UTF-8"
$ /usr/bin/python # Apple-supplied python
Python 2.6.1 (r261:67515, Jul 7 2009, 23:51:51)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale ; locale.getdefaultlocale()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/locale.py", line 459, in getdefaultlocale
return _parse_localename(localename)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/locale.py", line 391, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
^D
$ /usr/local/bin/python2.6 # python.org python
Python 2.6.4 (r264:75821M, Oct 27 2009, 19:48:32)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale ; locale.getdefaultlocale()
(None, 'mac-roman')
>>>
EDIT:
There may be another piece to the puzzle. A quick look at the bzr 2.0.1 I have installed indicates that the message you cite should only show up if locale.getpreferredencoding() raises a locale.Error. One way that can happen is if the python _locale.so C extension can't be loaded and that can happen if there are permission problems on it. For example, MacPorts currently is known to have problems setting permissions if you have a customized umask; I've been burned by that issue myself. Check the permissions of _locale.so in the python lib/python2.5/lib-dynload directory and ensure it is 755. The full path for MacPorts should be:
/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/
I faced the same problem. When I ran locale, I noticed that the LANG and LC_ALL were unset. So I fixed this by adding the following lines in the .bash_profile file:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
Then I simply ran:
source ~/.bash_profile
And this issue was fixed thereafter on my Mac.
It's a Mac OS X problem. To see your locale settings, run locale in terminal. locale -a should list all locales that you have defined (that you may use as argument to LC_ALL).
Notice that LC_ALL and other LC_* variables take precedence over LANG when defined.