Develop Reference

poetry [EnvCommandError] Fatal Python error: Py_Initialize: can't initialize sys standard streams

I have a pyproject.toml file that was created using poetry, and I'm trying to run the command poetry install in this directory to create a poetry.lock file. However, when I run poetry install, I get the following error:
EnvCommandError
Command C:\Users\myName\AppData\Local\pypoetry\Cache\virtualenvs\my-app-name-kS94etse-py3.8\Scripts\python.exe -
errored with the following return code 3, and output:
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "c:\users\myName\anaconda3\lib\io.py", line 52, in <module>
File "c:\users\myName\anaconda3\lib\abc.py", line 106
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
input was : import sys
This error seems pretty in depth but I dont understand why poetry thinks import sys is invalid syntax or even where that's being called to create an issue. Any suggestions on how to fix this?

Resolved here:
https://stackoverflow.com/a/67169273/12060361
Download a new, non-broken version of Python (the same version defined in your pyproject.toml file to use): https://www.python.org/downloads/release/python-385/
Navigate to the virtual env folder poetry says the error is coming from: C:\Users\myName\AppData\Local\pypoetry\Cache\virtualenvs\my-app-name-kS94etse-py3.8\Scripts\python.exe
Replace the python.exe currently in that folder with the new one you just downloaded

Fix "Fatal Python error: Py_Initialize: can't initialize sys standard streams"

I have two python environments with different versions running in parallel. When I execute a python script (test2.py) from one python environment in the other python environment, I get the following error:
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\io.py", line 52, in <module>
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\abc.py", line 147
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
So my setup is this:
Python 3.7
(test.py)
│
│ Python 3.5.6
├───────────────────────────────┐
┆ │
┆ execute test2.py
┆ │
┆ 🗲 Error
How can I fix this?
For dm-script-people: How can I execute a module with a different python version in Digital Micrograph?
Details
I have two python files.
File 1 (test.py):
# execute in Digital Micrograph
import os
import subprocess
command = ['C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe',
os.path.join(os.getcwd(), 'test2.py')]
print(*command)
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print("Subprocess result: '{}', '{}'".format(result.stdout.decode("utf-8"), result.stderr.decode("utf-8")))
and File 2 (test2.py)
# only executable in python 3.5.6
print("Hi")
in the same directory. test.py is executing test2.py with a different python version (python 3.5.6, legacy environment).
My python script (test.py) is running in the python interpreter in a third party program (Digital Micrograph). This program installs a miniconda python enviromnent called GMS_VENV_PYTHON (python version 3.7.x) which can be seen in the above traceback. The legacy miniconda environment is used only for running test2.py (from test.py) in python version 3.5.6.
When I run test.py from the command line (also in the conda GMS_VENV_PYTHON environment), I get the expected output from test2.py in test.py. When I run the exact same file in Digital Micrograph, I get the response
Subprocess result: '', 'Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\io.py", line 52, in <module>
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\abc.py", line 147
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
'
This tells me the following (I guess):
The test2.py is called since this is the error output of the subprocess call. So the subprocess.run() function seems to work fine
The paths are in the GMS_VENV_PYTHON environment which is wrong in this case. Since this is test2.py, they should be in the legacy paths
There is a SyntaxError because a f-string (Literal String Interpolation) is used which is introduced with python 3.6. So the executing python version is before 3.6. So the legacy python environment is used.
test2.py uses either use io nor abc (I don't know what to conclude here; are those modules loaded by default when executing python?)
So I guess this means, that the standard modules are loaded (I don't know why, probably because they are always loaded) from the wrong destination.
How can I fix this? (See What I've tried > PATH for more details)
What I've tried so far
Encoding
I came across this post "Fatal Python error: Py_Initialize: can't initialize sys standard streams LookupError: unknown encoding: 65001" telling me, that there might be problems with the encoding. I know that Digital Micrograph internally uses ISO 8859-1. I tried to use python -X utf8 and python -X utf8 (test2.py doesn't care about UTF-8, it is ASCII only) as shown below. But neither of them worked
command = ['C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe',
"-X", "utf8=0",
os.path.join(os.getcwd(), 'test2.py')]
PATH
As far as I can tell, I think this is the problem. The answer "https://stackoverflow.com/a/31877629/5934316" of the post "PyCharm: Py_Initialize: can't initialize sys standard streams" suggests to change the PYTHONPATH.
So to specify my question:
Is this the way to go?
How can I set the PYTHONPATH for only the subprocess (while executing python with other libraries in the main thread)?
Is there a better way to have two different python versions at the same time?
Thank you for your help.
Background
I am currently writing a program for handling an electron microscope. I need the "environment" (the graphical interface, the help tools but also hardware access) from Digital Micrograph. So there is no way around using it. And DigitalMicrograph does only support python 3.7.
On the other hand I need an external module which is only available for python 3.5.6. Also there is no way around using this module since it controlls other hardware.
Both rely on python C modules. Since they are compiled already, there is no possibility to check if they work with other versions. Also they are controlling highly sensitive aperatures where one does not want to change code. So in short words: I need two python versions parallel.

I was actually quite close. The problem is, that python imports invalid modules from a wrong location. In my case modules were imported from another python installation due to a wrong path. Modifying the PYTHONPATH according to "https://stackoverflow.com/a/4453495/5934316" works for my example.
import os
my_env = os.environ.copy()
my_env["PYTHONHOME"] = "C:\\ProgramData\\Miniconda3\\envs\\legacy"
my_env["PYTHONPATH"] = "C:\\ProgramData\\Miniconda3\\envs\\legacy;"
my_env["PATH"] = my_env["PATH"].replace("C:\\ProgramData\\Miniconda3\\envs\\GMS_VENV_PYTHON",
"C:\\ProgramData\\Miniconda3\\envs\\legacy")
command = ["C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe",
os.path.join(path, "test2.py")]
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=my_env)
For Digital Micrograph users: The python environment is saved in the global tags in "Private:Python:Python Path". So replace:
import DigitalMicrograph as DM
# ...
success, gms_venv = DM.GetPersistentTagGroup().GetTagAsString("Private:Python:Python Path")
if not success:
raise KeyError("Python path is not set.")
my_env["PATH"] = my_env["PATH"].replace(gms_venv, "C:\\ProgramData\\Miniconda3\\envs\\legacy")

I had set "PYTHONPATH" as "D:\ProgramData\Anaconda3" for my python (base) python environment before, but i found when I had changed to another env my python still import basic python package from "D:\ProgramData\Anaconda3",which means it get the wrong basic package with the wrong "System environment variables" config.
so I delete "PYTHONPATH" from my windows "System environment variables", and that will be fixed.

Using "python --version" command bringing up error message

I have Python 3.7.1 on my Windows 10 PC. I want to install Pandas and firstly need to do some checks to see if I have everything required. I tried typing the following to double check the Python version:
python --version
but it keeps spitting out the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'python' is not defined
According to the Pandas website, if this error occurs it's because I need to type it into Python Shell. But that's what I'm doing... I know I'm probably missing something really obvious, but I'd really appreciate any hints and tips for this very basic problem...
Thanks!

You shouldn't be entering python --version in a Python shell. Enter it in the cmd.exe shell.

You probably get this Error:
The answer from RasikhJ is right, except you have to use brackets in your Python3 version:
import sys
print (sys.version)
If you just want to know your Python version, then use the command line:
Search for cmd or command line on your Windows
Change your folder to the "python.exe" file (something like C:/Users/Username/AppData/Local/Programs/Python/Python36-32) with the "cd" command
Now you can type in "python --version" and you get your Version

Use the sys package:
import sys
print sys.version
Or use the platform module:
import platform
print platform.python_version()

You should try py --version in cmd.
C:\Users\XYZ>py --version
Python 3.8.5

Try py --version in cmd.
C:\Users\XYZ>py --version
Python 3.8.3

If you want to see python version in python prompt
import sys
print(sys.version)
The command which you are trying meant for cmd/terminal prompt. Also your python environment should be set to get below results

Executing a python script from scala giving error while running the same script from the command line does NOT

This is my python script:
!/usr/bin/python
import sys
import magic
m=magic.from_file('<file with absolute path goes here>')
print(m)
On running this from the command line:
$ python script.py
Microsoft Word 2007+
results in the output of the TYPE of the document that file is. Using python-magic.
Now running the same script from scala using below code:
import sys.process._
def compWithLibmagic(){
val result = "python /script.py" !
}
throws the below error:
Traceback (most recent call last):
File "script.py", line 3, in <module>
import magic
ImportError: No module named magic
PS: I have both python 2.7 and python 3.6 installed on my machine and running the script using any of them from the command line runs just fine so I guess both of them are bundled with the MAGIC packages correctly.
Would highly appreciate any kind of help.

I will try to answer my own question in the best possible way.
While trying to execute the script.py from the command line using python compiler python2.7 was being used and when I tried to invoke the same script.py from the scala compiler using
import sys.process._
def compWithLibmagic(){
val result = "python /script.py" !
}
it was using python3 which was getting unnoticed. It took me a couple of days to actually figure this thing out and finally I removed all the python2.7 and python3 libraries and installed python 2.7 again and it worked like a charm.

It works from both python3 command and Scala code for me.
This should work.
import sys.process._
def compWithLibmagic(){
val result = "python3 script.py".!!
}

Newbie issue with python, script works fine on other os

I am using a bot with python to create accounts on a website. When I use it on my windows machine with python installed, it works fine. I bought a linux VPS yesterday (Ubuntu 16.04) and my script apparently isn't working anymore. This is the error I get on my linux machine:
Traceback (most recent call last):
File "roblox.py", line 2, in <module>
from utils import *
File "/home/py/newbot/utils.py", line 18
key, *values = line.split(" ")
^
SyntaxError: invalid syntax
The line in the script its referring to is this
def string_to_dict(headers):
headers_dict = {}
for line in headers.split("\n"):
if not line: continue
line = line.strip()
key, *values = line.split(" ")
key = key[:-1]
if not (key and values): continue
headers_dict[key] = " ".join(values)
return headers_dict
Any ideas what could have gone wrong?

This syntax is invalid in python 2. Run your script with python3.

As mentioned in other answers, you need to run this in Python3. The shortest answer is:
Install python3, which may not already be on your machine: $ sudo apt-get update; sudo apt-get install python3
Get in the habit of using a shebang in your script files. Add #!/usr/bin/env python3 to the first line of your script to direct the shell to invoke python3 instead of python2 -- most systems come with /usr/bin/python aliased to python2 installations.
You can find out more about sourcing files in the Bash shell (default shell for Ubuntu 16.04) here.
The longer answer -- if you're wondering why the code doesn't work in Python2 but does work in Python3 -- is that Python3 introduced extended iterable unpacking in PEP 3132, but this feature was not implemented in Python2.