Develop Reference

How can I execute python.exe to open as if I opened it from the anaconda environment named "base"?

The file path: "C:\Users\Disander\anaconda3\python.exe"
When I open python from the path above, python shows the following warning:
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>>
Then when I try to import numpy or pandas, it throws the following error:
Traceback (most recent call last):
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
from . import multiarray
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\multiarray.py", line 14, in <module>
from . import overrides
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\__init__.py", line 54, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "C:\Users\Disander\anaconda3\python.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.1" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
>>>
If I launch python from the anaconda terminal, python works perfectly fine. I show this by importin numpy and pandas. I import sys and print sys.executable to show that anaconda is using the same python executable python.exe.
(base) C:\Users\Disander>python
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.executable)
C:\Users\Disander\anaconda3\python.exe
>>>
>>> import numpy
>>> import pandas
>>>
I also tried to activate the (base) environment after executing python.exe. Below, I show what happened when I tried using "os.system('conda activate base')"
>>> import os
>>> os.system('conda activate base')
'conda' is not recognized as an internal or external command,
operable program or batch file.
1
>>>
I would like to know if there is a way to execute the python.exe file as if I executed it from anaconda using the (base) environment. I need this because I have another application depending on this executable. The same application uses it to import modules such as numpy and pandas but it is not working.

You need first be sure that "Scripts" folder is in your path environment (C:\Anaconda3\Scripts, for my setup).
Then
activate environment_name
in CMD should work. Replace 'environment_name' with 'base' if you want 'base' activated.

Open the Anaconda prompt or conda activate. Then print the path environment to the console via
(base) C:\> path
Add every listed folder below the Anaconda installation folder ..\Anaconda3\ to your user environment PATH variable. This will allow Python to also find numpy's C-Libraries.
\Anaconda3;
\Anaconda3\Library\mingw-w64\bin;
\Anaconda3\Library\usr\bin;
\Anaconda3\Library\bin;
\Anaconda3\Scripts;
\Anaconda3\bin;
Of cause this all ruins conda's well elaborated environment concept. So make sure you know what you're doing.
EDIT AFTER QUESTION EDIT:
If you have another application that depends on Python running in an Anacoda environment - like for example PowerBI Desktop - simply run that other application from the Anaconda prompt too.
EDIT AFTER 2ND QUESTION EDIT
If conda isn't recognized from os you clearly missed to initialize your shells via
conda init --all
Basically this is done during Anaconda installation by default, but some people prefer to skip this step for whatever reason.

Python virtual environment does not default python packages

I have to create virtual environment and move it to some remote server.
For this i created environment but it does not contains default python packages like json and logging.
I am getting error when importing some package:
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'json'
As you can see json is coming from default path not from virtual environment
source venv3/bin/activate
(venv3) [user1#nn1 ~]$ python
Python 3.7.4 (default, Dec 12 2019, 12:03:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> print(json.__file__)
/usr/local/lib/python3.7/json/__init__.py
>>> import influxdb
>>> print(influxdb.__file__)
/home/user1/venv3/lib/python3.7/site-packages/influxdb/__init__.py

Virtualenvs are not movable between servers. Did you move the virtualenv to a new server? If so I think the easiest for you is to recreate it on the remote server.

Virtual environments use absolute paths (retrieved from environment variables, for example); you can find more information in the documentation or PEP 405. So even if you move your venv to another directory, it's most likely not going to work anymore. Venvs are not supposed to be moved around. Instead you have to recreate your venv wherever you copied your project.
Recreating your venv can be greatly simplified if you use libraries like poetry or pipenv, or simply a requirements.txt file.

Virtualenv does have the default package. If it doesn't have that then you would get the error on the import statement itself. Not on the further commands.

Importing a Python module works from command line, but not from PyCharm

My default Python binary is set to the one with the Anaconda distribution of Python. This is found at /home/karnivaurus/anaconda/bin/python, and I have made this the default by adding to my .bashrc file the following: export PATH=/home/karnivaurus/anaconda/bin:$PATH.
I also have a Python package called caffe, which is located at /home/karnivaurus/caffe/distribute/python, and I have added this to the package search path by adding to my .bashrc file the following: export PYTHONPATH=${PYTHONPATH}:/home/karnivaurus/caffe/distribute/python.
Now, I have a simple Python file, called test.py, with the following contents:
import caffe
print "Done."
If I run this by entering python test.py into the terminal, it runs fine, printing out "Done.". The problem I am having is when I run this in the PyCharm IDE. In PyCharm, I have set the interpreter to be /home/karnivaurus/anaconda/bin/python. But when I open test.py in PyCharm, and run the file in the IDE, I get the following error:
ImportError: No module named caffe
So my question is: Why can PyCharm not find the caffe module when it runs the Python script, but it can be found when I run the script from the terminal?

There are a few things that can cause this. To debug, please modify your test.py like so:
# Is it the same python interpreter?
import sys
print(sys.executable)
# Is it the same working directory?
import os
print(os.getcwd())
# Are there any discrepancies in sys.path?
# this is the list python searches, sequentially, for import locations
# some environment variables can fcuk with this list
print(sys.path)
import caffe
print "Done."
Try again in both situations to find the discrepancy in the runtime environment.
edit: there was a discrepancy in sys.path caused by PYTHONPATH environment variable. This was set in the shell via .bashrc file, but not set in PyCharm's runtime environment configuration.

For an additional option, you can use pycharm by terminal. And export the corresponding environment paths beforehand. This works for me. And I think it's better than make some changes in the code. You gonna need run the code by terminal after your debugging.
For example, in terminal type:
$ export LD_LIBRARY_PATH=~/build_master_release/lib:/usr/local/cudnn/v5/lib64:~/anaconda2/lib:$LD_LIBRARY_PATH
$ export PYTHONPATH=~/build_master_release/python:$PYTHONPATH
Then run pycharm by charm (pycharm can be soft linked by charm bash):
$ charm

Well this may be a redundant answer, however I think it's important to explicitly called out what causes this error.
It happened to me many times and I got it fixed by making sure that IDE ( pycharm or vscode or any other) is set to same working directory where the code resided.
for example : I have two files train.py and config.py in mlproject/src directory. I'm trying to run import config in train.py
**When run in /mlproject/ directory, I get error when try to import config **
(ml) dude#vscode101:~/mlproject$ python
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import config
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'config'
>>>
When run in /mlproject/src/` directory, I'm able to successfully import config
(ml) dude#vscode101:~/mlproject/src$ python
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import config
>>>

django.core.management not found but django exists when running manage.py

When running this command in my virtual environment:
sudo ./manage.py migrate wifipromo 0088_auto__add_adgrouptimesets
Traceback (most recent call last):
File "./manage.py", line 2, in <module>
from django.core.management import execute_manager
ImportError: No module named django.core.management
I get the error above. I checked the shebang in manage.py its "#!/usr/bin/env python" which should be correct.
Additionally when I enter the python console, I can run the following....
(ve)ubuntu#app02:/var/apps/nimbus/nimbus$ python
Python 2.6.5 (r265:79063, Feb 27 2014, 19:43:51)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django
<module 'django' from '/var/apps/nimbus/environments/767a6ca970ca5b5cc5724c3e158f8f53/ve/lib/python2.6/site-packages/django/__init__.pyc'>
>>> django.get_version()
'1.4'
which is strange. Anybody know what's wrong?

You're running with sudo (user root), so it can not get info from your virtualenv (user ubuntu). Try to run migrate as normal user.

Turns out the app I sshed into was configured for a specific user. Ubuntu and root aren't configured correctly. This turned out to be a very specific domain problem and I guess its not a good format for this site. Sorry. Thanks for helping everyone.

Local Appengine stopped working

I tried to run one of my AppEnigne projects (python) today but it will no longer launch, this is the stack trace I'm getting.
*** Running dev_appserver with the following flags:
--admin_console_server= --port=8080 --clear_datastore
Python command: /usr/bin/python2.5
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 77, in <module>
run_file(__file__, globals())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 73, in run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_main.py", line 138, in <module>
import logging
ImportError: No module named logging
I thought it could be a python 2.6 error but I adjusted my path to /usr/bin/python2.5 and its still not working. I'm running OSX 10.6.8 and have the latest AppEngineLauncher 1.5.4
The only thing I changed recently that might have affected this is when I updated my XCode to the latest version, v4.2 build 4C199
Has anyone else faced this issue recently?
EDIT
I can't import logging from the terminal either, same message. Here's Python's path.
Chriss-MacBook-Pro:bin chris$ /usr/bin/python2.5
Python 2.5.4 (r254:67916, Aug 2 2010, 20:09:39)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload',
'/Library/Python/2.5/site-packages',
'/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC',
'/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/wx-2.8-mac-unicode']
>>>

Thanks to Nick and Wobble I've figured it out. I recently updated my XCode install to the 4.2 GM release and removed the beta versions. Along the way OSX forgot where gcc was installed and prevented it from compiling the python modules like logging. This resulted in missing .pyo files inside /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25/logging which prevents import logging from working.
Logging wasn't the only module that wasn't compiled, just the first one AppEngine tried to import.
Solution: uninstall and do a clean install of XCode. Make sure gcc can be found on your PATH and everything should be fine.