Develop Reference

How can I execute python.exe to open as if I opened it from the anaconda environment named "base"?

The file path: "C:\Users\Disander\anaconda3\python.exe"
When I open python from the path above, python shows the following warning:
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>>
Then when I try to import numpy or pandas, it throws the following error:
Traceback (most recent call last):
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
from . import multiarray
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\multiarray.py", line 14, in <module>
from . import overrides
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\Users\Disander\anaconda3\lib\site-packages\numpy\core\__init__.py", line 54, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "C:\Users\Disander\anaconda3\python.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.1" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
>>>
If I launch python from the anaconda terminal, python works perfectly fine. I show this by importin numpy and pandas. I import sys and print sys.executable to show that anaconda is using the same python executable python.exe.
(base) C:\Users\Disander>python
Python 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.executable)
C:\Users\Disander\anaconda3\python.exe
>>>
>>> import numpy
>>> import pandas
>>>
I also tried to activate the (base) environment after executing python.exe. Below, I show what happened when I tried using "os.system('conda activate base')"
>>> import os
>>> os.system('conda activate base')
'conda' is not recognized as an internal or external command,
operable program or batch file.
1
>>>
I would like to know if there is a way to execute the python.exe file as if I executed it from anaconda using the (base) environment. I need this because I have another application depending on this executable. The same application uses it to import modules such as numpy and pandas but it is not working.

You need first be sure that "Scripts" folder is in your path environment (C:\Anaconda3\Scripts, for my setup).
Then
activate environment_name
in CMD should work. Replace 'environment_name' with 'base' if you want 'base' activated.

Open the Anaconda prompt or conda activate. Then print the path environment to the console via
(base) C:\> path
Add every listed folder below the Anaconda installation folder ..\Anaconda3\ to your user environment PATH variable. This will allow Python to also find numpy's C-Libraries.
\Anaconda3;
\Anaconda3\Library\mingw-w64\bin;
\Anaconda3\Library\usr\bin;
\Anaconda3\Library\bin;
\Anaconda3\Scripts;
\Anaconda3\bin;
Of cause this all ruins conda's well elaborated environment concept. So make sure you know what you're doing.
EDIT AFTER QUESTION EDIT:
If you have another application that depends on Python running in an Anacoda environment - like for example PowerBI Desktop - simply run that other application from the Anaconda prompt too.
EDIT AFTER 2ND QUESTION EDIT
If conda isn't recognized from os you clearly missed to initialize your shells via
conda init --all
Basically this is done during Anaconda installation by default, but some people prefer to skip this step for whatever reason.

Python: undefined symbol: TLSv1_method when importing ssl

I'm running Python 3.5.6 on a distribution where TLS versions below 1.2 have been compiled out of OpenSSL by passing these options to ./configure: no-ssl no-tls1 no-tls1_1 no-ssl3-method no-tls1-method no-tls1_1-method. The OpenSSL version is 1.1.1d. Python 3 is built from source at distro build time and linked against the version of OpenSSL included in the distro.
Everything builds successfully, but when I try to import the ssl library in Python, I get the following error:
$ python3
Python 3.5.6 (default, Mar 23 2020, 05:11:33)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/ssl.py", line 99, in <module>
import _ssl # if we can't import it, let the error propagate
ImportError: /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-aarch64-linux-gnu.so: undefined symbol: TLSv1_method
I don't understand why this error occurs at runtime. The only reference I can find in the Python 3.5.6 code to TLSv1_method is line 3088 of _ssl.c:
ctx = SSL_CTX_new(TLSv1_method());
Using no-tls1-method does compile out the implementation of TLSv1_method, and that line in the Python code is not guarded by any #ifdef. But I'd expect that to cause a failure at link time for the _ssl.cpython-35m-aarch64-linux-gnu.so module, not at runtime when Python tries to import the module. What's going on here, and is there a way to fix it without patching Python? I cannot upgrade the version of OpenSSL or Python in use.

It seems that my confusion resulted from misunderstanding how _ssl.cpython-35m-aarch64-linux-gnu.so links to OpenSSL. I assumed that it was statically linked, but it's actually dynamically linked to libssl.so. This is why the error occurs at runtime when the shared object is loaded.
It seems, then, that the only way to fix this without updating Python is to patch the call to TLSv1_method to use the generic TLS_method instead. I'll leave this question open for a few days though in case anyone has a better suggestion.
Edit: I filed a Python bug for this issue. It should be fixed in a future release.

Why imp can not find "paste" module?

I meet a strange problem:
I have installed paste successfully. And "import paste" is ok.
But i noticed imp can not find it.
"imp.find_module('paste')" always gives "ImportError: No module named paste".
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
>>> imp.find_module('paste')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named paste
>>> import paste
I have tested this on three different machines (Ubuntu14.04, Windows 10, Windows XP), all have similar problem.

I have duplicated this behavior. It seems a little odd to me. The docs describe some slight differences in find_module()'s behavior, but it doesn't seem to highlight anything matching this discrepancy. I suspect it's a bug or perhaps by design but not well documented.
The paste package in particular uses a pth configuration file. You can disable this behavior with -S to see its effect.
$ python -c 'import paste' && echo found
found
$ python -S -c 'import paste' && echo found
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named paste

The main problem seems that paste module doesnt have an __init__.py file
If you create it manually the problem is solved

How to properly install matplotlib on mac 10.7.5? And why is the command "conda" not found after Anaconda is successfully installed?

I tried for a long time to work with python on this Mac. I tried to remove python completely from my mac. Then I realized it was hard to get rid of this default python. Fortunately, I read in the internet that it's a bad idea to remove that one python that come with Mac before I figured out how to remove it. Then I also installed python, numpy, scipy, matplotlib using *.dmg packages. After all the effort, I still get this error message every time I import pylab. Please help me? Anyone?
$python
Python 2.7.4 (v2.7.4:026ee0057e2d, Apr 6 2013, 11:43:10)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pylab
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/pylab.py", line 1, in <module>
from matplotlib.pylab import *
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/pylab.py", line 221, in <module>
`from matplotlib import mpl # pulls in most modules
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/mpl.py", line 2, in <module>
`from matplotlib import axis
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/axis.py", line 14, in <module>
`import matplotlib.text as mtext
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/text.py", line 31, in <module>
`from matplotlib.backend_bases import RendererBase
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/backend_bases.py", line 48, in <module>
`import matplotlib.textpath as textpath
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/textpath.py", line 9, in <module>
`from matplotlib.mathtext import MathTextParser
File "/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/mathtext.py", line 52, in <module>
`import matplotlib._png as _png
ImportError: dlopen(/Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/_png.so, 2): Library not loaded: /opt/local/lib/libpng14.14.dylib
Referenced from: /Library/Python/2.7/site-packages/matplotlib-1.1.0-py2.7-macosx-10.7-intel.egg/matplotlib/_png.so
Reason: image not found
Besides, I followed the instruction on http://ipython.org/install.html to install Anaconda. After Anaconda is installed I went to the next step
$ conda update conda
and got this
"-bash: conda: command not found"
I searched in the net, but it seems no one else has this problem. Can anyone give me a hint here? Many thanks!
In my directory /usr/bin I have
python python-config python2.5 python2.5-config python2.6 python2.6-config python2.7 python2.7-config
With command
which python
I got /Library/Frameworks/Python.framework/Versions/2.7/bin/python
with
which ipython
I got /Library/Frameworks/Python.framework/Versions/2.7/bin/ipython
For numpy I downloaded the package numpy-1.7.0-py2.7-python.org-macosx10.6.dmg, scipy-0.12.0-py2.7-python.org-macosx10.6.dmg for scipy, and matplotlib-1.2.0-py2.7-python.org-macosx10.6.dmg for matplotbib.I downloaded *10.6.dmg because that's what I found...I can not find something like *10.7.dmg
I don't have .bashrc or .bashrc_profile. But I think .profile will do, and it reads like this
# MacPorts Installer addition on 2012-03-07_at_18:55:26: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.
export TERM="xterm-color"
alias ls="ls -G"
export PS1="[\[\e[33m\]\u#\H \[\e[32m\]\w\[\e[0m\]]\n[\[\e[31m\]\!\[\e[0m\]] > "
# Setting PATH for Python 2.7
# The orginal version is saved in .profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
When I type in
$port
it says -bash: port: command not found
Should that worry me?
Shall I also out comment the line
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
as well?

You appear to have MacPorts. Perhaps you can use MacPorts to install the Python of your choice, and then use the corresponding pip (probably /opt/local/bin/pip to install the necessary Python packages.
If MacPorts is something you're not using anymore, I suggest to use homebrew instead: http://mxcl.github.io/homebrew/ . Pay attention to the message you get after installing Python: it tells you that some Python scripts will be installed in /usr/local/share/python. In your case, this may not be an issue (the packages you listed don't install Python scripts afaik).
Keep in mind that using homebrew and MacPorts together may still mess with Python and its packages.
Finally, you need to make sure you don't keep on using the other python executable. Therefore, in your .profile, comment out the last two lines like this:
#PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
#export PATH
Update
Then, to get rid of the MacPorts reference and ensure your PATH picks up /usr/local/bin, change your .profile further to
# MacPorts Installer addition on 2012-03-07_at_18:55:26: adding an appropriate PATH variable for use with MacPorts.
#export PATH=/opt/local/bin:/opt/local/sbin:$PATH # <- comment out
# Set /usr/local/bin explicitly for Homebrew
export PATH=/usr/local/bin:$PATH
Start in a new Terminal (or tab) to have your settings updated.

Pandas installation on Mac OS X: ImportError (cannot import name hashtable)

I would like to build pandas from source rather than use a package manager because I am interested in contributing. The first time I tried to build pandas, these were the steps I took:
1) created the virtualenv
mkvirtualenv --no-site-packages pandas
2) activated the virtualenv
3) installed Anaconda CE. However, this was installed in ~/anaconda.
4) cloned pandas
5) built C extensions in place
(pandas)ems ~/.virtualenvs/pandas/localrepo/pandas> ~/anaconda/bin/python setup.py build_ext --inplace
6) built pandas
(pandas)ems ~/.virtualenvs/pandas/localrepo/pandas> ~/anaconda/bin/python setup.py build
7) ran nosetests on master branch
Tests failed:
(pandas)ems ~/.virtualenvs/pandas/localrepo/pandas> nosetests pandas
E
======================================================================
ERROR: Failure: ValueError (numpy.dtype has the wrong size, try recompiling)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/EmilyChen/.virtualenvs/pandas/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName
addr.filename, addr.module)
File "/Users/EmilyChen/.virtualenvs/pandas/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/EmilyChen/.virtualenvs/pandas/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/EmilyChen/.virtualenvs/pandas/localrepo/pandas/pandas/init.py", line 6, in
from . import hashtable, tslib, lib
File "numpy.pxd", line 156, in init pandas.hashtable (pandas/hashtable.c:20354)
ValueError: numpy.dtype has the wrong size, try recompiling
Ran 1 test in 0.001s
FAILED (errors=1)
Someone on the PyData mailing list said:
It looks like you have NumPy installed someplace else on your machine and AnacondaCE is not playing nicely in the virtualenv. The error you are getting is a Cython error message which occurs when the NumPy version it built against doesn't match the installed version on your system-- I had thought that 1.7.x was supposed to be ABI compatible with 1.6.x (so this would not happen) but I guess not. Sigh
The numpy version in Anaconda CE library is 1.7.0b2 and my system numpy installation is version 1.5.1. Setup.py linked to the numpy in the Anaconda distribution's libraries when it built pandas but my guess is it's linking to my system version when nosetests runs /pandas/init.py
Next, I repeated the steps outside a virtualenv, but got the same error. Finally, I decided to install all the dependencies in a new virtualenv instead of using the Anaconda distribution to build pandas. This way, I can see that dependencies like numpy reside in the lib directory of the virtualenv python installation, which takes precedent when pandas.init runs import statements. This is what I did:
1) installed numpy, dateutil, pytz, cython, scipy, matplotlib and openpyxl using pip
2) built c extensions in place
3) pandas install output here: http://pastebin.com/3CKf1f9i
4) pandas did not install correctly
(pandas)ems ~/.virtualenvs/pandas/localrepo/pandas> python
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
cannot import name hashtable
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pandas/__init__.py", line 6, in <module>
from . import hashtable, tslib, lib
ImportError: cannot import name hashtable
I took a look at this question but cython installed in my case, and I am trying to build successfully from source rather than using pip like the answer recommended..
(pandas)ems ~/.virtualenvs/pandas/localrepo/pandas> which cython
/Users/EmilyChen/.virtualenvs/pandas/bin/cython

I've received the same error (ImportError: cannot import name hashtable) when trying to import pandas from the source code directory. Try starting the python interpreter from a different directory and import pandas again.