My Python version is 3.6.0 and my operating system is
Windows.
I want to install jupyter notebook using the order pip install jupyter.
But it failed, I got the following error:
Three Ways to Run Jupyter In Windows
The "Pure Python" Way
Make your way over to python.org, download and install the latest version (3.5.1 as of this writing) and make sure that wherever you install it, the directory containing python.exe is in your system PATH environment variable. I like to install it in the root of my C: drive, e.g. C:\Python35, so my PATH contains that directory.
Once that's installed, you'll want to create a virtual environment, a lightweight, disposable, isolated python installation where you can experiment and install 3rd party libraries without affecting your "main" installation. To do this, open up a Powershell window, and enter the following commands (where "myenv" is the name of the virtualenv we're going to create, you can use any name you like for this):
PS C:\> python -m venv myenv
PS C:\> myenv\Scripts\activate
Then, let's install jupyter and start up a notebook:
PS C:\> pip install jupyter
PS C:\> jupyter notebook
Incidentally, if you get a warning about upgrading pip, make sure to use the following incantation to upgrade (to prevent an issue on windows where pip is unable to upgrade its own executable in-place):
PS C:\> python -m pip install --upgrade pip
Advantages: Uses "pure" python, official tools, and no external dependencies. Well supported, with plenty of online documentation and support communities.
Disadvantages: While many popular data analysis or scientific python libraries can be installed by pip on windows (including Pandas and Matplotlib), some (for example SciPy) require a C compiler and the presence of 3rd party C libraries on the system which are difficult to install on Windows.
Who is it for? Python users comfortable with the command line and the tools that ship with Python itself.
The Python Distributions
Because of the difficulty mentioned above in getting packages like SciPy installed on Windows, a few commercial entities have put together pre-packaged Python "distributions" that contain most, if not all, of the commonly used libraries for data analysis and/or scientific computing.
Anaconda is an excellent option for this. Download their Python 3.5 installer for Windows, run it, and in your Start menu you'll have a bunch of neat new tools, including an entry for Jupyter Notebook. Click to start it up and it'll launch in the background and open up your browser to the notebook console. It doesn't get any easier than that.
Advantages: Simplest, fastest way to get started and it comes with probably everything you need for your scientific computing projects. And anything it doesn't ship with you can still instalAl via its built in conda package manager.
Disadvantages: No virtualenv support, although the conda package manager provides very similar functionality with the conda create command. Relies on a commercial 3rd party for support.
Who is it for? People who want the quickest, easiest way to get Jupyter notebook up and running (IE, most people).
Docker
Docker is a platform for running software in "containers", or self-contained, isolated processes. While it may sound similar in concept to python virtual environments, Docker containers are an entirely different kind of technology offering vast flexibility and power. Don't let the flexibility and power and confusing terminology put you off though -- Docker can be easy to get up and running on your PC and has some advantages of its own with respect to Python and Jupyter.
To get started on Windows, download the Docker Toolbox, which contains the tools you need to get up and running. Run the installer and make sure the checkbox to install Virtualbox is checked if you don't already have Virtualbox or another virtualization platform (like VMWare Workstation) installed.
Once installed, you'll have a "Docker Quickstart Terminal" shortcut in your Start Menu. Double click that shortcut and it will create your first Docker engine for you and set up everything you need automatically. Once you see a prompt in the terminal, you can use the docker run command to run Docker "images", which you can think of as pre-packaged bundles of software that will be automatically downloaded from the Docker Hub when you run them. There are many images on Docker Hub that offer Jupyter, including the official Jupyter Notebook image, and Anaconda itself if you want the full SciPy stack.
To run just the official Jupyter Notebook image in your Docker engine, type the following into the Docker Quickstart Terminal:
$ docker run --rm -it -p 8888:8888 -v "$(pwd):/notebooks" jupyter/notebook
After all the image's "layers" are downloaded, it will start up. Make a note of the IP address listed in the terminal (for example, 192.168.99.100), and point your browser at that IP address, port 8888 (e.g. http://192.168.99.100:8888) and you'll see the familiar Jupyter console, with both Python 2 and Python 3 kernels available.
Advantages: Use the flexibility and power of Docker! Honestly one of my favorite things about Docker is thinking of it as an open software distribution platform for things like the SciPy stack that are hard to install.
Disadvantages: Grapple with the flexibility and power of Docker! There are quite a few "gotchas" to be aware of when dealing with Docker, such as immutable containers, data volumes, arcane commands, and rapidly developing, occasionally buggy tooling.
I met the same problem as yours. I updated my Python to version 3.6.1 and re-install Jupyter notebook. I encounter the first problem UnicodeDecodeError when pip setting up tornado, my system is Windows 8.1 64bit.
Here is what I do to solve it:
I open the directory ...\Python\Python36\Lib\site-packages\tornado where new python installed
open the directory ...\Python\Python35\Lib\site-packages\tornado where old python installed
Then I copy and overwrite all *.py files in new tornado directory form old tornado (Note: my old tornado is the latest 4.4.2)
Then I run instruction pip install jupyter notebook again
then the same problem when pip setting up MarkupSafe, I do same as above. This time instruction pip install jupyter notebook do not throw error anymore. And Jupyter notebook works.
I think if you have the old version modules of Jupyter, you can try as what I do, Or think about download the error module manually and place them into right directory.
Hope this can help you.
Related
I have a Python Project that has a bunch of dependencies (in my VirtualEnv). I need to run my project in my school computer for Demonstration. My School computer does not have python installed, and lets assume also wont have an Internet connection to install it. I have written the program in Windows 10, and the school computer runs Windows 7.
I have looked at these solutions so far, and also here is why I think they may not work.
Copy and pasting my virtual Env - Doesnt work because venv's have their own structures and has my username in its paths which it will look for in the other system.
Using Py2Exe. I have an Exe file, that I can now run on other systems running Windows 10 without them having python or any of my packages. But I am not sure the VC++ dependencies will be present in windows 7. It may also have some other weird issue that I cant risk.
Docker. I am not familiar with Docker, but can do it if this happens to be the only way.
How can I run the python file in that computer?
Also note that I will not have the time to mess around in the other system. Ideally I must plug in my USB and Open the file to run it. If you think there isn't a solution to this, please let me know by telling so.
Thanks!
I see two options.
Use an online IDE and Python Interpreter (assuming you did not have internet for downloading Python, but do have internet in general). I suggest replit.
Use a portable version of Python. Those are available in the official website and are called "Windows embeddable package". You can test downloading it to a usb, and running it in some computer without Python; it should work.
You can use PyInstaller to generate an exe file from your code. It runs without installation.
Or you have a look at the WinPython distribution which is portable and comes with several tools and packages pre-installed.
Note that Windows 7 only supports Python up to version 3.8. 3.9 is only supported on Windows 10 and will silently fail to run without giving you any hint.
Try python as a portable version.
Download the python Windows embeddable package(zip package) and extract it to your flash drive.
https://www.python.org/downloads/windows/
In extracted python folder, press the shift key + right click and select open command window(windows 7) / open powershell window here(windows 10) option.
Type './python' and hit the enter key.
Convert that python file to a .exe file using auto-py-to-exe. This would convert your .py to a .exe file which you can run anywhere.
To use auto-py-to-exe, just execute the following command on terminal pip install auto-py-to-exe.
Now on the terminal write auto-py-to-exe and press enter. Select the python file you wanna execute anywhere and click on Convert .py to .exe and you would get the folder containing the .exe file. Transfer this folder to any computer and just by clicking the .exe file that is there inside the folder, the program would start executing normally no matter if the computer does not have python or pycharm installed.
This is how I cloned a Windows Python project from a source machine to a target machine without internet connection where Python isn't installed.
Thanks to conda-pack tool (https://conda.github.io/conda-pack/).
On the source machine
Install Anaconda (https://www.anaconda.com/products/individual).
Then from Anaconda prompt type the following commands.
conda activate
conda update -c defaults conda
conda install conda-pack
conda create -n <my_env_name> python=<python_version_number>
conda activate <my_env_name>
# if using Python Windows extensions:
conda install pywin32
Now install the packages you need for your Python project using conda or pip (https://www.anaconda.com/blog/using-pip-in-a-conda-environment).
For instance "conda install <package_name>" or "pip install <package_name>".
And finally export everything to a zip file:
# Pack Python environment my_env_name into my_env.zip
conda pack -n <my_env_name> -o my_env.zip
On the target machine
The OS of the source machine must match the OS of the target. This means that environments built on Windows can’t be relocated to Linux.
Unpack my_env.zip and then execute the following commands from Command Prompt.
call Scripts\activate.bat
conda-unpack
# At this point the Python environment is exactly as if you installed it here directly
I use https://colab.research.google.com. It will work on any computer, but some codes can't be submitted there.
I've been coding with R for quite a while but I want to start learning and using python more for its machine learning applications. However, I'm quite confused as to how to properly install packages and set up the whole working environment. Unlike R where I suppose most people just use RStudio and directly install packages with install.packages(), there seems to be a variety of ways this can be done in python, including pip install conda install and there is also the issue of doing it in the command prompt or one of the IDEs. I've downloaded python 3.8.5 and anaconda3 and some of my most burning questions right now are:
When to use which command for installing packages? (and also should I always do it in the command prompt aka cmd on windows instead of inside jupyter notebook)
How to navigate the cmd syntax/coding (for example the python documentation for installing packages has this piece of code: py -m pip install "SomeProject" but I am completely unfamiliar with this syntax and how to use it - so in the long run do I also have to learn what goes on in the command prompt or does most of the operations occur in the IDE and I mostly don't have to touch the cmd?)
How to set up a working directory of sorts (like setwd() in R) such that my .ipynb files can be saved to other directories or even better if I can just directly start my IDE from another file destination?
I've tried looking at some online resources but they mostly deal with coding basics and the python language instead of these technical aspects of the set up, so I would greatly appreciate some advice on how to navigate and set up the python working environment in general. Thanks a lot!
Python uses a different way of installing packages. Python has a thing named venv which stands for Virtual Environment. You install all of your packages in venv. Usually for each new project you make a new venv.
By using Anaconda on windows you install everything within the anaconda environment that you have specified.
python -m pip install "modulename" is a command that will install modulename to your default venv. You will be able to use this module when no other venv is specified. Here is the docs page. And here is a tutorial on how to use venv
By default python uses the same directory you have your code in. e.g. C:/Users/me/home/mypythonfile.py will run in C:/Users/me/home/ and will be able to access files in this directory. However you can use ../ to navigate directories or you can specify an absolute path to file you want to open e.g. with open("C:/system32/somesystemfile.sys") as file
Going over the technical differences of conda and pip:
So Conda is a packaging tool and installer that aims to do more than what pip does; handle library dependencies outside of the Python packages as well as the Python packages themselves. Both have many similar functionalities as well, you can install packages or create virtual environments with both.
It is generally advisable to generally have both conda and pip installed since there are some packages which might not be available with conda but with pip and vice versa.
The commands to install in both the ways is easy enough, but one thing to keep in mind is that
conda stores packages in the anaconda/pkgs directory
pip stores it in directory under /usr/local/bin/ for a Unix-based system, or \Program Files\ for Windows
You can use both pip or conda inside the jupyter notebook, it will work just fine, but it may be possible that you get multiple versions of the same package.
Most of the times, you will use cmd only to install a module used in your code, or to create environments, py -m pip install "SomeProject" here basically means that the module "SomeProject" will be downloaded in base env.
You could think of conda as python with a variety of additional functionalities, such as certain pre-installed packages and tools, such as spyder and jupyter. Hence, you must be precise when you say:
I've downloaded python 3.8.5 and anaconda3
Does it mean you installed python in your computer and then also anaconda?
In general, or at least in my opinion, using anaconda has advantages for development, but typically you'll just use a simple python installation in production (if that applies to you).
Anaconda has it's own package registry/repository . When you call conda install <package>, it will search for the package there and install it if available. You would better search it first, for instance matplotlib.
pip is a package manager for the Python Package Index. pip also ships with anaconda. Hence, in an anaconda environment you may install packages from either sources (either using pip install or conda install). For instance, pandas from PyPI and pandas from conda. There is no guarantee that packages exist in both sources. You must either search it first or simply try it.
In your first steps, I would suggest you to stick to only one dev env (either simple python or anaconda, recommend the second). Because that simplifies the question: "which python and which pip is executed in the cmd line?". That said, those commands should work as expected in any terminal, it be a simple cmd or an embedded one like in PyCharm or VS Code.
You could inspect that by running (on windows and linux at least):
which python, which pip.
Honestly, this is a question/answer that falls outside the scope of SO and for more info you would better check official websites, such as for anaconda or search for python vs anaconda blogs.
I have python installed through windows store and I can install programs using pip, but when I try to run said programs, they fail to execute in powershell.
How can I make sure that the necessary "scripts" folder is in my path? I never faced these problems when installing from executable.
For example, "pip install ntfy" runs successfully in Powershell.
The command "ntfy send test" fails telling me the term is not part of a cmdlet, function, etc. etc.
The 'ntfy' program is located here /mnt/c/Users/vlouvet/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0/LocalCache/local-packages/Python37/Scripts/ntfy.exe
What is the recommended way of editing my path so that programs installed via pip are available across windows store updates of the Python language?
In advance
I highly recommend you not to use python installed from the Windows Store, because you'll face such errors, and even more nasty ones.
The easy solution
Create a virtual environment on a more accessible folder, for example in C:\Users\<user>\python. To do so, do the following:
Using PowerShell, go to your user folder, using cd(Note that usually PowerShell already starts inside your user folder. This is an important setting to have, and if not, you should change your powershell starting point to this folder for the future.);
Now that you're in your user folder, type in the PowerShell mkdir python; cd python;
Now, to create a virtual environment, type python -m venv venv;
(You can verify that your virtual enviroment has been created by listing the folders, with the command ls);
You have created a virtual environment. Now, you need to activate it. To activate, run the following: ./venv/Scripts/activate;
Now, you have fully created and activated a virtual environment for your current PowerShell session. You can now install any packages / programs using pip.
After that, the only thing that you need to do is to add C:\Users\<user>\python\venv\Scripts to your Path, and you're good to go.
Caveats
By adding this folder to your Path, you may be using an outdated python version in the future, since the Scripts folder inside your virtual environment also adds a python executable that will be enabled in the path.
The recommended solution
As I stated before, I do not recommend to have the Microsoft Store version of python installed on your machine. That said, you're probably using it for the conveniences of having the latest Python version installed as soon as they're released. To alleviate this need while also getting rid of your MS Store Python. I recommend you using Chocolatey to install python (and pretty much any other programs for development).
What is Chocolatey?
Chocolatey is a package manager for Windows, pretty much like apt-get for Ubuntu Linux or HomeBrew for MacOS. By using a package manager, you get rid of the hassle of always having to execute the (mostly annoying) install wizards on windows.
To install Chocolatey:
Go to chocolatey.org/install and follow the install instructions;
(Recommended: Take a look on their documentation later to see what Chocolatey is capable of);
With Chocolatey installed, take a test drive and see if it is working properly by running choco -v in PowerShell;
By having Chocolatey installed, you can now run choco install python -y. Let's break down this command:
choco install -> The package installer of chocolatey
python -> the name of the package you want to install
-y -> This tells the installer to skip install verification by saying "Yes to All" scripts that will be executed in order to install a package.
With python installed from chocolatey, you can also see that Python is already added to your path - This means that any python package or executable installed globally will be now available on your machine!
Hope I could help you!
The above answer is good but I managed to get it to work by doing the following.
Find your installation under C:\Users\"your user"\AppData\Local\Packages it will be named something like PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0
Open your windows settings in the start menu
In search type Environment variables. Edit environment variables for your account should pop up. Click it
In the top box find Path, click it
On the right Click new and enter C:\Users\"your user"\AppData\Local\Packages\"python install directory name from 1. here"\LocalCache\local-packages\Python37\Scripts inside the little box under the last item in the list
open a new cmd prompt and type the script you wanted it should work.
On Windows you can find the user base binary directory by running
python -m site --user-site
and replacing site-packages with Scripts.
For example, this could return
C:\Users\Username\AppData\Roaming\Python36\site-packages
so you would need to set your PATH to include
C:\Users\Username\AppData\Roaming\Python36\Scripts
You can set your user PATH permanently in the Control Panel. You may need to log out for the PATH changes to take effect.
I use software at work that has python scripting built in. The only downside is it doesn't carry many of the useful modules out there today. Looking at the forum and support for the software the knowledge base had instructions on how to add new modules. All was well for a few things but when trying to install Pandas I kept getting a failed dependency for Numpy, even though Numpy was in fact in the site-packages for that python version.
I made several other attempts and then came across this line of code.
python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose
I replaced the python with the path to that version of Python and the install went through. But now that Python version is broken. When I open the script editor in the software and try to run ANY code I get an error that says
Cannot initialize the script system: Error in line 1:PyEval_EvalFramesEx returned NULL without setting an error.
Even when I try to run that version of Python in the console the cursor just moves down a line and nothing happens. I can't even type any more.
The older version of the software still works fine, I believe this is due to it using Python 3.4 while this newer version is running Python 3.5. I have even tried to uninstall and reinstall the software for a clean install and I still get the same error, even if I reinstall in a new location. I fear its something system wide specific to that software version of Python 3.5. I have an external Python 3.5 I installed just to check and it runs fine. It's just the software version of Python that is now broken.
You have installed Python projects with compiled components that are loaded into the Python binary. Those components are conflicting with the embedded Python version of your software package (which may not be able to load them).
Because you used the --user switch to install these packages there is still a way out to recover your installation. The packages were all installed into the site.USER_SITE location, which means you can remove them again, manually.
First, you need to run Python with a switch that disables loading code from that location. You can try to do this with either the -s switch (which disables the USER_SITE option) or the -I switch (enabling isolated mode, ignoring environment variables and the USER_SITE option), to verify that disabling these options actually gives you a working Python prompt:
path/to/software/python -s # or -I
When this works, you can run the site module as a script, to print out where the USER_SITE directory is:
path/to/software/python -s -m site # or -I -m site
Look for the USER_SITE: <path> line, it should end in site-packages. Then, move the contents of that directory somewhere else (to keep as a backup). Then try to run Python without the -s / -I switch again.
I was using the Anaconda 3.5 distro in a Windows 10 machine. Due to dependencies in libraries that I want to work with, I had to have the 2.7 version installed as well.
The good news is that the libraries I needed can now work with the 2.7 version smoothly and Visual Studio 2015 automagically detected my new Python environment.
The problem comes when using the command line. Upon issuing the command
conda info --envs
I get
root * C:\Users\norah\Anaconda2
i.e. a single environment (to my understanding and search so far, according to this I should see two envs listed). This means I can't use conda to acquire new packages for my Python3.5 installation, at least not at the command line as I used to since conda only refers to Python2.7. The same goes for the GUI version, Anaconda navigator (I'm not very fond of the GUI version but I tried it out).
There's also no way of launching python3 from the command line since
$python
always fires up python2.7 and issuing $python3 or $python3.5 in the command line doesn't seem to work (nor would adding the path of python3 to the system since the actual executable has the same name as python2 i.e. python.exe)
Is my system taken over by Python2.7? Is anyone here using them both smoothly and if so could you please elaborate on how to do that? Was it a "no no" move to install both versions of Anaconda?
You can make Python 3.5 environment with your Anaconda 2.7:
conda create -n py35 python=3.5
Now, activate it:
activate py35
Finally you can install the desired packages:
conda install numpy
or, the whole anaconda:
conda install anaconda
The advantage of this approach is that you can also create Python 3.4 or 3.6 environments. Furthermore, you can create environments with different combinations and versions of libraries.
Actually, it makes sense to create a new environment for each larger project.
I also despise the virtual environment switch that Anaconda tries to force on us. I prefer to have both executables always instantly available from the command line. I'm pretty sure I had this working on a Windows machine once:
Install Anaconda2 and Anaconda3 to the C:\ drive as "C:\Anaconda2\" and "C:\Anaconda3\" respectively.
Edit your "Path" environment variable (Control Panel -> System and Security -> System -> Advanced system settings -> Environment Variables) and make sure that "C:\Anaconda2;C:\Anaconda2\Scripts;C:\Anaconda2\Library\bin" is in front of "C:\Anaconda3;C:\Anaconda3\Scripts;C:\Anaconda3\Library\bin".
Copy and rename the file "C:\Anaconda3\python.exe" to "C:\Anaconda3\python3.exe".
Copy and rename the file "C:\Anaconda3\Scripts\conda.exe" to "C:\Anaconda3\Scripts\conda3.exe"
Copy and rename any other scripts you might use in "C:\Anaconda3\Scripts\", such as "pip.exe" to "pip3.exe", etc.
Now, when you type "python" or "conda" at the command line you will get the python2 version, and when you type "python3" or "conda3", etc. at the command line you will get the python3 version.
The best way to use both Python 2.7x and Python 3.5x together is Jupyter Notebook.
http://jupyter.org/
You will be able to work on your browser with IPython Notebook style interface which is great for working with scripting languages.
I found some of these videos on YouTube very informative:
1) https://www.youtube.com/watch?v=HW29067qVWk
2) https://www.youtube.com/watch?v=e9cSF3eVQv0
Besides Python 2.7 and 3.5, there are a bunch of other languages that you will be able run with your Jupyter Notebook. The various Kernels are available in this link below:
https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
After installation, as you start your Jupyter Notebook, your browser will open up a new page showing your Home Directory. Among the 4 tabs (Files|Running|Clusters|Conda):
1) The Files tab shows all the files in your Home Directory.
2) Running tab shows all the Notebooks that are running.
3) Clusters tab is provided by IPython parallel.
4) Conda tab is where you need to add the Python version 3.5 (and other languages if needed) to your Jupyter Notebook (If Python 2.7 is default).
If you are interested to try C++ with your Jupyter Notebook, there are a couple of Kernels available.
1) JupyRoot - https://github.com/root-mirror/root/tree/master/bindings/pyroot/JupyROOT
2) Cling - https://github.com/root-mirror/cling