Installing Scrapy-Python and Easy_install on windows 7 - python

I'm trying to install Scrapy on windows 7. I'm following these instructions:
http://doc.scrapy.org/en/0.24/intro/install.html#intro-install
I’ve downloaded and installed python-2.7.5.msi for windows following this tutorial https://adesquared.wordpress.com/2013/07/07/setting-up-python-and-easy_install-on-windows-7/, and I set up the environment variables as mentioned, but when I try to run python in my command prompt I get this error:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\>python
‘python’ is not recognized as an internal or external command,
operable program or batch file.
C:\> python ez_setup.py install
‘python’ is not recognized as an internal or external command,
operable program or batch file.
C:\>
Could you please help me solve this?

Scrapy doesn't work with Python 3 as mentioned in their FAQ
you should install Python 2.7

ur is python2 syntax you are trying to install an incompatible package meant for python2 not python3:
_ajax_crawlable_re = re.compile(ur'<meta\s+name=["\']fragment["\']\s+content=["\']!["\']/?>')
^^ python2 syntax
Also pip is installed by default for python3.4

Step By Step way to install scrapy on Windows 7
Install Python 2.7 from Python Download link (Be sure to install Python 2.7 only because currently scrapy is not available for Python3 in Windows)
During pyhton install there is checkbox available to add python path to system variable click that option. Otherwise you can add path variable manually.
You need to adjust PATH environment variable to include paths to the Python executable and additional scripts.
The following paths need to be added to PATH C:\Python27\;C:\Python27\Scripts\;
If you have any other problem in adding path variable please refer to this link
3. To update the PATH open a Command prompt in administration mode and run: :\python27\python.exe c:\python27\tools\scripts\win_add2path.py.Close the command prompt window and reopen it so changes take effect, run the following command, to check ever thing added to path variable.
python -–version which will give output as Python 2.7.12 (your version might be different than mine)
pip --version which will give output as pip 9.0.1 (your version might be different than mine)
4. You need to install visual basic C++ Python complier. You can download that from Download link
5. Then you install to install libxml which python library used by scrapy. You download it by writing a command pip install libxml into command prompt.
but if you face some problem in pip installation you can download it from http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml download libxml package according to your system architecture. Open command prompt into that download directory and pip install NAME_OF_PACKAGE.whl
6. Install pywin32 from Download link. Be sure you download the architecture (win32 or amd64) that matches your system
7. Then open command prompt and run this command pip install scrapy
I hope this will help in successful installing scrapy
8. For Reference use can your these links Scrapy official Page and Blog on how to install scrapy on windows

How to install Scrapy 1.4 on Python 3.6 on Windows 8.1 Pro x64
pip install virtualenv
pip install virtualenvwrapper
pip install virtualenvwrapper-win
mkvirtualenv my_scrapy_project
I advice to use virtualenv. In my example I am using name my_scrapy_project for my virtual environment.
If you want to go out of virtualenv, simply type deactivate, if you want to go back into, simply type workon my_scrapy_project.
Go to: http://landinghub.visualstudio.com/visual-cpp-build-tools
Click on button: Download Visual C++ Build Tools 2015
Install these tools.
Go to: https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml
Find and download: lxml-4.1.1-cp36-cp36m-win32.whl
Move this file to your active directory in commandline and install it:
pip install lxml-4.1.1-cp36-cp36m-win32.whl
pip install scrapy
And that is all, it should work.

Related

Python was not found after closing commandline. Windows opens app store when I type python

I have downloaded and installed python from here.
Then I restarted and opened a commandline and typed
python --version
I recorded all my actions, so I can post a screenshot of that:
It works and shows the version number.
Then I performed the following steps to install Jupyter because I want to use a local runtime with Google Colab:
In the Windows commandline (not in the Python window!!), I type this:
pip install virtualenv
python -m pip install --upgrade pip
virtualenv opencv (I guess I can choose a random name here because we will not be using OpenCV, right? But for simplicity, I keep this name now)
I CD to c:\users\MYUSERNAME\opencv\Scripts
activate.bat
pip install numpy
python -m pip install jupyter
Now my browser opens this page: http://localhost:8888/tree
Now I click "New" and "Python 3 (ipykernel)"
Now I want to install the extension jupyter_http_over_ws.
I type:
pip install jupyter_http_over_ws
Nothing shows up when I type text on the keyboard. It looks like this:
Because I want to be able to type again, I close the command line and open it again.
Now I type again:
pip install jupyter_over_http_ws
It says
The command "pip" has been misspelt oder was not found.
Because I have no idea what is happening, I type:
python --version
I am expecting that it shows the version number like before.
But instead it shows
"Python was not found."
Why is that?
ps: When I type "python", it starts the Microsoft App Store and offers me to download it:
Thank you!
What happens if you add your python installation path to the environment variables?
https://www.educative.io/answers/how-to-add-python-to-path-variable-in-windows
Don't forget to restart Windows after adding the path!

Upgrade python to 3.10 in windows; Do I have to reinstall all site-packages manually?

I have in windows 10 64 bit installed python 3.9 with site-packages. I would like to install python 3.10.2 on windows 10 64 bit and find a way to install packages automatically in python 3.10.2, the same ones I currently have installed in python 3.9. I am also interested in the answer to this question for windows 11 64 bit.
I upgraded to python 3.10.2 in windows 10 64 bit. To properly install the packages, install the appropriate version of the Microsoft Visual C++ compiler if necessary. Details can be read https://wiki.python.org/moin/WindowsCompilers . With the upgrade to python 3.10.2 from 3.9, it turned out that I had to do it, due to errors that are appearing during the installation of the packages. Before the installing python 3.10.2, type and execute the following command in the windows command prompt:
pip freeze > reqs.txt
This command writes to the reqs.txt file the names of all installed packages in the version suitable for pip. If you run the command prompt with administrator privileges, the reqs.txt file will be saved in the directory C:\WINDOWS\system32.
Then, after the installing of python 3.10.2 and the adding it to the paths in PATH, with the help of the command prompt you need to issue the command:
pip install -r reqs.txt
This will start the installing of the packages in the same versions as for python 3.9. If problems occur, e.g. an installation error appears during the installation of lxml, then you can remove from the regs.txt file the entry with the name of the package whose installation is causing the problem and then install it manually. To edit the reqs.txt file you need the administrator privileges. The easiest way is to run the command prompt in the administrator mode, type reqs.txt and click Enter to edit it.
I decided later to update the missing packages to the latest version, because I suspected that with python 3.10.2 older versions were not compatible.
This means that when upgrading to python 3.10.2 it is worth asking yourself whether it is better to upgrade for all packages. To do this, you can generate the list of the outdated packages using the command:
pip list –-outdated
After the printing of the list in the command prompt, you can upgrade the outdated packages using the command:
pip install --upgrade <package-name>
This can be automated by the editing of the reqs.txt file and the changing of the mark == to > which will speed up the upgrade. The mark > should only be changed for the outdated packages or you will get an error: "Could not find a version that satisfies the requirement ... ".
Supplement to virtual environments:
When you enter a virtual environment directory (in the windows command prompt):, such as D:\python_projects\data_visualization\env\Scripts, type activate to activate it. Then create the reqs.txt file analogous to the description above. Then, copy the file to a temporary directory. After this delete the virtual environment, e.g. using the windows explorator by the deleting of the contents of the env directory. Then, using the version of python in windows of our choice, create a virtual environment using the env directory (see: https://docs.python.org/3/library/venv.html). Copy the regs.txt file to the newly created D:\python_projects\data_visualization\env\Scripts directory. Install site-packages with the support of the regs.txt file as described above.
No you need not reinstall python packages.
conda create --name project-env python=3.x
You can also look at an alternative method to install python versions Install python version

Installation of TensorFlow on windows 7 - 'pip3' is not recognized as an internal or external command,

When following the Installing TensorFlow for Windows guide https://www.tensorflow.org/install/install_windows, after executing
C:\> pip3 install --upgrade tensorflow
I get the following error:
'pip3' is not recognized as an internal or external command,
It looks like pip3 isn't recognized at all (although PATH to python is set)
Run the following
python -m pip install --upgrade tensorflow
Assuming python is working, TensorFlow should get installed (at least the "Validate the installation" step is green).
This will work, if you are facing pip3 or pip is not recognized as an internal or external command issue on windows:
From the desktop, right click the Computer icon.
Choose Properties from the context menu.
Click the Advanced system settings link.
Click Environment Variables. In the section System Variables, find the PATH environment variable and select it. Click Edit.
A new pop up will open. Variable name will remain Path. We will change the Variable value to the location of the folder where your python scripts folder is located. Find it.
For e.g. I changed its value to C:\Users\rgupta6\AppData\Local\Programs\Python\Python35\Scripts
Close all remaining windows. Reopen Command prompt window, and run your pip3 install --upgrade tensorflow command or pip3 install tensorflow command
That is because you haven't setup the environment variable yet.
Follow the steps by #rajesh
I had the same problem and i found his answer helpful
Right click on This PC > Select Properties
Select Advanced system settings on the left
In the dialog box select Environment Variables
In the system variables section select path and cllck on edit
Select new and enter the path where the python scripts are..
it is mostly in C:\Users[your user name]\AppData\Local\Programs\Python\Python36\Scripts
Then ok.. to all the boxes opened
Close cmd if it is already open and now try installing tensorflow using pip again like this
pip3 install --upgrade tensorflow
Before running pip3 install --upgrade tensorflow
you need check if you are using the correct Python 3.5 installation:
Python 3.5.2 [MSC v.1900 64 bit (AMD64)] on win32
Notice the 64 bit part.
Otherwise, it gives the above error.
You are going to install tensorflow-1.0.1-cp35-cp35m-win_amd64.whl, therefore double check your correct version (mostly this is happening you have both Python 2.7 and 3.5).
when installing python, on the install window, check the box "Active path"(something like that), which builds up a path link. So you can run "pip3 install" at anywhere.
Typing the python command before that should do the trick. In my case (on Windows 8.1 with Python 3.6), I had to type 'py' instead of 'python' as follows:
py -m pip install --upgrade tensorflow
The answer depends on the system you're using.
The issue is your path in the cmd is not that of where your python scripts are placed. In this case you can either navigate to "AppData\Local\Programs\Python\Python36\Scripts" in your terminal and then run the command or you can simply put C:\Users\Your User Name\AppData\Local\Programs\Python\Python36\Scripts to Path variable.
Then re-launch your cmd and type "pip3 install tensorflow" and see the feel the happiness :)
I just experienced the same issue, most likely you downloaded a zipped version of python, then unzipped it, and added it to $PATH just like me, python can work but pip3 cannot, and python -m pip cannot either.
fix solution is to download a executable version of python, then follow common installation steps, pip3 is selected by default, then everything is OK now.

'pip' not recognised command when installing packages

I am trying to install the oauth package from the PYPI list of standard directories by submitting the following code to command prompt:
pip install oauth==1.0.1
This is to install the specific version 1.0.1 of 'oauth'. However I am getting the following error message:
pip is not recognised as an internal or external command
I am using Python 3.3 on Windows 8 and have installed Pip via Command Prompt using the code:
Get-Pip.py
What am I doing wrong?
Your python Scripts directory wasn't added to your Windows path. Typically, this path is C:\Python3x\Scripts where the x is the minor version of Python you have installed (ie. 3 for Python 3.3 or 4 for Python 3.4)
Once that is added to your path, you can open a new command prompt and use pip.
If you don't want (or can't) alter your path variable, you can also add the full path to your command to get it to work
C:\Python3x\Scripts\pip.exe install oauth==1.0.1
Again, the x will follow the above values for the minor version of Python.
Look at the subfolder \Scripts in your Python3 instalation path.
Check if there is the pip.exe file or pip3.exe or pip3.3.exe.
In my instalation there is the pip3.exe file, so I need to write pip3 install oauth==1.0.1

Upgrade Python in Windows [duplicate]

I have a Python 2.7.11 installed on one of my LAB stations. I would like to upgrade Python to at least 3.5.
How should I do that ?
Should I prefer to completely uninstall 2.7.11 and than install the new one ?
Is there a way to update it ? Is an update a good idea ?
Every minor version of Python, that is any 3.x and 2.x version, will install side-by-side with other versions on your computer. Only patch versions will upgrade existing installations.
So if you want to keep your installed Python 2.7 around, then just let it and install a new version using the installer. If you want to get rid of Python 2.7, you can uninstall it before or after installing a newer version—there is no difference to this.
Current Python 3 installations come with the py.exe launcher, which by default is installed into the system directory. This makes it available from the PATH, so you can automatically run it from any shell just by using py instead of python as the command. This avoids you having to put the current Python installation into PATH yourself. That way, you can easily have multiple Python installations side-by-side without them interfering with each other. When running, just use py script.py instead of python script.py to use the launcher. You can also specify a version using for example py -3 or py -3.6 to launch a specific version, otherwise the launcher will use the current default (which will usually be the latest 3.x).
Using the launcher, you can also run Python 2 scripts (which are often syntax incompatible to Python 3), if you decide to keep your Python 2.7 installation. Just use py -2 script.py to launch a script.
As for PyPI packages, every Python installation comes with its own folder where modules are installed into. So if you install a new version and you want to use modules you installed for a previous version, you will have to install them first for the new version. Current versions of the installer also offer you to install pip; it’s enabled by default, so you already have pip for every installation. Unless you explicitly add a Python installation to the PATH, you cannot just use pip though. Luckily, you can also simply use the py.exe launcher for this: py -m pip runs pip. So for example to install Beautiful Soup for Python 3.6, you could run py -3.6 -m pip install beautifulsoup4.
If you are upgrading any 3.x.y to 3.x.z (patch) Python version, just go to Python downloads page get the latest version and start the installation. Since you already have Python installed on your machine installer will prompt you for "Upgrade Now". Click on that button and it will replace the existing version with a new one. You also will have to restart a computer after installation.
If you are upgrading from 3.x to 3.y (minor) then you will be prompted with "Install Now". In this case, you are not upgrading, but you are installing a new version of Python. You can have more than one version installed on your machine. They will be located in different directories. When you have more than one Python version on your machine you will need to use py lanucher to launch a specific version of Python.
For instance:
py -3.7
or
py -3.8
Make sure you have py launcher installed on your machine. It will be installed automatically if you are using default settings of windows installer. You can always check it if you click on "Customize installation" link on the installation window.
If you have several Python versions installed on your machine and you have a project that is using the previous version of Python using virtual environment e.g. (venv) you can upgrade Python just in that venv using:
python -m venv --upgrade "your virtual environment path"
For instance, I have Python 3.7 in my ./venv virtual environment and I would like upgrade venv to Python 3.8, I would do following
python -m venv --upgrade ./venv
Installing/Upgrading Python Using the Chocolatey Windows Package Manager
Let's say you have Python 2.7.16:
C:\Windows\system32>python --version
python2 2.7.16
...and you want to upgrade to the (now current) 3.x.y version. There is a simple way to install a parallel installation of Python 3.x.y using a Windows package management tool.
Now that modern Windows has package management, just like Debian Linux distributions have apt-get, and RedHat has dnf: we can put it to work for us! It's called Chocolatey.
What's Chocolatey?
Chocolatey is a scriptable, command line tool that is based on .NET 4.0 and the nuget package manager baked into Visual Studio.
If you want to learn about Chocolatey and why to use it, which some here reading this might find particularly useful, go to https://chocolatey.org/docs/why
Installing Chocolatey
To get the Chocolatey Package Manager, you follow a process that is described at https://chocolatey.org/docs/installation#installing-chocolatey,
I'll summarize it for you here. There are basically two options: using the cmd prompt, or using the PowerShell prompt.
CMD Prompt Chocolatey Installation
Launch an administrative command prompt. On Windows 10, to do this:
Windows + R
Type cmd
Press Ctrl + ⇧ Shift + ↵ Return
If you don't have administrator rights on the system, go to the Chocolatey website. You may not be completely out of luck and can perform a limited local install, but I won't cover that here.
Copy the string below into your command prompt and type Enter:
#"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
Chocolatey will be downloaded and installed for you as below:
Getting latest version of the Chocolatey package for download.
Getting Chocolatey from https://chocolatey.org/api/v2/package/chocolatey/0.10.11.
Downloading 7-Zip commandline tool prior to extraction.
Extracting C:\Users\blahblahblah\AppData\Local\Temp\chocolatey\chocInstall\chocolatey.zip to C:\Users\blahblahblah\AppData\Local\Temp\chocolatey\chocInstall...
Installing chocolatey on this machine
Creating ChocolateyInstall as an environment variable (targeting 'Machine')
Setting ChocolateyInstall to 'C:\ProgramData\chocolatey'
WARNING: It's very likely you will need to close and reopen your shell
before you can use choco.
Restricting write permissions to Administrators
We are setting up the Chocolatey package repository.
The packages themselves go to 'C:\ProgramData\chocolatey\lib'
(i.e. C:\ProgramData\chocolatey\lib\yourPackageName).
A shim file for the command line goes to 'C:\ProgramData\chocolatey\bin'
and points to an executable in 'C:\ProgramData\chocolatey\lib\yourPackageName'.
Creating Chocolatey folders if they do not already exist.
WARNING: You can safely ignore errors related to missing log files when
upgrading from a version of Chocolatey less than 0.9.9.
'Batch file could not be found' is also safe to ignore.
'The system cannot find the file specified' - also safe.
chocolatey.nupkg file not installed in lib.
Attempting to locate it from bootstrapper.
PATH environment variable does not have C:\ProgramData\chocolatey\bin in it. Adding...
WARNING: Not setting tab completion: Profile file does not exist at 'C:\Users\blahblahblah\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1'.
Chocolatey (choco.exe) is now ready.
You can call choco from anywhere, command line or powershell by typing choco.
Run choco /? for a list of functions.
You may need to shut down and restart powershell and/or consoles
first prior to using choco.
Ensuring chocolatey commands are on the path
Ensuring chocolatey.nupkg is in the lib folder
Either Exit the CMD prompt or type the following command to reload the environment variables:
refreshenv
PowerShell Chocolatey Installation
If you prefer PowerShell to the cmd prompt, you can do this directly from there, however you will have to tell PowerShell to run with a proper script execution policy to get it to work. On Windows 10, the simplest way I have found to do this is to type the following into the Cortana search bar next to the Windows button:
PowerShell.exe
Next, right click on the 'Best Match' choice in the menu that pops up and select 'Run as Administrator'
Now that you're in PowerShell, hopefully running with Administrator privileges, execute the following to install Chocolatey:
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
PowerShell will download Chocolatey for you and launch the installation. It only takes a few moments. It looks exactly like the CMD installation, save perhaps some fancy colored text.
Either Exit PowerShell or type the following command to reload the environment variables:
refreshenv
Upgrading Python
The choco command is the same whether you use PowerShell or the cmd prompt. Launch your favorite using the instructions as above. I'll use the administrator cmd prompt:
C:\WINDOWS\system32>choco upgrade python -y
Essentially, chocolatey will tell you "Hey, Python isn't installed" since you're coming from 2.7.x and it treats the 2.7 version as completely separate. It is only going to give you the most current version, 3.x.y (as of this writing, 3.7.2, but that will change in a few months):
Chocolatey v0.10.11
Upgrading the following packages:
python
By upgrading you accept licenses for the packages.
python is not installed. Installing...
python3 v3.x.y [Approved]
python3 package files upgrade completed. Performing other installation steps.
Installing 64-bit python3...
python3 has been installed.
Installed to: 'C:\Python37'
python3 can be automatically uninstalled.
Environment Vars (like PATH) have changed. Close/reopen your shell to
see the changes (or in powershell/cmd.exe just type `refreshenv`).
The upgrade of python3 was successful.
Software installed as 'exe', install location is likely default.
python v3.x.y [Approved]
python package files upgrade completed. Performing other installation steps.
The upgrade of python was successful.
Software install location not explicitly set, could be in package or
default install location if installer.
Chocolatey upgraded 2/2 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Either exit out of the cmd/Powershell prompt and re-enter it, or use refreshenv then type py --version
C:\Windows\System32>refreshenv
Refreshing environment variables from registry for cmd.exe. Please wait...Finished..
C:\Windows\system32>py --version
Python 3.7.2
Note that the most recent Python install will now take over when you type Python at the command line. You can run either version by using the following commands:
py -2
Python 2.7.16 (v2.7.16:413a49145e, Mar 4 2019, 01:37:19) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
C:\>py -3
Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>exit()
C:\>
From here I suggest you use the Python pip utility to install whatever packages you need. For example, let's say you wanted to install Flask. The commands below first upgrade pip, then install Flask
C:\>py -3 -m pip install --upgrade pip
Collecting pip
Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)
100% |████████████████████████████████| 1.4MB 1.6MB/s
Installing collected packages: pip
Found existing installation: pip 18.1
Uninstalling pip-18.1:
Successfully uninstalled pip-18.1
Successfully installed pip-19.0.3
c:\>py -3 -m pip install Flask
...will do the trick. Happy Pythoning!
A quick and painless way for me was to do the following:
Do a pip freeze > requirements.txt on my affected environments (or whatever method you want for backing up your requirements)
Remove the Old version of Python (in my case it was 3.8)
Remove the associated environments
Install the new version (3.9.5 in my case)
Recreate my environments python -m venv venv or however you wish
Reinstall my plug-ins/apps pip install -r requirements.txt or however you wish
Python 2.x and Python 3.x are different. If you would like to download a newer version of Python 2, you could just download and install the newer version.
If you want to install Python 3, you could install Python 3 separately then change the path for Python 2.x to Python 3.x in Control Panel > All Control Panel Items > System > Advanced System Settings > Environment Variables.
In 2019, you can install using chocolatey. Open your cmd or powershell, type:
choco install python
Nowadays you can just install several Python versions from Microsoft Store. Those are published there by Python Software Foundation.
https://apps.microsoft.com/store/search?hl=en-en&gl=EN&publisher=Python%20Software%20Foundation
Just install python newest version's installer it will automatically detect your python version and will say upgrade python and starts upgrading
I was able to execute PowerShell with the following command and python upgraded with no issue. python -m pip install --upgrade pip please see image
You can use pyenv. It let you easily switch between all versions of python.

Categories

Resources