I am trying to use this Python urllib2 Basic Auth Problem bit of code to download a webpage content from an URL which requires authentication. The code I am trying is:
import urllib2, base64
request = urllib2.Request("http://api.foursquare.com/v1/user")
base64string = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
It's showing me:
./xx.py: line 1: import: command not found
./xx.py: line 3: syntax error near unexpected token `('
./xx.py: line 3: `request = urllib2.Request("http://api.foursquare.com/v1/user")'
I am wondering what I am doing wrong? I am using Python 2.7.5. How can I download file contents from an URL which requires authentication?
It's not an issue related to authentication at the first step. Your import is not working. So, try writing this on first line:
#!/usr/bin/python
and for the time being run using
python xx.py
For you here is one explanation:
>>> abc = "Hei Buddy"
>>> print "%s" %abc
Hei Buddy
>>>
>>> print "%s" %xyz
Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
print "%s" %xyz
NameError: name 'xyz' is not defined
At first, I initialized abc variable and it works fine. On the otherhand, xyz doesn't work as it is not initialized!
When you see "import: command not found" on the very first import, it is caused by the parser not using the character encoding matching your py file. Especially when you are not using ASCII encoding in your py file.
The way to get it right is to specify the correct encoding on top of your py file to match your file character encoding.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import os
Are you using a UNIX based OS such as Linux? If so, add a shebang line to the very top of your script:
#!/usr/bin/python
Underneath which you would have the rest of the code (xx.py in your case) that you already have. Then run that same command at the terminal:
$ python xx.py
This should then work fine, as it is now interpreting this as Python code. However when running from the terminal this does not matter as python tells how to interpret it here. What it does allow you to do is execute it outside the terminal, i.e. executing it from a file browser.
If you run a script directly e.g., ./xx.py and your script has no shebang such as #!/usr/bin/env python at the very top then your shell may execute it as a shell script. POSIX says:
If the execl() function fails due to an error equivalent to the
[ENOEXEC] error defined in the System Interfaces volume of
POSIX.1-2008, the shell shall execute a command equivalent to having a
shell invoked with the pathname resulting from the search as its first
operand, with any remaining arguments passed to the new shell, except
that the value of "$0" in the new shell may be set to the command
name. If the executable file is not a text file, the shell may bypass
this command execution. In this case, it shall write an error message,
and shall return an exit status of 126.
Note: you may get ENOEXEC if your text file has no shebang.
Without the shebang, you shell tries to run your Python script as a shell script that leads to the error: import: command not found.
Also, if you run your script as python xx.py then you do not need the shebang. You don't even need it to be executable (+x). Your script is interpreted by python in this case.
On Windows, shebang is not used unless pylauncher is installed. It is included in Python 3.3+.
I've experienced the same problem and now I just found my solution to this issue.
#!/usr/bin/python
import sys
import os
os.system('meld "%s" "%s"' % (sys.argv[2], sys.argv[5]))
This is the code[1] for my case. When I tried this script I received error message like :
import: command not found
I found people talks about the shebang. As you see there is the shebang in my python code above.
I tried these and those trials but didn't find a good solution.
I finally tried to type the shebang my self.
#!/usr/bin/python
and removed the copied one.
And my problem solved!!!
I copied the code from the internet[1].
And I guess there had been some unseeable(?) unseen special characters in the original copied shebang statement.
I use vim, sometimes I experience similar problems.. Especially when I copied some code snippet from the internet this kind of problems happen.. Web pages have some virus special characters!! I doubt. :-)
Journeyer
PS) I copied the code in Windows 7 - host OS - into the Windows clipboard and pasted it into my vim in Ubuntu - guest OS. VM is Oracle Virtual Machine.
[1] http://nathanhoad.net/how-to-meld-for-git-diffs-in-ubuntu-hardy
It's about Shebang
#!usr/bin/python
This will tell which interpreter to wake up to run the code written in file.
Related
I have two python environments with different versions running in parallel. When I execute a python script (test2.py) from one python environment in the other python environment, I get the following error:
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\io.py", line 52, in <module>
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\abc.py", line 147
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
So my setup is this:
Python 3.7
(test.py)
│
│ Python 3.5.6
├───────────────────────────────┐
┆ │
┆ execute test2.py
┆ │
┆ 🗲 Error
How can I fix this?
For dm-script-people: How can I execute a module with a different python version in Digital Micrograph?
Details
I have two python files.
File 1 (test.py):
# execute in Digital Micrograph
import os
import subprocess
command = ['C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe',
os.path.join(os.getcwd(), 'test2.py')]
print(*command)
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print("Subprocess result: '{}', '{}'".format(result.stdout.decode("utf-8"), result.stderr.decode("utf-8")))
and File 2 (test2.py)
# only executable in python 3.5.6
print("Hi")
in the same directory. test.py is executing test2.py with a different python version (python 3.5.6, legacy environment).
My python script (test.py) is running in the python interpreter in a third party program (Digital Micrograph). This program installs a miniconda python enviromnent called GMS_VENV_PYTHON (python version 3.7.x) which can be seen in the above traceback. The legacy miniconda environment is used only for running test2.py (from test.py) in python version 3.5.6.
When I run test.py from the command line (also in the conda GMS_VENV_PYTHON environment), I get the expected output from test2.py in test.py. When I run the exact same file in Digital Micrograph, I get the response
Subprocess result: '', 'Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\io.py", line 52, in <module>
File "C:\ProgramData\Miniconda3\envs\GMS_VENV_PYTHON\lib\abc.py", line 147
print(f"Class: {cls.__module__}.{cls.__qualname__}", file=file)
^
SyntaxError: invalid syntax
'
This tells me the following (I guess):
The test2.py is called since this is the error output of the subprocess call. So the subprocess.run() function seems to work fine
The paths are in the GMS_VENV_PYTHON environment which is wrong in this case. Since this is test2.py, they should be in the legacy paths
There is a SyntaxError because a f-string (Literal String Interpolation) is used which is introduced with python 3.6. So the executing python version is before 3.6. So the legacy python environment is used.
test2.py uses either use io nor abc (I don't know what to conclude here; are those modules loaded by default when executing python?)
So I guess this means, that the standard modules are loaded (I don't know why, probably because they are always loaded) from the wrong destination.
How can I fix this? (See What I've tried > PATH for more details)
What I've tried so far
Encoding
I came across this post "Fatal Python error: Py_Initialize: can't initialize sys standard streams LookupError: unknown encoding: 65001" telling me, that there might be problems with the encoding. I know that Digital Micrograph internally uses ISO 8859-1. I tried to use python -X utf8 and python -X utf8 (test2.py doesn't care about UTF-8, it is ASCII only) as shown below. But neither of them worked
command = ['C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe',
"-X", "utf8=0",
os.path.join(os.getcwd(), 'test2.py')]
PATH
As far as I can tell, I think this is the problem. The answer "https://stackoverflow.com/a/31877629/5934316" of the post "PyCharm: Py_Initialize: can't initialize sys standard streams" suggests to change the PYTHONPATH.
So to specify my question:
Is this the way to go?
How can I set the PYTHONPATH for only the subprocess (while executing python with other libraries in the main thread)?
Is there a better way to have two different python versions at the same time?
Thank you for your help.
Background
I am currently writing a program for handling an electron microscope. I need the "environment" (the graphical interface, the help tools but also hardware access) from Digital Micrograph. So there is no way around using it. And DigitalMicrograph does only support python 3.7.
On the other hand I need an external module which is only available for python 3.5.6. Also there is no way around using this module since it controlls other hardware.
Both rely on python C modules. Since they are compiled already, there is no possibility to check if they work with other versions. Also they are controlling highly sensitive aperatures where one does not want to change code. So in short words: I need two python versions parallel.
I was actually quite close. The problem is, that python imports invalid modules from a wrong location. In my case modules were imported from another python installation due to a wrong path. Modifying the PYTHONPATH according to "https://stackoverflow.com/a/4453495/5934316" works for my example.
import os
my_env = os.environ.copy()
my_env["PYTHONHOME"] = "C:\\ProgramData\\Miniconda3\\envs\\legacy"
my_env["PYTHONPATH"] = "C:\\ProgramData\\Miniconda3\\envs\\legacy;"
my_env["PATH"] = my_env["PATH"].replace("C:\\ProgramData\\Miniconda3\\envs\\GMS_VENV_PYTHON",
"C:\\ProgramData\\Miniconda3\\envs\\legacy")
command = ["C:\\ProgramData\\Miniconda3\\envs\\legacy\\python.exe",
os.path.join(path, "test2.py")]
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=my_env)
For Digital Micrograph users: The python environment is saved in the global tags in "Private:Python:Python Path". So replace:
import DigitalMicrograph as DM
# ...
success, gms_venv = DM.GetPersistentTagGroup().GetTagAsString("Private:Python:Python Path")
if not success:
raise KeyError("Python path is not set.")
my_env["PATH"] = my_env["PATH"].replace(gms_venv, "C:\\ProgramData\\Miniconda3\\envs\\legacy")
I had set "PYTHONPATH" as "D:\ProgramData\Anaconda3" for my python (base) python environment before, but i found when I had changed to another env my python still import basic python package from "D:\ProgramData\Anaconda3",which means it get the wrong basic package with the wrong "System environment variables" config.
so I delete "PYTHONPATH" from my windows "System environment variables", and that will be fixed.
This is a very basic question on how to code in python and run your script from a very beginner.
I'm writing a script using Xcode9.4.1 which is supposed to be for python3.6. I then have an sh script run.sh, in the same folder of the script (say "my_folder") which simply looks like
python my_script.py
The python script looks like
from tick.base import TimeFunction
import numpy as np
import matplotlib.pyplot as plt
v = np.arange(0., 10., 1.)
f_v = v + 1
u = TimeFunction((v, f_v))
plt.plot(v, u.value(v))
print('donne!\n')
But as I try to run my_script.sh from the terminal I get a "ImportError: No module named tick.base" error.
But the tick folder is actually present in "my_computer/anaconda3/lib/python3.6/site-packages" and up to last week I was using Spyder from anaconda navigator and everything was correctly working, so no "import error" occurred.
The question is quite trivial, in some sense it simply is "what's the typical procedure to code and run python script and how modules are supposed to be imported-downloaded when running on a given machine?"
I need it since my script is to be run on another machine through ssh and using my laptop to make some attempts. Up to last year I used to work in C and only need to move some folders with code and .h files.
Thank for help!
EDIT 1:
From the Spyder 3.2.7 setting, where the script was giving non problem, I printed the
import sys
print(sys.path)
The -manually- copied the content to the sys.path variable in my_script.py and rerun 'run.sh' and now getting a new (strange) error:
Traceback (most recent call last):
[...]
File "/Users/my_computer/anaconda3/lib/python3.6/site-packages/tick/array/build/array.py", line 106
def tick_double_array_to_file(_file: 'std::string', array: 'ArrayDouble const &') -> "void":
^
SyntaxError: invalid syntax
First, check the python which you are calling the script with is pointing to the anaconda python and it is of the same version you are expecting it to be. You can do "which python" command in Linux and Mac to which the path which points to python. It if is pointing to some different version or build of python than the one which you are expecting then add the needed path to the system environment PATH variable. In Linux and Mac this can be done by adding the following line in the .bashrc file at the /home/ folder:
export PATH=/your/python/path:$PATH
And then source the .bashrc file.
source .bashrc
If you are on a operating system like cent os ,breaking the default python path can break your yum so be careful before changing it.
I am running a script in PyCharm and under the Project Interpretor I have the path
C:\envs\conda\keras2\python.exe
When I try to run the script via ssh on the server I get a 'no module named' error. I get
/usr/bin/python as the ans to 'which python' on the server itself. Could you tell me which path I must add for the script to run properly?
I am just a beginner in Python. I created a file called as cc.py and saved in the following path :
C:/Python33/cc.py.
I am trying to run this file but nothing is happening.
In the python shell I am typing Python cc.py but I am getting the following error:
SyntaxError: invalid syntax
I tried an alternative :
>>> execfile('cc.py');
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
execfile('cc.py');
NameError: name 'execfile' is not defined
The file contains the following lines of code :
import urllib
htmlfile = urllib.urlopen("http://google.com")
htmltext = htmlfile.read()
print htmltext
How should I run this file? I am totally confused. Can someone help me out?
print htmltext should be print(htmltext). Also, execfile() was removed from Python 3. It seems you are using a Python 2 book but running Python 3. These different versions of Python are incompatible, stick to one. For choosing which version, see this question.
An implemention of execfile():
def execfile(filename, *args, **kwargs):
with open(filename) as fp:
exec(fp.read(), *args, **kwargs)
In python 3, execfile no longer exists. You can open it and execute it manually:
def xfile(afile, globalz=None, localz=None):
with open(afile, "r") as fh:
exec(fh.read(), globalz, localz)
And to execute:
>>> xfile(r'C:\path\to\file\script.py')
Credit to: What is an alternative to execfile in Python 3?
That's how you execute files from the interpreter.
Another way, you can execute it from the command prompt. Just open it and type:
$ cd filepath
$ python file.py
About the script you're running, there's also a confusion. Whatever example you are following, it's a Python 2 example, yet you are using Python 3. Change the request line to this:
htmlfile = urllib.request.urlopen("http://google.com")
Hope this helps!
You wrote:
In the python shell I am typing Python cc.py but I am getting the following error:
SyntaxError: invalid syntax
If you want to run the python script, don't do it from the python shell. The "python" (not "Python") command needs to be run from a command prompt (DOS shell, terminal window, etc).
From a command prompt you should issue the command:
$ python cc.py
For a more complete description of the problem and solution, see Executing Scripts in the windows section of the python user guide, and also How do I run a python program under windows in the frequently asked question section of the python users guide.
import urllib.request
with urllib.request.urlopen("http://www.yourwebsiteurl.com") as url:
htmltext = url.read()
print (htmltext)
I'm facing of a strange issue, and after a couple of hour of research I'm looking for help / explanation about the issue.
It's quite simple, I wrote a cgi server in python and I'm working with some libs including pynetlinux for instance.
When I'm starting the script from terminal with any user, it works fine, no bug, no dependency issue. But when I'm trying to start it using a script in rc.local, the following code produce an error.
import sys, cgi, pynetlinux, logging
it produce the following error :
Traceback (most recent call last):
File "/var/simkiosk/cgi-bin/load_config.py", line 3, in
import cgi, sys, json, pynetlinux, loggin
ImportError: No module named pynetlinux
Other dependencies produce similar issue.I suspect some few things like user who executing the script in rc.local (root normaly) and trying some stuff found on the web without success.
Somebody can help me ?
Thanx in advance.
Regards.
Ollie314
First of all, you need to make sure if the module you want to import is installed properly. You can check if the name of the module exists in pip list
Then, in a python shell, check what the paths are where Python is looking for modules:
import sys
sys.path
In my case, the output is:
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
Finally, append those paths to $PATH variable in /etc/rc.local. Here is an example of my rc.local:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing
export PATH="$PATH:/usr/lib/python3.4:/usr/lib/python3.4/plat-x86_64-linux-gnu:/usr/lib/python3.4/lib-dynload:/usr/local/lib/python3.4/dist-packages:/usr/lib/python3/dist-packages"
# Do stuff
exit 0
The path where your modules are install is probably normally sourced by .bashrc or something similar. .bashrc doesn't get sourced when it's not an interactive shell. /etc/profile is one place that you can put system wide path changes. Depending on what Linux version/distro it may use /etc/profile.d/ in which case /etc/profile runs all the scripts in /etc/profile.d, add a new shell script there with execute permissions and a .sh extention.
Am trying to run the following python program
import re
regex=re.compile("http...imgs.xkcd.com.comics.[\\S]*.[jpg|png]")
f=open('out.txt')
for a in f:
print regex.findall(a)
print '\n'
when I type the code into the interpreter manually, it works as expected
but when i save it as a file and try to run it , it gives errors.
The command i used to run it is
chmod +x
sudo ./pymod.py
ERROR:
./pymod.py: 2: Syntax error: "(" unexpected
if i dont use sudo, the error i get is
./pymod.py: line 2: syntax error near unexpected token `('
./pymod.py: line 2: `regex=re.compile("http...imgs.xkcd.com.comics.[\\S]*.[jpg|png]")'
am using ubuntu 10.04 with everything on default
it takes about 10-15 seconds for the error to appear
Your file should start with shebang. You should include the path to the python interpreter
#!/usr/bin/env python
import re
regex=re.compile("http...imgs.xkcd.com.comics.[\\S]*.[jpg|png]")
Check out : http://en.wikipedia.org/wiki/Shebang_(Unix)
This is probably executing as a bash script instead of in Python. Put
#!/usr/bin/env python
at the beginning of your script.
When you set something as executable, you have to specify what you want it to run it with, or Linux will consider it to be a bash script.
Add this as the first line of the file:
#!/usr/bin/python
Or run it like:
python pymod.py
Cheers!
Either use the "shebang". I.e. put
#! /usr/bin/python
as the first line of your script.
Or teach your ubuntu how to treat python scripts without it
as described here: http://www.daniweb.com/code/snippet241988.html