I'm reading from CAT pipe in Linux, using subprocess:
stdout=subprocess.PIPE
so some line has BAD EOL, it's huge file and I want to skip such lines and go for the next one. how I can do this in Python?
PS: I always get:
SyntaxError: EOL while scanning string literal
and seems some socket stopped while writing to that file,because I see really huge spaces in the end of that file. Don't want to fix it, want to skip it
here is my code :
import sys,os
import subprocess
import traceback
import re
import ast
try :
cat = subprocess.Popen(["hadoop", "dfs", "-cat", "PATH TO FILE"], stdout=subprocess.PIPE)
for data in cat.stdout:
data = re.sub(' +',' ',data)
msg= ast.literal_eval(data)
if msg['some_string'] == 'some_string' :
print msg['status']
else :
continue
except :
print traceback.format_exc()
pass
exit()
so the output before the programs exits :
many empty spaces and ...
^
SyntaxError: EOL while scanning string literal
Here, try this:
import sys,os
import subprocess
import traceback
import re
import ast
try :
cat = subprocess.Popen(["hadoop", "dfs", "-cat", "PATH TO FILE"], stdout=subprocess.PIPE)
for data in cat.stdout:
data = re.sub(' +',' ',data)
try:
msg= ast.literal_eval(data)
if msg['some_string'] == 'some_string' :
print msg['status']
else :
continue
except SyntaxError:
continue #skip this line
except :
print traceback.format_exc()
pass
exit()
Hope it helps!
to skip errors you can just code smth like:
try:
your code
except {Your error}:
pass
or
try:
your code
except:
pass
for all errors
you could also use smth like:
import sys
import traceback
try:
{code}
except Exception:
_type, _value, _trace = sys.exc_info()
print "Type:\n\t{0}\nException:\n\t\t{1}\nTraceback:\n\t{2}".format(
_type, _value, traceback.format_tb(_trace))
Related
I have to launch a python sql file.
The file is for mysql.
I tried it like this:
from subprocess import Popen, PIPE
import sys
class ImportSql:
def execImport(self, fileSql):
try:
with open(fileSql, 'r') as fileInput:
proc = Popen(["mysql", "DB_NAME", "-u", "USER", "-pPASSWORD"], stdin=PIPE, stdout=PIPE)
proc.communicate('source ' + fileInput)[0]
except BaseException as ex:
print("ERROR:", ex)
sys.exit()
But I get this error:
ERROR: must be str, not _io.TextIOWrapper
how can I do?
You need to pass the contents of the file, not the file object.
proc.communicate('source ' + fileInput.read())
Also, please don't catch exceptions just to print them and exit. That's what Python does already. Leave out that try/except.
Ok, I moved the mysql instructions inside a bat.
from subprocess import Popen
class ImportSql:
def execImport(self):
p = Popen("import.bat")
p.communicate()
it's ok!
thanks!
I have this code:
from subprocess import Popen
link="abc"
theproc = Popen([sys.executable, "p1.py",link])
I want to send the variable "link" to p1.py,
and p1.py will print it.
something like this. here is p1.py:
print "in p1.py link is "+ link
How can I do that?
I'm assuming python refers to Python 2.x on your system.
Retrieve the command line argument in p1.py using sys.argv:
import sys
if not len(sys.argv) > 1:
print "Expecting link argument."
else:
print "in p1.py link is " + sys.argv[1]
There's a function subprocess.check_output that is easier to use if you only want to call a program and retrieve its output:
from subprocess import check_output
output = check_output(["python", "p1.py", "SOME_URL"])
print "p1.py returned the following output:\n'{}'".format(output)
Example output:
$ python call_p1.py
p1.py returned the following output:
'in p1.py link is SOME_URL
'
You have to parse the command line arguments in your p1.py to get it in a variable:
import sys
try:
link = sys.argv[1]
except IndexError:
print 'argument missing'
sys.exit(1)
I had almost finished on a software i was working on that should have worked but didn't. The software was intended to do the following:
Download a python script from the internet.
Automatically run it with the argument provided for the downloaded
file
But an unusual error occurred in the process of running the downloaded script.
python netscript.py [URL FOR TARGET FILE] [ARGUMENTS FOR TARGET FILE|
Error:
File "temp_dl/431341022.py", line 146
^
IndentationError: expected an indented block
Code:
from bs4 import BeautifulSoup
from subprocess import call
from sys import argv
import urllib2
import random
import os
script, url, ns_dledscript_arg = argv
def ns_runscript(ns_randfn, ns_dledscript_arg):
print "Running downloaded content..."
os.system("python temp_dl/%s.py %s" % (ns_randfn, ns_dledscript_arg))
def ns_getcontent(url, ns_dledscript_arg):
print "Getting content..."
ns_openurl = urllib2.urlopen(url).read()
print "Filtering content..."
#ns_filtered_data = re.sub( r'<[^>]*>', ' ', ns_openurl).strip()
ns_filtered_data = BeautifulSoup(ns_openurl, "html.parser")
ns_randfn = random.randrange(100000000, 999999999)
file = open("temp_dl/%s.py" % (ns_randfn), "w") #Create writable file with random name.
file.write("""%s""" % (ns_filtered_data)) #writes pretty code into the file with the random name.
ns_question_viewdata = raw_input("Do you want to print data on console? 'y' or 'n': ")
ns_runscript(ns_randfn, ns_dledscript_arg)
if ns_question_viewdata == 'y':
print ns_filtered_data
elif ns_question_viewdata == 'n':
exit()
else:
exit()
ns_getcontent(url, ns_dledscript_arg)
In the code below, Whenever exception is caught, I would like to exit from the program & print the exception on shell
#stream.py
import hashlib
import sys
import os
import importlib
for line in sys.stdin.readlines():
try:
inpFile = "temp.py"
execfile(inpFile)
line1 = line.strip('\n').split('\t')
print "\t".join(line1)
except:
#exception expected "temp.py:File Not Found", how do I exit the code & print the exception on console ?
sys.exit(1)
Here is the Transform query to call the UDF:
Create table newtable as Select TRANSFORM(id,name) USING
'python stream.py' as (id,name) from mytable;
ideas appreciated.
if you want to catch a specific type of exception (for examle an IOError) you can use
except IOError as e:
and access the error string with e.strerror
if you want to catch all exception you can use sys.exc_info()[0] to access the last error message
for example
try:
1/0
except:
print sys.exc_info()[0]
would print
<type 'exceptions.ZeroDivisionError'>
If you are expecting a specific exception then you should trap that, not everything, which is what you are doing. But there is nothing in your code to generate a "File Not Found" anyway!
EDIT: question code changed! We really do wish to trap "everything". I am using Exception, which traps "almost" everything, see https://docs.python.org/2/library/exceptions.html
This uses Python 2 syntax:
import sys
for line in sys.stdin.readlines():
try:
inpFile = "temp.py"
execfile(inpFile)
line1 = line.strip('\n').split('\t')
print "\t".join(line1)
except Exception as err:
print >> sys.stderr, err # print to stderr (python 2 syntax)
sys.exit(1)
I wrote a python script like this:
#!/usr/bin/python
import sys
import requests
if len(sys.argv) != 2:
print "Usage: python %s <IP-LIST>" % (sys.argv[0])
sys.exit();
InputFile = sys.argv[1]
try:
File = open(InputFile)
for ip in File:
IP = ip.rstrip()
out = requests.get("http://" + IP, timeout=5)
out.status_code
except (KeyboardInterrupt, SystemExit):
print "\nKeyboardInterruption with Ctrl+c signal"
except IOError as e:
print "The file \"%s\" does not exist!" % (sys.argv[1])
When url has nothing to respond the following output appears for me:
The file "list.txt" does not exist!
Why?
requests uses exceptions that subclass IOError, which you are catching and assuming to be file not found. If you were using Python 3 you could catch the more specific FileNotFoundError. Here you should put an except requests.RequestException block above your except IOError block (or break it out into specific requests errors if you want.