I am trying to convert the following code which is written in python to use cat | grep instead of opening a file.
The original code:
LOG_NAME="/raft/log/{}{}{}_{}_Rx_flow.log".format(NTIME.tm_year,str(NTIME.tm_mon).zfill(2),str(NTIME.tm_mday).zfill(2),str(NTIME.tm_hour).zfill(2))
print time.strftime("%Y/%m/%d %H:%M:%S") + " Log file name, RX restart, is: " + LOG_NAME
print time.strftime("%Y/%m/%d %H:%M:%S") + " ERRTIMEMIN value: " + ERRTIMEMIN + " RXRESTART message value: " + RXRESTART
LINK_LOG_FILE = open(LOG_NAME, "r")
ISRXRESTART=0
for REPline in LINK_LOG_FILE:
***if RXRESTART in REPline and (ERRTIMEMIN in REPline or ERRTIMEMIN1 in REPline) and ISRXRESTART==0:***
#Link restarted - correct behaviour.
print time.strftime("%Y/%m/%d %H:%M:%S") + " RX restarted - This is correct behaviour"
ISRXRESTART=1
I have to delete the line which opens the file and change the following line with the *** *** to something with cat and grep
for example:
os.popen("sshpass -p ssh root#"+self.ipaddr+" cat "+LOG_NAME+" | egrep `"+device_start+" "+ERRTIMEMIN+`").read().strip()
But I don't know how to combine or & and in the same grep
"OR" can be simulated by simply grepping multiple patterns at once:
egrep -E 'thing1|thing2' <file.txt
-E tells egrep to use extended regex.
To my knowledge there is no "AND" operator in grep but again can be simulated by grepping for patterns in both forward and backward order.
egrep -E 'thing1.*thing2|thing2.*thing1' <file.txt
Use -v for "NOT".
egrep -E 'thing1' <file.txt | egrep -v 'thing2'
This will find everything with "thing1" then grab only the stuff without "thing2".
Hope this helped.
Related
I have to use the below bash command in a python script which includes multiple pip and grep commands.
grep name | cut -d':' -f2 | tr -d '"'| tr -d ','
I tried to do the same using subprocess module but didn't succeed.
Can anyone help me to run the above command in Python3 scripts?
I have to get the below output from a file file.txt.
Tom
Jack
file.txt contains:
"name": "Tom",
"Age": 10
"name": "Jack",
"Age": 15
Actually I want to know how can run the below bash command using Python.
cat file.txt | grep name | cut -d':' -f2 | tr -d '"'| tr -d ','
This works without having to use the subprocess library or any other os cmd related library, only Python.
my_file = open("./file.txt")
line = True
while line:
line = my_file.readline()
line_array = line.split()
try:
if line_array[0] == '"name":':
print(line_array[1].replace('"', '').replace(',', ''))
except IndexError:
pass
my_file.close()
If you not trying to parse a json file or any other structured file for which using a parser would be the best approach, just change your command into:
grep -oP '(?<="name":[[:blank:]]").*(?=",)' file.txt
You do not need any pipe at all.
This will give you the output:
Tom
Jack
Explanations:
-P activate perl regex for lookahead/lookbehind
-o just output the matching string not the whole line
Regex used: (?<="name":[[:blank:]]").*(?=",)
(?<="name":[[:blank:]]") Positive lookbehind: to force the constraint "name": followed by a blank char and then another double quote " the name followed by a double quote " extracted via (?=",) positive lookahead
demo: https://regex101.com/r/JvLCkO/1
I have written a script to get some information from a file.
#!/usr/bin/python
import pxssh
import os
import sys
path = os.getcwd()
conf = sys.argv[1]
print type(path)
print type(conf)
print path
print conf
HOST_IP=os.system("cat %s/%s | grep 'HOST_IP'| cut -d '=' -f2")%(path,conf)
Here is the error I am getting .
`[root#135 bin]# ./Jboss64.py ../conf/samanoj.conf
<type 'str'>
<type 'str'>
/root/Sanity/bin
../conf/samanoj.conf --> This is the file present under conf folder
cat: %s/%s: No such file or directory
Traceback (most recent call last):
File "./Jboss64_EA_EM_FM.py", line 11, in <module>
LIVEQ_HOST_IP=os.system("cat %s/%s | grep 'LIVEQ_HOST_IP'| cut -d '=' -f2")%(path,conf)
TypeError: unsupported operand type(s) for %: 'int' and 'tuple'`
Please help me to resolve this.
You should write like that:
os.system("cat %s/%s | grep 'HOST_IP'| cut -d '=' -f2" % (path,conf))
In your expression, first is executed os.system, only after that executing format string operator.
os.system return 0 that's why you got this error
Would be better if you use format method:
os.system("cat {}/{} | grep 'HOST_IP'| cut -d '=' -f2".format(path, conf))
Also would be better if you use subprocess.Popen instead of os.system
Popen("cat {}/{} | grep 'HOST_IP'| cut -d '=' -f2".format(path, conf), shell=True)
Couple of things that may help:
os.system is being deprecated, I suggest you use subprocess.Popen
Docs here:https://docs.python.org/2/library/subprocess.html
Where you have:
LIVEQ_HOST_IP=os.system("cat %s/%s | grep 'LIVEQ_HOST_IP'| cut -d '=' -f2")%(path,conf)
it may be easier to construct the string in Python, then pass to bash. So something like:
output_str = "cat " +str(path) + "/" + str(conf) + " | grep 'LIVEQ_HOST_IP'| cut -d '=' -f2"
LIVEQ_HOST_IP=subprocess.Popen(output_str)
subprocess (and os) take a string that is used to call system functions, so make sure that the string is correct before you pass to Popen. Hope this helps.
You can try:
cmd = "cat {}/{} | grep 'HOST_IP'| cut -d '=' -f2".format(str(path), str(conf))
HOST_IP = subprocess.Popen(cmd)
I am having an issue with newlines in my command which involves the use of sed.
The scenario is as follows. When I execute the following command from Bash, I get:
cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e s/ph/\\nPH/g | grep -v ^$
PHysical id : 0core id : 0
PHysical id : 0core id : 1
As you can see, the sed command replaced ph with \nPH, such that I get a new line for each 'physical id...'
Now, I am calling this bash command from Python. Here is a small snippet of my code containing all relevant library imports.
import subprocess
cmd = 'cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e s/ph/\\nPH/g | grep -v ^$ '
subprocess.call(cmd, shell=True, universal_newlines=True, stdin=subprocess.PIPE)
The problem is that I get:
nPHysical id : 0core id : 0nPHysical id : 0core id : 1
on one line. It appears that the '\n' is not processed as the letter n is printed before PH.
I need to get the output nicely printed so that I can later add | sort | uniq | wc -l to my command to count the lines.
I would appreciate some help from the Bash-and-Python gurus out there.
Thank you.
Try:
cmd = r'cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e s/ph/\\nPH/g | grep -v ^$ '
The r means is a raw string. You might not be escaping some characters correctly.
Simply enclose the substitution command for sed with commas ("") to ensure it is passed correctly into python subprocess:
import subprocess
cmd = 'cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e "s/ph/\\nPH/g" | grep -v ^$ '
subprocess.call(cmd, shell=True, universal_newlines=True, stdin=subprocess.PIPE)
Output:
PHysical id : 0core id : 0
PHysical id : 0core id : 0
I'm trying to grep a list of file from the "*.nasl" of "Openvas" which contains a certain port's number.
I can make it directly in the terminal with the command :
egrep --only-match '111' /home/gwvm/Openvas/var/lib/openvas/plugins/*.nasl |cut -d ":" -f1
This command return all the name of the nasl file which contains 111.
like :
/home/gwvm/Openvas/var/lib/openvas/plugins/SolarWinds_TFTP.nasl:111
/home/gwvm/Openvas/var/lib/openvas/plugins/trojan_horses.nasl:111
and after the cut :
/home/gwvm/Openvas/var/lib/openvas/plugins/SolarWinds_TFTP.nasl
/home/gwvm/Openvas/var/lib/openvas/plugins/trojan_horses.nasl
But when I'm in python(3.1.3) the output give me an error :
egrep:/home/gwvm/Openvas/var/lib/openvas/plugins/*.nasl: No such file or directory
i was thinking about a problem because of the "*.nasl" but when I'm trying with an existing file, same result.
Here is the part of code :
command = ("egrep --only-match '"+ str(port[0]) +"' "+ openvas_directory["locate"]["nasl"] + '*.nasl' + ' |cut -d ":" -f1 ')
process=sp.Popen(command,shell=True, stdout= sp.PIPE)
or
exec(command)
I was thinking too of a bad construction but wen I'm printing the command it gives me what i want :
egrep --only-match '111' /home/gwvm/Openvas/var/lib/openvas/plugins/*.nasl |cut -d ":" -f1
If there are any idea!
from subprocess import PIPE, Popen
x = Popen('egrep --only-match \'111\' /home/gwvm/Openvas/var/lib/openvas/plugins/*.nasl', stdout=PIPE, stderr=PIPE, shell=True)
y = Popen('cut -d ":" -f1', stdin=x.stdout, stdout=PIPE, stderr=PIPE, shell=True)
for row in y.stdout.readline():
print row
Or just use check_output()
And this is btw how you | in Popen ;)
Guidelines:
When using Popen, if you supply a command as a string, use shell=True.
If you however supply Popen with a list ['ls, '-l'] then use shell=False, that's just how it works.
If you're piping data, execute two different Popen's and use the output from the first command as stdin for the second command, this is equivilant to doing | in Linux.
Question:
How do I use sed with python successfully? I have to run this command on a remote server to get a list of comma delimited hosts. When ran from bash I get what I want which is something like host1, host2, host3
Here is what I have:
process = subprocess.Popen(["ssh $USER#mychefserver knife search node "chef_environment:*" | grep -i "node name" | egrep -i "stuff|ruff" | uniq -u | sort -n | cut -d ":" -f 2 | sed -e 's/^[ \t]*//' | tr '\n' ', '
"], shell=False, stdout=PIPE)
I know I'll have to escape the \n, \t, etc, but I'm having trouble with the rest. Whenever I try to run it from my Python script I get an error for invalid syntax even though I've tried a cornucopia of escapes.
You string quoting is broken as you use " inside a double quoted string. You have to escape the " like \". Further note, that most of the double quotes in command line can be replaced by single quotes '. The following code should work:
process = subprocess.Popen(["ssh $USER#mychefserver knife search node \"chef_environment:*\" | grep -i 'node name' | egrep -i 'stuff|ruff' | uniq -u | sort -n | cut -d':' -f 2 | sed -e 's/^[ \t]*//' | tr '\n' ', '"], shell=False, stdout=subprocess.PIPE)