Develop Reference

Extract parameters from variable length python f-string

Using Python 3.6
I have an f-string in main that looks like
title =f'Backup errors in {folder} on {host} at {datetime.datetime.today()}'
used in the following function call send_mail(title, message, to, from)
I am not allowed to change the function call.
The question is, inside send_email can I extract the folder, and host variables from the f-string?
I would normally try something like:
extracted_folder = title.split()[17:30]
but folder and host are both going to be variable length.

you can do a combination of split and slice like this :
title = 'Backup errors in folder 1234 on host1234 at today1234'
folder = title[17:].split(' on ')[-2]
host = title[17:].split(' on ')[-1].split(' at ')[0]
print(folder)
print(host)
output :
folder 1234
host1234
I also work with spaces and with " on " in the folder name if you do an other trick :
title = 'Backup errors in folder on 1234 on host1234 at today1234'
folder = " on ".join(title[17:].split(' on ')[:-1])
host = title[17:].split(' on ')[-1].split(' at ')[0]
print(folder)
print(host)
output :
folder on 1234
host1234

You can use the re module. E.g.:
import re
# regular expression version of text for matching
retext = r'Backup errors in (?P<folder>\S+) on (?P<hostname>\S+) at'
# string containing information
text = 'Backup errors in myfolder on myhost at 17:23:45'
# get dictionary containing the parts of your text
info = re.match(retext, text).groupdict()
print(info)
{'folder': 'myfolder', 'hostname': 'myhost'}

If folder and host are sigle words (no spaces):
>>> title = "Backup errors in /var/mail/filename.log on myserver.domain at ..."
>>> words = title.split(' ')
>>> words[3]
'/var/mail/filename.log'
>>> words[5]
'myserver.domain'

Python getting ips and macs from network

I'm trying to get all IP's and their associated MAC address from the network.
Till now, i have the following code:
eth = Ether(dst = "ff:ff:ff:ff:ff:ff")
arp = ARP(pdst = '198.13.13.1')
answered = srp1(eth / arp)
print answered[1].hwsrc
print answered[1].psrc
But this only gets me the MAC of what Ip i input.
I think i have to use:
answered, unanswered = srp(eth/arp)
And modify pdst with the address of the subnet but i don't know how to do it.
Any ideas?

You just need to enter your network as pdst field. You want to use a timeout because some probes probably won't get an answer. For example:
ans, unans = srp(Ether(dst=ETHER_BROADCAST) / ARP(pdst="198.13.13.0/24"), timeout=1)
Then you need to parse ans. If you want to get the MAC & IP addresses associated, you can create a list of tuples. For example:
res = [(pr.psrc, pr.hwsrc) for _, pr in ans]
print res
You can also use arping(), a specialized function for that purpose:
ans, unans = arping("198.13.13.0/24")
ans.show()

regex to grep string from config file in python

I have config file which contains network configurations something like given below.
LISTEN=192.168.180.1 #the network which listen the traffic
NETMASK=255.255.0.0
DOMAIN =test.com
Need to grep the values from the config. the following is my current code.
import re
with open('config.txt') as f:
data = f.read()
listen = re.findall('LISTEN=(.*)',data)
print listen
the variable listen contains
192.168.180.1 #the network which listen the traffic
but I no need the commented information but sometimes comments may not exist like other "NETMASK"

If you really want to this using regular expressions I would suggest changing it to LISTEN=([^#$]+)
Which should match anything up to the pound sign opening the comment or a newline character.

I come up with solution which will have common regex and replace "#".
import re
data = '''
LISTEN=192.168.180.1 #the network which listen the traffic
NETMASK=255.255.0.0
DOMAIN =test.com
'''
#Common regex to get all values
match = re.findall(r'.*=(.*)#*',data)
print "Total match found"
print match
#Remove # part if any
for index,val in enumerate(match):
if "#" in val:
val = (val.split("#")[0]).strip()
match[index] = val
print "Match after removing #"
print match
Output :
Total match found
['192.168.180.1 #the network which listen the traffic', '255.255.0.0', 'test.com']
Match after removing #
['192.168.180.1', '255.255.0.0', 'test.com']

data = """LISTEN=192.168.180.1 #the network which listen the traffic"""
import re
print(re.search(r'\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}', data).group())
>>>192.168.180.1
print(re.search(r'[0-9]+(?:\.[0-9]+){3}', data).group())
>>>192.168.180.1

In my experience regex is slow runtime and not very readable. I would do:
with open('config.txt') as f:
for line in f:
if not line.startswith("LISTEN="):
continue
rest = line.split("=", 1)[1]
nocomment = rest.split("#", 1)[0]
print nocomment

I think the better approach is to read the whole file as the format it is given in. I wrote a couple of tutorials, e.g. for YAML, CSV, JSON.
It looks as if this is an INI file.
Example Code
Example INI file
INI files need a header. I assume it is network:
[network]
LISTEN=192.168.180.1 #the network which listen the traffic
NETMASK=255.255.0.0
DOMAIN =test.com
Python 2
#!/usr/bin/env python
import ConfigParser
import io
# Load the configuration file
with open("config.ini") as f:
sample_config = f.read()
config = ConfigParser.RawConfigParser(allow_no_value=True)
config.readfp(io.BytesIO(sample_config))
# List all contents
print("List all contents")
for section in config.sections():
print("Section: %s" % section)
for options in config.options(section):
print("x %s:::%s:::%s" % (options,
config.get(section, options),
str(type(options))))
# Print some contents
print("\nPrint some contents")
print(config.get('other', 'use_anonymous')) # Just get the value
Python 3
Look at configparser:
#!/usr/bin/env python
import configparser
# Load the configuration file
config = configparser.RawConfigParser(allow_no_value=True)
with open("config.ini") as f:
config.readfp(f)
# Print some contents
print(config.get('network', 'LISTEN'))
gives:
192.168.180.1 #the network which listen the traffic
Hence you need to parse that value as well, as INI seems not to know #-comments.

How to Read the serial port name from text box (editable)

I am trying to open the serial port in Linux with Python 2.7 PyQt4 with the below code and it works fine:
serialport.port = "/dev/ttyACM1"
serialport.baudrate = 115200
serialport.open()
I don't want to hard-code the serial port name as above. I want to take the serial port name as the input from user from an editable text box:
textbox.setText("/dev/ttyACM1")
serialport.port = textbox.text()
serialport.baudrate = 115200
serialport.open()
But I am unable to convert textbox.text() format to serialport.port.
The following error occurs:
ValueError: "port" must be None or a string, not < class 'PyQt4.QtCore.QString' >

You are using PyQt and have something like this:
w = QWidget()
textbox = QLineEdit(w)
right?!
The error message tells you that the result of textbox.text()is of type QString. But you need a string instead.
You can simply convert the result using str(textbox.text())
serialport.port = str(textbox.text())
should solve the problem.

How to extract a word from text in Python

I have this string "IP 1.2.3.4 is currently trusted in the white list, but it is now using a new trusted certificate." in a log file. What I need to do is look for this message and extract the IP address (1.2.3.4) from the log file.
import os
import shutil
import optparse
import sys
def main():
file = open("messages", "r")
log_data = file.read()
file.close()
search_str = "is currently trusted in the white list, but it is now using a new trusted certificate."
index = log_data.find(search_str)
print index
return
if __name__ == '__main__':
main()
How do I extract the IP address? Your response is appreciated.

Really simple answer:
msg = "IP 1.2.3.4 is currently trusted in the white list, but it is now using a new trusted certificate."
parts = msg.split(' ', 2)
print parts[1]
results in:
1.2.3.4
You could also do REs if you wanted, but for something this simple...

There will be dozens of possible approaches, pros and cons depend on the details of your log file. One example, using the re module:
import re
x = "IP 1.2.3.4 is currently trusted in the white list, but it is now using a new trusted certificate."
pattern = "IP ([0-9\.]+) is currently trusted in the white list"
m = re.match(pattern, x)
for ip in m.groups():
print ip
If you want to print out every instance of that string in your log file, you'd do something like this:
import re
pattern = "(IP [9-0\.]+ is currently trusted in the white list, but it is now using a new trusted certificate.)"
m = re.match(pattern, log_data)
for match in m.groups():
print match

Use regular expressions.
Code like this:
import re
compiled = re.compile(r"""
.*? # Leading junk
(?P<ipaddress>\d+\.\d+\.\d+\.\d+) # IP address
.*? # Trailing junk
""", re.VERBOSE)
str = "IP 1.2.3.4 is currently trusted in the white list, but it is now using a new trusted certificate."
m = compiled.match(str)
print m.group("ipaddress")
And you get this:
>>> import re
>>>
>>> compiled = re.compile(r"""
... .*? # Leading junk
... (?P<ipaddress>\d+\.\d+\.\d+\.\d+) # IP address
... .*? # Trailing junk
... """, re.VERBOSE)
>>> str = "IP 1.2.3.4 is currently trusted in the white list, but it is now using a new trusted certificate."
>>> m = compiled.match(str)
>>> print m.group("ipaddress")
1.2.3.4
Also, I learned there there is a dictionary of matches, groupdict():
>>>> str = "Peer 10.11.6.224 is currently trusted in the white list, but it is now using a new trusted certificate. Consider removing its likely outdated white list entry."
>>>> m = compiled.match(str)
>>>> print m.groupdict()
{'ipaddress': '10.11.6.224'}
Later: fixed that. The initial '.*' was eating your first character match. Changed it to be non-greedy. For consistency (but not necessity), I changed the trailing match, too.

Regular expression is the way to go. But if you fill uncomfortably writing them, you can try a small parser that I wrote (https://github.com/hgrecco/stringparser). It translates a string format to a regular expression. In your case, you will do the following:
from stringparser import Parser
parser = Parser("IP {} is currently trusted in the white list, but it is now using a new trusted certificate.")
ip = parser(text)
If you have a file with multiple lines you can replace the last line by:
with open("log.txt", "r") as fp:
ips = [parser(line) for line in fp]
Good luck.