I am trying to run a routine on two lists, sourced from MySQL queries - one contains CIDR networks and the other contains IP addresses. I am trying to compare entries in the IP list to see whether they are in the networks described by the CIDR list, and act on those that are not.
I am trying to use the netaddr module but this doesn't appear to be implemented on Python3 yet?
I have also tried to use ipaddress but I can't seem to get them to compare correctly. Using ipaddress I have the following code:
networks = (('1.1.6.0/20',), ('2.8.2.0/19',), ('7.2.2.0/19',), ('2.2.0.0/19',))
ips = ((8888, 'customer', b'2.8.4.64', '8888*200'),(8888, 'customer', b'1.1.6.3', '8888*201'), (8888, 'customer', b'122.223.159.3', '8888*202'))
straglers = list()
for ip in ips:
exclude = 0
for network in networks:
subnet = ip_network(network[0])
if ip_address(ip[2]) in subnet:
exclude = 1
if exclude == 0:
straglers.append([ip[3],ip[2],ip[1]]) # extension, customer_ip, company
As things stand the code gives a value error as follows: ValueError: b'82.148.47.64' does not appear to be an IPv4 or IPv6 address
I have tried converting ip[2] to an utf-8 string but this makes no difference.
What you are describing as a list is actually a tuple.
First, when I ran your code I did not receive the error you are getting
ValueError: b'82.148.47.64' does not appear to be an IPv4 or IPv6 address
Instead I received the following
raise ValueError('%s has host bits set' % self)
ValueError: 1.1.6.0/20 has host bits set
Is this the error your are actually receiving? If so, this is how to properly correct it.
Referenced from ipaddress module Defining Networks:
By default, attempting to create a network object with host bits set will result in ValueError being raised. To request that the additional bits instead be coerced to zero, the flag strict=False can be passed to the constructor:
This is because the host bits are set and will need to be coerced to zero, as the documentation states above. Pass the following flag strict=False to the constructor.
For example.
subnet = ip_network(network[0], strict=False)
Also, in your ips contained in your tuple need only to be formatted to a string.
For example.
ips = ((8888, 'customer', '2.8.4.64', '8888*200')
OR the following will be presented to you.
'ValueError: b'2.8.4.64' does not appear to be an IPv4 or IPv6 address'
The full working code.
from ipaddress import ip_network, ip_address
networks = (('1.1.6.0/20',), ('2.8.2.0/19',), ('7.2.2.0/19',), ('2.2.0.0/19',))
ips = ((8888, 'customer', b'2.8.4.64', '8888*200'),(8888, 'customer', b'1.1.6.3', '8888*201'), (8888, 'customer', b'122.223.159.3', '8888*202'))
straglers = list()
for ip in ips:
exclude = 0
for network in networks:
subnet = ip_network(network[0], strict=False)
print(ip_address(ip[2].decode('utf-8')))
print(subnet)
if ip_address(ip[2].decode('utf-8')) in subnet:
exclude = 1
if exclude == 0:
straglers.append([ip[3],ip[2],ip[1]]) # extension, customer_ip, company
print(straglers)
Related
I've decided to try to request an IP using scapy. I am able to send out a discover and receive an offer in the variable ansD. Unfortunately I'm having trouble accessing the field that contains the offered IP address which should be ansD[BOOTP].yiaddr . It tells me that the field does not exist. I have looked around and have seen similar issues but cannot seem to understand why I can access normal packet fields, but fail to do so with BOOTP fields.
receivedIP = 0
conf.checkIPaddr = False
fam,hw = get_if_raw_hwaddr(conf.iface)
dhcp_discover = Ether(dst="ff:ff:ff:ff:ff:ff")/IP(src="0.0.0.0",dst="255.255.255.255")/UDP(sport=68,dport=67)/BOOTP(chaddr=hw)/DHCP(options=[("message-type","discover"),"end"])
ansD,unans = srp(dhcp_discover, multi=True)
if True:
dhcp_request=Ether(dst="ff:ff:ff:ff:ff:ff")/IP(src="0.0.0.0",dst="255.255.255.255")/UDP(sport=68,dport=67)/BOOTP(chaddr=hw,yiaddr=ansD[BOOTP].yiaddr)/DHCP(options=[("message-type","request"),"end"])
ansR, unans = srp(dhcp_request,multi=True)
Object Error 'list' object has no attribute 'yiaddr'
I figured it out not two seconds after posting but hopefully this helps others in the future.
I used srp() instead of srp1(), the former returns multiple packets so I would need to index the specific packet I wanted to look at ansD[0][BOOTP].yiaddr . I have since changed my code to use srp1 instead since this is a DHCP request expecting only one specific "Offer" reply from the DHCP server.
Fixed code below
import sys
from scapy.all import *
receivedIP = 0
conf.checkIPaddr = False
fam,hw = get_if_raw_hwaddr(conf.iface)
dhcp_discover=Ether(dst="ff:ff:ff:ff:ff:ff")/IP(src="0.0.0.0",dst="255.255.255.255")/UDP(sport=68,dport=67)/BOOTP(chaddr=hw)/DHCP(options=[("message-type","discover"),"end"])
ansD = srp1(dhcp_discover, multi=True)
if True:
//Request using the IP the server offered us in ansD[BOOTP].yiaddr
dhcp_request = Ether(dst="ff:ff:ff:ff:ff:ff")/IP(src="0.0.0.0",dst="255.255.255.255")/UDP(sport=68,dport=67)/BOOTP(chaddr=hw,yiaddr=ansD[BOOTP].yiaddr)/DHCP(options=[("message-type","request"),"end"])
ansR, unans = srp(dhcp_request,multi=True)
ansR.summary()
I need to get primary domain name from ip. I have some doubts about how functions like gethostbyaddr and getfqdn work.
In the following example I'm going to reverse ip a random domain and then try to get the domain name back:
import socket
domain = 'heroku.com'
# get ip from domain
ip = socket.gethostbyname(domain)
print('ip =', ip)
# get domain from ip
print(socket.gethostbyaddr(ip))
print(socket.getfqdn(ip))
# OUTPUT
# ip = 50.19.85.154
# ('ec2-50-19-85-154.compute-1.amazonaws.com', ['154.85.19.50.in-addr.arpa'], ['50.19.85.154'])
# ec2-50-19-85-154.compute-1.amazonaws.com
It seems both gethostbyaddr and getfqdn are returning the public DNS of one of the load balanced ec2 on AWS. My question is why they don't return the domain heroku.com which is probably the domain registered on Route53?
Another example with google.com:
import socket
domain = 'google.com'
# get ip from domain
ip = socket.gethostbyname(domain)
print('ip =', ip)
# get domain from ip
print(socket.gethostbyaddr(ip))
print(socket.getfqdn(ip))
# OUTPUT
# ip = 216.58.208.174
# ('mil07s10-in-f14.1e100.net', ['174.208.58.216.in-addr.arpa', 'lhr25s09-in-f14.1e100.net', 'lhr25s09-in-f174.1e100.net'], ['216.58.208.174'])
# mil07s10-in-f14.1e100.net
Here again it seems they are returning the public DNS of some machine on GCP. How can I get the real primary domain name from an ip address (heroku.com and google.com in these examples)?
When we do a DNS lookup of a hostname, in the most of the cases we are returned with the CNAME. We take that CNAME, and further resolve it to get an IP. But multiple CNAME's in the (n-1)th stage can be mapped to the CNAME in the (n)th stage. Therefore getting back the CNAME from the CNAME of the later stages is a not a trivial task.
Another Possible Way
Well, now the discussion is moving away from the DNS, but I hope it helps you. Every router or node in the internet is mapped to a Autonomous System, and there are some organizations or sites which maintain this mapping database. So by having the IP, we can contact one such database to get its Autonomous System Number (ASN) and the organization to which the node belongs to. whois.cymru.com:43 is one such site. You can use simple network client like nc to query its database. Below I attached the screenshot of one such query.
SIOCGIFADDR = 0x8915
def getIpAddr(iface = 'eth0'):
ifreq = struct.pack('16sH14s', iface, socket.AF_INET, '\x00'*14)
try:
res = fcntl.ioctl(sock, SIOCGIFADDR, ifreq)
except:
return None
ip = struct.unpack('16sH2x4s8x', res)[2]
return socket.inet_ntoa(ip)
At each step what are the return values of the functions?
And, what is SIOCGIFADDR?
Also, why [2] has been used following the unpack() function?
SIOCGIFADDR : is stands for get internet interface address means 'eth0'.
This is CPU macro which is written at address 0x8915.
You cant take access of that cpu address so you have to go through pack and unpack function using parameters
"16sH2x4s8x"
IP address which u want from machine it have 4 fields like "192.168.5.20"
so that (4*4) 16 is required likewise search more fields of pack unpack functions.
I'm trying to write some code that can get all the IP addresses associated with a given hostname.
This is what I have so far:
def getips(hostname):
try:
result = socket.getaddrinfo(hostname, None, socket.AF_INET,\
socket.SOCK_DGRAM, socket.IPPROTO_IP, socket.AI_CANONNAME)
list = [x[4][0] for x in result]
return list
except Exception, err:
print "error"
return ""
ips = getips('bbc.co.uk')
print ips
The problem is, sometimes it returns all 4 IPs associated with the specific host in this example, sometimes it returns just one. Is there any way to do this in Python so it consistently returns all the IPs associated with a host?
getaddrinfo() calls the resolver library on your host to lookup IP addresses for any given host. There is no special magic in python that can force it to get a different set of results than what the resolver shows.
For e.g if you run strace on your python script, you will notice that the resolver is invoked:
open("/lib/x86_64-linux-gnu/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3
The user will input either hostname or the IP address. If the user enters the IP address, I want to leave as it is but if the user enters the hostname I want to convert it into IP address using the following method:
def convert(hostname):
command = subprocess.Popen(['host', hostname],
stdout=subprocess.PIPE).communicate()[0]
progress1 = re.findall(r'\d+.', command)
progress1 = ''.join(progress1)
return progress1
How do I do it?
To get ip whether input is ip or hostname:
ip4 = socket.gethostbyname(ip4_or_hostname)
you can use a regex to match your input and test if it is a ip address or not
test = re.compile('\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b')
result = test.match(hostname)
if not result:
# no match -> must be an hostname #
convert(hostname)
that regex allows invalid ip addresses (like 999.999.999.999) so you may want to tweak it a bit, it's just a quick example
There are a number of questions on stackoverflow already about validating an IP address.
IP Address validation in python
Validating IP Addresses in python
I would like to ask why you are communicating with a subprocess when you can do this within the standard python library.
I would recommend resolving a host name into a IP address by using some of pythons built in functionality.
You can do this by importing and using the python sockets library
For example using the code found in link 1:
import socket
import re
regex = re.compile(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$")
result = regex.match(address)
if not result:
address = socket.gethostbyname(address)
In my case, host name can only contain - as a separator. So you can uncomment and use it according to your requirement.
import re
regex = "^((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])$"
# string_check= re.compile('[#_!#$%^&*()<>?/\|}{~:.]')
string_check= re.compile('[-]')
ip_host_detail = {}
def is_valid_hostname_ip(IpHost):
# pass regular expression and ip string into search() method
if (re.search(regex, IpHost)):
print("Valid Ip address")
ip_host_detail['is_ip'] = 'True'
ip_host_detail['is_hostname'] = 'False'
return True
elif(string_check.search(IpHost)):
print("Contain hostname")
ip_host_detail['is_hostname'] = 'True'
ip_host_detail['is_ip'] = 'False'
return True
else:
print("Invalid Ip address or hostname:- " + str(IpHost))
ip_host_detail['is_hostname'] = 'False'
ip_host_detail['is_ip'] = 'False'
return False
IpHost = sys.argv[1]
# IpHost = 'RACDC1-VM123'
is_valid_hostname_ip(IpHost)
print(ip_host_detail)