Finding all the websites which are on a particular hostname - python

I did find this function on stackoverflow which extract hostname, aliaslist, and ipaddrlist,
but how can I make a function which take a hostname and search all the websites associated with it (hosted by it)?
>>> import socket
>>> def get_ips_for_host(host):
try:
ips = socket.gethostbyname_ex(host)
except socket.gaierror:
ips=[]
return ips
>>> ips = get_ips_for_host('www.slowtravelmagazine.com')
>>> print(repr(ips))
('ext-cust.squarespace.com', ['www.slowtravelmagazine.com'],
['198.185.159.144', '198.185.159.145', '198.49.23.144',
'198.49.23.145'])

You can't. At least not with a built in function. You would need a datasource that contains all websites with their associated IP addresses.
There are probably some providers out there that have an API for this.

Related

python get primary domain name from ip

I need to get primary domain name from ip. I have some doubts about how functions like gethostbyaddr and getfqdn work.
In the following example I'm going to reverse ip a random domain and then try to get the domain name back:
import socket
domain = 'heroku.com'
# get ip from domain
ip = socket.gethostbyname(domain)
print('ip =', ip)
# get domain from ip
print(socket.gethostbyaddr(ip))
print(socket.getfqdn(ip))
# OUTPUT
# ip = 50.19.85.154
# ('ec2-50-19-85-154.compute-1.amazonaws.com', ['154.85.19.50.in-addr.arpa'], ['50.19.85.154'])
# ec2-50-19-85-154.compute-1.amazonaws.com
It seems both gethostbyaddr and getfqdn are returning the public DNS of one of the load balanced ec2 on AWS. My question is why they don't return the domain heroku.com which is probably the domain registered on Route53?
Another example with google.com:
import socket
domain = 'google.com'
# get ip from domain
ip = socket.gethostbyname(domain)
print('ip =', ip)
# get domain from ip
print(socket.gethostbyaddr(ip))
print(socket.getfqdn(ip))
# OUTPUT
# ip = 216.58.208.174
# ('mil07s10-in-f14.1e100.net', ['174.208.58.216.in-addr.arpa', 'lhr25s09-in-f14.1e100.net', 'lhr25s09-in-f174.1e100.net'], ['216.58.208.174'])
# mil07s10-in-f14.1e100.net
Here again it seems they are returning the public DNS of some machine on GCP. How can I get the real primary domain name from an ip address (heroku.com and google.com in these examples)?
When we do a DNS lookup of a hostname, in the most of the cases we are returned with the CNAME. We take that CNAME, and further resolve it to get an IP. But multiple CNAME's in the (n-1)th stage can be mapped to the CNAME in the (n)th stage. Therefore getting back the CNAME from the CNAME of the later stages is a not a trivial task.
Another Possible Way
Well, now the discussion is moving away from the DNS, but I hope it helps you. Every router or node in the internet is mapped to a Autonomous System, and there are some organizations or sites which maintain this mapping database. So by having the IP, we can contact one such database to get its Autonomous System Number (ASN) and the organization to which the node belongs to. whois.cymru.com:43 is one such site. You can use simple network client like nc to query its database. Below I attached the screenshot of one such query.

Have good way to python bottle web framework allow range of ip addresses?

I want let some IPs can access to site.
example :
bottle server IP : 192.168.0.1
and I want let 192.168.0.1/29 can access to site,
so 192.168.0.2 can access to site, 192.168.0.11 can't access to site.
my way is create a function to check client IP,
if out of range return status 403.
check IP function like this:
from netaddr import IPSet,IPAddress
def authIP(clientIP=None):
rules = IPSet(['192.168.0.1/29'])
if(IPAddress(clientIP) in rules):
return 'ok.'
else:
abort(403,'access denied.')
but, use this way,I will add this function to every route function to check it.
Like:
#route('/ip')
def tip():
cip = request.environ['REMOTE_ADDR']
return authIP(cip)
Have any other ideas ...?

Using django GeoIP and MaxMind database

I'm trying to setup geoip in Django to identify the source of a connection (to tailor content for different countries) but running into a problem.
First I execute:
from django.contrib.gis import geoip
geo = geoip.GeoIP('path to maxmind db')
Then geo.country('www.google.com') returns the US as you'd expect. Other popular websites also work fine.
However when I try it on my own client IP I get an empty record.
For example: geo.country('127.6.89.129')
returns {'country_name': None, 'country': None}
What am I missing here? Does the maxmind database only cover popular sites so can't be used if I want to identify the source of the connection?
I'm also using the browser locale settings to identify language but unfortunately I need geo-location to tailor some of the content independently of language.
The IP address you used in the example is a local IP address, you cannot use it outside your network, did you try with a real public IP address?
Your ip could be forwarded
def foo(request):
g = GeoIP()
country = g.country(get_client_ip(request))
country_code = country['country_code']
def get_client_ip(request):
x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
if x_forwarded_for:
ip = x_forwarded_for.split(',')[0]
else:
ip = request.META.get('REMOTE_ADDR')
return ip

Get all IP addresses associated with a host

I'm trying to write some code that can get all the IP addresses associated with a given hostname.
This is what I have so far:
def getips(hostname):
try:
result = socket.getaddrinfo(hostname, None, socket.AF_INET,\
socket.SOCK_DGRAM, socket.IPPROTO_IP, socket.AI_CANONNAME)
list = [x[4][0] for x in result]
return list
except Exception, err:
print "error"
return ""
ips = getips('bbc.co.uk')
print ips
The problem is, sometimes it returns all 4 IPs associated with the specific host in this example, sometimes it returns just one. Is there any way to do this in Python so it consistently returns all the IPs associated with a host?
getaddrinfo() calls the resolver library on your host to lookup IP addresses for any given host. There is no special magic in python that can force it to get a different set of results than what the resolver shows.
For e.g if you run strace on your python script, you will notice that the resolver is invoked:
open("/lib/x86_64-linux-gnu/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3

Python: List of addressable IP addresses

What's the most Pythonic way to create a list of the addressable IP addresses given a netaddr IPRange or netaddr IPNetwork.
If I use these, then it includes subnet and broadcast addresses:
hosts = list(IPRange('212.55.64.0','212.55.127.255'))
hosts = IPNetwork('192.168.0.1/24')
So what I need for say IPNetwork(192.168.0.0/27) is a list from 192.168.0.1 to 192.168.0.31 note that 192.168.0.0 and 192.168.0.32 must not be included.
EDIT
Thanks for info on how to do it with IPy. Does anybody know if it can be done with netaddr?
The following is a quick script to do what you want using netaddr
(Python 2.7, linux)
from netaddr import *
def addr(address, prefix):
ip = IPNetwork(address)
ip.prefixlen = int(prefix)
return ip
myip = addr('192.168.0.0', '27')
for usable in myip.iter_hosts():
print '%s' % usable
After doing a bit of research I found this way:
l = list(netaddr.IPNetwork('192.168.0.0/27').iter_hosts())

Categories

Resources