Related
How do I ping a website or IP address with Python?
See this pure Python ping by Matthew Dixon Cowles and Jens Diemer. Also, remember that Python requires root to spawn ICMP (i.e. ping) sockets in linux.
import ping, socket
try:
ping.verbose_ping('www.google.com', count=3)
delay = ping.Ping('www.wikipedia.org', timeout=2000).do()
except socket.error, e:
print "Ping Error:", e
The source code itself is easy to read, see the implementations of verbose_ping and of Ping.do for inspiration.
Depending on what you want to achive, you are probably easiest calling the system ping command..
Using the subprocess module is the best way of doing this, although you have to remember the ping command is different on different operating systems!
import subprocess
host = "www.google.com"
ping = subprocess.Popen(
["ping", "-c", "4", host],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
out, error = ping.communicate()
print out
You don't need to worry about shell-escape characters. For example..
host = "google.com; `echo test`
..will not execute the echo command.
Now, to actually get the ping results, you could parse the out variable. Example output:
round-trip min/avg/max/stddev = 248.139/249.474/250.530/0.896 ms
Example regex:
import re
matcher = re.compile("round-trip min/avg/max/stddev = (\d+.\d+)/(\d+.\d+)/(\d+.\d+)/(\d+.\d+)")
print matcher.search(out).groups()
# ('248.139', '249.474', '250.530', '0.896')
Again, remember the output will vary depending on operating system (and even the version of ping). This isn't ideal, but it will work fine in many situations (where you know the machines the script will be running on)
You may find Noah Gift's presentation Creating Agile Commandline Tools With Python. In it he combines subprocess, Queue and threading to develop solution that is capable of pinging hosts concurrently and speeding up the process. Below is a basic version before he adds command line parsing and some other features. The code to this version and others can be found here
#!/usr/bin/env python2.5
from threading import Thread
import subprocess
from Queue import Queue
num_threads = 4
queue = Queue()
ips = ["10.0.1.1", "10.0.1.3", "10.0.1.11", "10.0.1.51"]
#wraps system ping command
def pinger(i, q):
"""Pings subnet"""
while True:
ip = q.get()
print "Thread %s: Pinging %s" % (i, ip)
ret = subprocess.call("ping -c 1 %s" % ip,
shell=True,
stdout=open('/dev/null', 'w'),
stderr=subprocess.STDOUT)
if ret == 0:
print "%s: is alive" % ip
else:
print "%s: did not respond" % ip
q.task_done()
#Spawn thread pool
for i in range(num_threads):
worker = Thread(target=pinger, args=(i, queue))
worker.setDaemon(True)
worker.start()
#Place work in queue
for ip in ips:
queue.put(ip)
#Wait until worker threads are done to exit
queue.join()
He is also author of: Python for Unix and Linux System Administration
http://ecx.images-amazon.com/images/I/515qmR%2B4sjL._SL500_AA240_.jpg
It's hard to say what your question is, but there are some alternatives.
If you mean to literally execute a request using the ICMP ping protocol, you can get an ICMP library and execute the ping request directly. Google "Python ICMP" to find things like this icmplib. You might want to look at scapy, also.
This will be much faster than using os.system("ping " + ip ).
If you mean to generically "ping" a box to see if it's up, you can use the echo protocol on port 7.
For echo, you use the socket library to open the IP address and port 7. You write something on that port, send a carriage return ("\r\n") and then read the reply.
If you mean to "ping" a web site to see if the site is running, you have to use the http protocol on port 80.
For or properly checking a web server, you use urllib2 to open a specific URL. (/index.html is always popular) and read the response.
There are still more potential meaning of "ping" including "traceroute" and "finger".
I did something similar this way, as an inspiration:
import urllib
import threading
import time
def pinger_urllib(host):
"""
helper function timing the retrival of index.html
TODO: should there be a 1MB bogus file?
"""
t1 = time.time()
urllib.urlopen(host + '/index.html').read()
return (time.time() - t1) * 1000.0
def task(m):
"""
the actual task
"""
delay = float(pinger_urllib(m))
print '%-30s %5.0f [ms]' % (m, delay)
# parallelization
tasks = []
URLs = ['google.com', 'wikipedia.org']
for m in URLs:
t = threading.Thread(target=task, args=(m,))
t.start()
tasks.append(t)
# synchronization point
for t in tasks:
t.join()
Here's a short snippet using subprocess. The check_call method either returns 0 for success, or raises an exception. This way, I don't have to parse the output of ping. I'm using shlex to split the command line arguments.
import subprocess
import shlex
command_line = "ping -c 1 www.google.comsldjkflksj"
args = shlex.split(command_line)
try:
subprocess.check_call(args,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
print "Website is there."
except subprocess.CalledProcessError:
print "Couldn't get a ping."
Most simple answer is:
import os
os.system("ping google.com")
I develop a library that I think could help you. It is called icmplib (unrelated to any other code of the same name that can be found on the Internet) and is a pure implementation of the ICMP protocol in Python.
It is completely object oriented and has simple functions such as the classic ping, multiping and traceroute, as well as low level classes and sockets for those who want to develop applications based on the ICMP protocol.
Here are some other highlights:
Can be run without root privileges.
You can customize many parameters such as the payload of ICMP packets and the traffic class (QoS).
Cross-platform: tested on Linux, macOS and Windows.
Fast and requires few CPU / RAM resources unlike calls made with subprocess.
Lightweight and does not rely on any additional dependencies.
To install it (Python 3.6+ required):
pip3 install icmplib
Here is a simple example of the ping function:
host = ping('1.1.1.1', count=4, interval=1, timeout=2, privileged=True)
if host.is_alive:
print(f'{host.address} is alive! avg_rtt={host.avg_rtt} ms')
else:
print(f'{host.address} is dead')
Set the "privileged" parameter to False if you want to use the library without root privileges.
You can find the complete documentation on the project page:
https://github.com/ValentinBELYN/icmplib
Hope you will find this library useful.
read a file name, the file contain the one url per line, like this:
http://www.poolsaboveground.com/apache/hadoop/core/
http://mirrors.sonic.net/apache/hadoop/core/
use command:
python url.py urls.txt
get the result:
Round Trip Time: 253 ms - mirrors.sonic.net
Round Trip Time: 245 ms - www.globalish.com
Round Trip Time: 327 ms - www.poolsaboveground.com
source code(url.py):
import re
import sys
import urlparse
from subprocess import Popen, PIPE
from threading import Thread
class Pinger(object):
def __init__(self, hosts):
for host in hosts:
hostname = urlparse.urlparse(host).hostname
if hostname:
pa = PingAgent(hostname)
pa.start()
else:
continue
class PingAgent(Thread):
def __init__(self, host):
Thread.__init__(self)
self.host = host
def run(self):
p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
m = re.search('Average = (.*)ms', p.stdout.read())
if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
else: print 'Error: Invalid Response -', self.host
if __name__ == '__main__':
with open(sys.argv[1]) as f:
content = f.readlines()
Pinger(content)
import subprocess as s
ip=raw_input("Enter the IP/Domain name:")
if(s.call(["ping",ip])==0):
print "your IP is alive"
else:
print "Check ur IP"
If you want something actually in Python, that you can play with, have a look at Scapy:
from scapy.all import *
request = IP(dst="www.google.com")/ICMP()
answer = sr1(request)
That's in my opinion much better (and fully cross-platform), than some funky subprocess calls. Also you can have as much information about the answer (sequence ID.....) as you want, as you have the packet itself.
using system ping command to ping a list of hosts:
import re
from subprocess import Popen, PIPE
from threading import Thread
class Pinger(object):
def __init__(self, hosts):
for host in hosts:
pa = PingAgent(host)
pa.start()
class PingAgent(Thread):
def __init__(self, host):
Thread.__init__(self)
self.host = host
def run(self):
p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
m = re.search('Average = (.*)ms', p.stdout.read())
if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
else: print 'Error: Invalid Response -', self.host
if __name__ == '__main__':
hosts = [
'www.pylot.org',
'www.goldb.org',
'www.google.com',
'www.yahoo.com',
'www.techcrunch.com',
'www.this_one_wont_work.com'
]
Pinger(hosts)
You can find an updated version of the mentioned script that works on both Windows and Linux here
using subprocess ping command to ping decode it because the response is binary:
import subprocess
ping_response = subprocess.Popen(["ping", "-a", "google.com"], stdout=subprocess.PIPE).stdout.read()
result = ping_response.decode('utf-8')
print(result)
you might try socket to get ip of the site and use scrapy to excute icmp ping to the ip.
import gevent
from gevent import monkey
# monkey.patch_all() should be executed before any library that will
# standard library
monkey.patch_all()
import socket
from scapy.all import IP, ICMP, sr1
def ping_site(fqdn):
ip = socket.gethostbyaddr(fqdn)[-1][0]
print(fqdn, ip, '\n')
icmp = IP(dst=ip)/ICMP()
resp = sr1(icmp, timeout=10)
if resp:
return (fqdn, False)
else:
return (fqdn, True)
sites = ['www.google.com', 'www.baidu.com', 'www.bing.com']
jobs = [gevent.spawn(ping_site, fqdn) for fqdn in sites]
gevent.joinall(jobs)
print([job.value for job in jobs])
On python 3 you can use ping3.
from ping3 import ping, verbose_ping
ip-host = '8.8.8.8'
if not ping(ip-host):
raise ValueError('{} is not available.'.format(ip-host))
If you only want to check whether a machine on an IP is active or not, you can just use python sockets.
import socket
s = socket.socket()
try:
s.connect(("192.168.1.123", 1234)) # You can use any port number here
except Exception as e:
print(e.errno, e)
Now, according to the error message displayed (or the error number), you can determine whether the machine is active or not.
Use this it's tested on python 2.7 and works fine it returns ping time in milliseconds if success and return False on fail.
import platform,subproccess,re
def Ping(hostname,timeout):
if platform.system() == "Windows":
command="ping "+hostname+" -n 1 -w "+str(timeout*1000)
else:
command="ping -i "+str(timeout)+" -c 1 " + hostname
proccess = subprocess.Popen(command, stdout=subprocess.PIPE)
matches=re.match('.*time=([0-9]+)ms.*', proccess.stdout.read(),re.DOTALL)
if matches:
return matches.group(1)
else:
return False
I would like timeout the function sftp.put(), I have tried with signal Module but the script doesn't die if the upload time is over 10s.
I use that to transfer files by ssh (paramiko).
[...]
def handler(signum, frame):
print 'Signal handler called with signal', signum
raise IOError("Couldn't upload the fileeeeeeeeeeee!!!!")
[...]
raspi = paramiko.SSHClient()
raspi.set_missing_host_key_policy(paramiko.AutoAddPolicy())
raspi.connect(ip , username= "", password= "" , timeout=10)
sftp = raspi.open_sftp()
[...]
signal.signal(signal.SIGALRM, handler)
signal.alarm(10)
sftp.put(source, destination , callback=None, confirm=True)
signal.alarm(0)
raspi.close()
[...]
Update 1:
I want to abort the transfer if the server stops responding for a while. Actually, my python script check (in loop) any files in a folder, and send it to this remote server. But in the problem here I want to leave this function in the case of the server become inaccessible suddenly during a transfer (ip server changing, no internet anymore,...). But when I simulate a disconnection, the script stays stuck at this function sftp.put anyway...)
Update 2:
When the server goes offline during a transfer, put() seems to be blocked forever. This happens with this line too:
sftp.get_channel().settimeout(xx)
How to do when we lose the Channel?
Update 3 & script goal
Ubuntu 18.04
and paramiko version 2.6.0
Hello,
To follow your remarks and questions, I have to give more details about my very Ugly script, sorry about that :)
Actually, I don’t want to have to kill a thread manually and open a new one. For my application I want that the script run totally in autonomous, and if something wrong during the process, it can still go on. For that I use the Python exception handling. Everything does what I want except when the remote server going off during a transfer: The script stays blocked in the put() function, I think inside a loop.
Below, the script contains in total 3 functions to timeout this thanks to your help, but apparently nothing can leave this damned sftp.put()! Do you have some new idea ?
Import […]
[...]
def handler(signum, frame):
print 'Signal handler called with signal', signum
raise IOError("Couldn't upload the fileeeeeeeeeeee!!!!")
def check_time(size, file_size):
global start_time
if (time.time() - start_time) > 10:
raise Exception
i = 0
while i == 0:
try:
time.sleep(1) # CPU break
print ("go!")
#collect ip server
fichierIplist = open("/home/robert/Documents/iplist.txt", "r")
file_lines = fichierIplist.readlines()
fichierIplist.close()
last_line = file_lines [len (file_lines)-1]
lastKnowip = last_line
data = glob.glob("/home/robert/Documents/data/*")
items = len(data)
if items != 0:
time.sleep(60) #anyway
print("some Files!:)")
raspi = paramiko.SSHClient()
raspi.set_missing_host_key_policy(paramiko.AutoAddPolicy())
raspi.connect(lastKnowip, username= "", password= "" , timeout=10)
for source in data: #Upload file by file
filename = os.path.basename(source) #
destination = '/home/pi/Documents/pest/'+ filename #p
sftp = raspi.open_sftp()
signal.signal(signal.SIGALRM, handler)
signal.alarm(10)
sftp.get_channel().settimeout(10)
start_time = time.time()
sftp.put(source, destination, callback=check_time)
sftp.close()
signal.alarm(0)
raspi.close()
else:
print("noFile!")
except:
pass
If you want to timeout, when the server stops responding:
set the timeout argument of SSHClient.connect (your doing that already),
and set sftp.get_channel().settimeout as already suggested by #EOhm
If you want to timeout even when the server is responding, but slowly, implement the callback argument to abort the transfer after certain time:
start_time = time.time()
def check_time(size, file_size):
global start_time
if (time.time() - start_time) > ...:
raise Exception
sftp.put(source, destination, callback=check_time)
This won't cancel the transfer immediately. To optimize transfer performance, Paramiko queues the write requests to the server. Once you attempt to cancel the transfer, Paramiko has to wait for the responses to those requests in SFTPFile.close() to clear the queue. You might solve that by using SFTPClient.putfo() and avoiding calling the SFTPFile.close() when the transfer is cancelled. But you won't be able to use the connection afterwards. Of course, you can also not use the optimization, then you can cancel the transfer without delays. But that kind of defies the point of all this, doesn't it?
Alternatively, you can run the transfer in a separate thread and kill the thread if it takes too long. Ugly but sure solution.
Use sftp.get_channel().settimeout(s) for that instead.
After trying a lot of things and with your help and advice, I have found a reliable solution for what I wanted. I execute sftp.put in a separate Thread and my script do what I want.
Many thanks for your help
Now if the server shuts down during a transfer, after 60 sec, my script goes on using:
[...]
import threading
[...]
th = threading.Thread(target=sftp.put, args=(source,destination))
th.start()
h.join(60)
[...]
Im writing a program which automatically creates servers in openstack when needed. The problem is that I want the program to wait until the instance has gotten its ip address before proceeding. If the instance has not gotten its ip, novaclient will throw an exception and the object will die. Using the sleep function makes it work, but I dont want that to be the permanent solution.
ipAddress = None
try:
instance = nova.servers.create(name=self.hostName, image=image,
flavor=flavor, key_name="mykey",
nics=nics)
while(ipAddress == None): #<---Something like this, just actually working
for network in instance.networks['my_net']:
if re.match('\d+\.\d+\.\d+\.\d+', network):
self.ipAddress = network
break
print 'The server is waiting at IP address {0}.'.format(self.ipAddress)
except:
print "could not create webserver"
#WebManager.exception(hostname)
finally:
print("Execution Completed")
self.addToLoadbalancer()
Is there a way to write a sort of spinlock or similar that will wait unntil the server have gotten its ip?
Any tips would be great.
I managed to fix the issue. It turned out it was hard to detect when the machine was ready by using only novaclient. By using nova list I managed to get the ip address.
while 1 == 1:
result = getbash("nova list" + " | grep " + hostname + \\
" | awk '{print $12}'").split('=')
if re.match('\d+\.\d+\.\d+\.\d+', result[-1]):
self.ipAddress = result[-1]
print 'The server is waiting at IP address {0}.'.format(self.ipAddress)
break
sleep(1)
This code queries for the hostname and checks if the instance has recieved an ip address. The getbash() function is a simple subprocess function which return the output of subprocess.Popen(command,stdout=subprocess.PIPE, shell=True)
I'm trying to write a multithreaded server in python using twisted. callInThread(self.task) is to create a new thread to run task() every time a client requests sth from the server. When the client sends requests one by one(all through port 53), everything works but when there are multiple requests at the same time, it says
File "", line 1, in bind
socket.error: [Errno 98] Address already in use
Is there sth wrong with my threads, only one can use the port at a time? If so, how am I supposed to go about with multithreading my server?
Thanks a lot!
class BaseThreadedUDPServer(DatagramProtocol):
def datagramReceived(self, datagram, (host, port)):
print "received %r from %s:%d" % (datagram, host, port)
reactor.callInThread(self.task)
def task(a):
print "waiting on port:", csport
while 1:
## RCV QUERY ##
query, addr = csSocket.recvfrom(csbuf)
## GET ANS ##
ans = socket.gethostbyname(query)
## SEND ANS ##
scSocket.sendto(ans, scaddr)
def main():
print "main"
reactor.listenUDP(53, BaseThreadedUDPServer())
reactor.run()
You don't need threads. This is horribly buggy. Twisted is already calling recv for you: and it is the result of that which is passed to datagramReceived. Don't call it again yourself. You don't need a thread.
However, that probably has nothing to do with your problem. 53 is the default DNS port: the problem you have is that another server, probably a DNS server is already running on that computer. Try changing 53 to some other value.
But I'm not really sure; in the future, please paste a full traceback. That traceback line obviously didn't come from the example that you've pasted, since there's nothing on line 1 except a 'class' statement. Also, since this code is indented wrong and raises a SyntaxError, it's obviously not exactly the same as what you're running.
Assuming you are actually doing something with DNS, Twisted has its own DNS server; you should be using twisted.names rather than implementing your own DNS packet parsing.
How would I check if the remote host is up without having a port number? Is there any other way I could check other then using regular ping.
There is a possibility that the remote host might drop ping packets
This worked fine for me:
HOST_UP = True if os.system("ping -c 1 " + SOMEHOST) is 0 else False
A protocol-level PING is best, i.e., connecting to the server and interacting with it in a way that doesn't do real work. That's because it is the only real way to be sure that the service is up. An ICMP ECHO (a.k.a. ping) would only tell you that the other end's network interface is up, and even then might be blocked; FWIW, I have seen machines where all user processes were bricked but which could still be pinged. In these days of application servers, even getting a network connection might not be enough; what if the hosted app is down or otherwise non-functional? As I said, talking sweet-nothings to the actual service that you are interested in is the best, surest approach.
HOST_UP = True if os.system("ping -c 5 " + SOMEHOST.strip(";")) is 0 else False
to remove nasty script execution just add .strip(";")
-c 5
to increase the number of ping requests, if all pass than True
PS. Works only on Linux, on Windows always returns True
The best you can do is:
Try and connect on a known port (eg port 80 or 443 for HTTP or HTTPS); or
Ping the site. See Ping a site in Python?
Many sites block ICMP (the portocol used to ping sites) so you must know beforehand if the host in question has it enabled or not.
Connecting to a port tells you mixed information. It really depends on what you want to know. A port might be open but the site is effectively hung so you may get a false positive. A more stringent approach might involve using a HTTP library to execute a Web request against a site and see if you get back a response.
It really all depends on what you need to know.
Many firewalls are configured to drop ping packets without responding. In addition, some network adapters will respond to ICMP ping requests without input from the operating system network stack, which means the operating system might be down, but the host still responds to pings (usually you'll notice if you reboot the server, say, it'll start responding to pings some time before the OS actually comes up and other services start up).
The only way to be certain that a host is up is to actually try to connect to it via some well-known port (e.g. web server port 80).
Why do you need to know if the host is "up", maybe there's a better way to do it.
What about trying something that requires a RPC like a 'tasklist' command in conjunction with a ping?
I would use a port scanner. Original question states that you don't want to use a port. Then you need to specify which Protocol (Yes, this needs a port) you want to monitor: HTTP, VNC, SSH, etc. In case you want to monitor via ICMP you can use subprocess and control ping parameters, number of pings, timeout, size, etc.
import subprocess
try:
res = subprocess.Popen(['ping -t2 -c 4 110.10.0.254 &> /dev/null; echo $?'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
out, err = res.communicate()
out = out.rstrip()
err = err.rstrip()
print 'general.connectivity() Out: ' + out
print 'general.connectivity() Err: ' + err
if(out == "0"):
print 'general.connectivity() Successful'
return True
print 'general.connectivity() Failed'
return False
except Exception,e:
print 'general.connectivity() Exception'
return False
In case you want a port scanner
import socket
from functools import partial
from multiprocessing import Pool
from multiprocessing.pool import ThreadPool
from errno import ECONNREFUSED
NUM_CORES = 4
def portscan(target,port):
try:
# Create Socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socketTimeout = 5
s.settimeout(socketTimeout)
s.connect((target,port))
print('port_scanner.is_port_opened() ' + str(port) + " is opened")
return port
except socket.error as err:
if err.errno == ECONNREFUSED:
return False
# Wrapper function that calls portscanner
def scan_ports(server=None,port=None,portStart=None,portEnd=None,**kwargs):
p = Pool(NUM_CORES)
ping_host = partial(portscan, server)
if portStart and portStart:
return filter(bool, p.map(ping_host, range(portStart, portStart)))
else:
return filter(bool, p.map(ping_host, range(port, port+1)))
# Check if port is opened
def is_port_opened(server=None,port=None, **kwargs):
print('port_scanner.is_port_opened() Checking port...')
try:
# Add More proccesses in case we look in a range
pool = ThreadPool(processes=1)
try:
ports = list(scan_ports(server=server,port=int(port)))
print("port_scanner.is_port_opened() Port scanner done.")
if len(ports)!=0:
print('port_scanner.is_port_opened() ' + str(len(ports)) + " port(s) available.")
return True
else:
print('port_scanner.is_port_opened() port not opened: (' + port +')')
return False
except Exception, e:
raise
except Exception,e:
print e
raise