Develop Reference

Curl "Connection Refused" when connecting to server with external IP address

I've created a RESTful server and I can successfully make requests and get responses when I use my local IP address. I would like it to be exposed externally to the internet. I set up a port forwarding rule but I cannot seem to get things working. From what I'm reading the "Connection Refused" with (7) means something is blocking it whether it's a firewall or ISP issue. Any ideas on what to do?
Here's my curl command that works with my local IP:
Pako-2:Pokebot pako$ curl -X GET http://192.168.1.8:30000/api/v1/getrecord/test
{"data": [{"id": 1, "title": "learn python"}, {"id": 2, "title": "get paid"}]}
This is what I see when I try using my external IP address given to me by What's my ip
Pako-2:Pokebot pako$ curl -X GET http://MyIpHere:30000/api/v1/getrecord/test
curl: (7) Failed to connect to myIPAddress port 30000: Connection refused
Here's the port forwarding rule I made in my router/modem
Here are my router/modem advanced settings options. I tried tweaking my firewall settings, but no luck. I tried with NAT only and also with Low Security Level with all ports checked.
-----------------Edit-----------------
Here is the port forwarding screen, should I just set 30000 as the begin and end?
Here is some python code for my server:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from SocketServer import ThreadingMixIn
import threading
import argparse
import re
import cgi
import json
TODOS = [
{'id': 1, 'title': 'learn python'},
{'id': 2, 'title': 'get paid'},
]
class LocalData(object):
records = {}
class HTTPRequestHandler(BaseHTTPRequestHandler):
print "HTTPRequestHandler BaseHTTPRequestHandler = ", BaseHTTPRequestHandler
def do_POST(self):
print "do_POST"
if None != re.search('/api/v1/addrecord/*', self.path):
ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
if ctype == 'application/json':
length = int(self.headers.getheader('content-length'))
data = cgi.parse_qs(self.rfile.read(length), keep_blank_values=1)
recordID = self.path.split('/')[-1]
LocalData.records[recordID] = data
print "record %s is added successfully" % recordID
else:
data = {}
self.send_response(200)
self.end_headers()
else:
self.send_response(403)
self.send_header('Content-Type', 'application/json')
self.end_headers()
return
def do_GET(self):
print "do_GET"
self.send_response(200)
self.end_headers()
self.wfile.write(json.dumps({'data': TODOS}))
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
allow_reuse_address = True
def shutdown(self):
self.socket.close()
HTTPServer.shutdown(self)
class SimpleHttpServer():
def __init__(self, ip, port):
self.server = ThreadedHTTPServer((ip, port), HTTPRequestHandler)
def start(self):
self.server_thread = threading.Thread(target=self.server.serve_forever)
self.server_thread.daemon = True
self.server_thread.start()
def waitForThread(self):
self.server_thread.join()
def addRecord(self, recordID, jsonEncodedRecord):
LocalData.records[recordID] = jsonEncodedRecord
def stop(self):
self.server.shutdown()
self.waitForThread()
if __name__ == '__main__':
server = SimpleHttpServer("0.0.0.0", 30000)
print 'HTTP Server Running...........'
server.start()
server.waitForThread()
------------------- Edit 2--------------------
Tried only having 1 port, not a begin/end range and it's expecting a range...

I find it strange that you'd be getting connection refused from within the local network because the router should be able to detect that the external IP is a reference to itself.
That ability may depend on the router, but that's been my experience with networking.
That being said, if you'd like to test the external firewall after playing with the port forwarding rules, then you need a device not on the local network.
If another device like a smart phone with a cell plan isn't available to you, then you could try to use CanYouSeeMe to test if the port is being opened.
You may also want to set DHCP reservation for your server. In other words, set a static IP address.

python multithreading server

I am new to networking programming and python.
I am trying to figure out how to run different jobs at the server side.
For example, I want one function to create connections for incoming clients but in the same time I can still do some administration work from the terminal.
My code is as below but it doesn't work:
Edited: it doesn't work means it will get stuck in the init_conn() function
Like:
starting up on localhost port 8887
Thread: 0 Connected with 127.0.0.1:48080
# waiting
I am looking into SocketServer framework but don't know how that works.
from thread import *
import socket
def init_conn():
thread_count =0
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 8887)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(10)
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = sock.accept()
print 'Thread: '+ str(thread_count) + ' Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
thread_count +=1
sock.close()
def clientthread(conn):
# receive data from client and send back
def console():
print 'this is console'
option = raw_input('-v view clients')
if option == 'v':
print 'you press v'
def main():
start_new_thread( init_conn(),() )
start_new_thread( console(),() )
if __name__ == "__main__":
main()

Your problem is probably that you start the program, sometimes it prints "this is console" and then it ends.
The first bug is that you call the methods instead of passing the handle to start_new_thread. It must be:
start_new_thread( init_conn, () )
i.e. no () after the function name.
The program doesn't do much because start_new_thread() apparent starts a thread and then waits for it to stop. The documentation is pretty unclear. It's better to use the new threading module; See http://pymotw.com/2/threading/
def main():
t = threading.Thread( target=init_conn )
t.daemon = True
t.start()
console()
so the code will run until console() ends.
I suggest to split the server and the command line tool. Create a client which accepts commands from the command line and sends them to the server. That way, you can start the console from anywhere and you can keep the code for the two separate.

Seeing that you're new to python, have you tried taking a look at the threading module that comes with the standard library?
import threading
... #rest of your code
while conditions==True:
i = threading.Thread(target=init_conn)
c = threading.Thread(target=console)
i.start()
c.start()
Can't say I've done too much with networking programming with python, so I don't really have much to say in that manner, but at least this should get you started with adding multithreading to your project.

Using SocketServer you may implement a client/server system. The documentation gives small examples which may be useful for you. Here is an extended example from there:
server.py :
import SocketServer
import os
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
class MyServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
# By setting this we allow the server to re-bind to the address by
# setting SO_REUSEADDR, meaning you don't have to wait for
# timeouts when you kill the server and the sockets don't get
# closed down correctly.
allow_reuse_address = True
request_queue_size = 10
def __init__(self, port):
self.host = os.uname()[1]
self.port = port
SocketServer.TCPServer.__init__(self, (self.host,self.port), MyTCPHandler)
logging.info( "Server has been started on {h}:{p}".format(h=self.host,p=self.port) )
class MyTCPHandler(SocketServer.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
# max length is here 1024 chars
self.data = self.request.recv(1024).strip()
logging.info( "received: {d}".format(d=self.data) )
# here you may execute different functions according to the
# request string
# here: just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
PORT = 8887
if __name__ == "__main__":
# Create the server, binding to localhost on port 8887
#server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
server = MyServer( PORT )
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
client.py
import socket
import sys
import logging
FORMAT = '[%(asctime)-15s] %(message)s'
logging.basicConfig(format=FORMAT, level=logging.DEBUG)
HOST, PORT = "workstation04", 8887
logging.info( "connect to server {h}:{p}".format(h=HOST,p=PORT ) )
# read command line
data = " ".join(sys.argv[1:])
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Connect to server and send data
sock.connect((HOST, PORT))
sock.sendall(data + "\n")
# Receive data from the server and shut down
received = sock.recv(1024)
finally:
sock.close()
logging.info( "Sent: {}".format(data) )
logging.info( "Received: {}".format(received) )
The output looks something like:
server side:
> python server.py
[2015-05-28 11:17:49,263] Server has been started on disasterarea:8887
[2015-05-28 11:17:50,972] received: my message
client side:
[2015-05-28 11:17:50,971] connect to server disasterarea:8887
[2015-05-28 11:17:50,972] Sent: my message
[2015-05-28 11:17:50,972] Received: MY MESSAGE
You can run several clients (from different consoles) in parallel. You may implement a request processor on the server side which processes the incoming requests and executes certain functions.
Alternatively, you may use the python module ParallelPython which executes python code locally on a multicore system or on a cluster and clusters. Check the http examples.
I had to force pip to install this module:
pip install --allow-external pp --allow-unverified pp pp

peer to peer socket based chat python and ios

I successfully created a simple chat server using this tutorial of reywenderlich
http://www.raywenderlich.com/3932/
Code:
from twisted.internet.protocol import Protocol, Factory
from twisted.internet import reactor
class IphoneChat(Protocol):
def connectionMade(self):
#self.transport.write("""connected""")
self.factory.clients.append(self)
print "clients are ", self.factory.clients
def connectionLost(self, reason):
self.factory.clients.remove(self)
def dataReceived(self, data):
#print "data is ", data
a = data.split(':')
if len(a) > 1:
command = a[0]
content = a[1]
msg = ""
if command == "iam":
self.name = content
msg = self.name + " has joined"
elif command == "msg":
msg = self.name + ": " + content
print msg
for c in self.factory.clients:
c.message(msg)
def message(self, message):
self.transport.write(message + '\n')
factory = Factory()
factory.protocol = IphoneChat
factory.clients = []
reactor.listenTCP(79, factory)
print "Iphone Chat server started"
reactor.run()
Basically the whole tutorial shows a chat implementation where user join a specific chat room and chat with each other real-time. But my client wants a peer-to-peer chat mechanism where the user choose a specific user to chat with so I really don't have any idea to make it to reality. I would really appreciate your help. Thanks

To create a p2p connection you need to create a protocol that would allow one of the sides to know the IP address and a listening port of the other side, and the other side to listen on a specific port.
This can be done by creating a request to the chat server to start a p2p connection. The chat server will then send the other client a message asking for permission and after a positive response which should include the listening port transfer that information to the initiating party along with the IP address of his peer.
At this point the initiating client should create a TCP connection to the other party and from the moment the connection is established you can transfer whatever data you'd like.
A simple way to do that would be to allow ASCII-escaped commands to the chat server, for instance /p2p <NICKNAME> which would tell the server that the client wants to start a p2p connection. The server will then be able to send something like /p2p-request from <NICKNAME> to the requested chatter and he could respond with something like /p2p-accept <port> or /p2p-reject <reason>. If the response was /p2p-accept the server can then return to the originating client with something line /p2p-accepted <IP> <port>.
Along with the /p2p-accept the client will need to create a listening socket:
p2p_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
p2p_socket.bind(('', port))
p2p_socket.listen(1)
p2p_chat, p2p_address = p2p_socket.accpet()
The initiating party will need to connect to this socket:
p2p_chat = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
p2p_chat.connect((ip, port))
And once connection is established send() and recv() can be used to send data between the clients.
This can, of course, be written over Twisted, but I'm not well versed in it so I'd rather stick to the low APIs.

Getting iOS app to talk to my home server

Currently, I have a home server in my closet basically doing nothing. It has Ubuntu Server 8.0.4 installed, apache for web development ill be using later, ssh, and python/twisted installed.
Here's the issue:
I created an app to talk to "localhost" port-40, using socket implementation, here is a link of what I did but what I want to develop off of : http://www.raywenderlich.com/3932/how-to-create-a-socket-based-iphone-app-and-server
Now connecting to the localhost is no problem but I want to expand this to work with my server.
I implemented the python protocol onto my server and changed the ip address im accessing in the iOS app. Here's the implementation I have, it's exactly the same as the tutorial except for the port i'm accessing.
from twisted.internet.protocol import Factory, Protocol
from twisted.internet import reactor
class IphoneChat(Protocol):
def connectionMade(self):
self.factory.clients.append(self)
print "clients are ", self.factory.clients
def connectionLost(self, reason):
self.factory.clients.remove(self)
def dataReceived(self, data):
a = data.split(':')
print a
#b = password.split(':')
#print b
if len(a) > 1:
command = a[0]
content = a[1]
#username = a[2]
#password = a[2]
msg = ""
#msg2 = ""
if command == "username":
self.name = data
msg = self.name + " has joined"
#self.name = password
#msg2 = self.name + " this is his password"
#print msg
elif command == "password":
self.name = data
msg = self.name + " this is their password"
elif command == "msg":
msg = self.name + ": " + data
print msg
#msg2 = self.name + ": " + password
#print msg2
for c in self.factory.clients:
c.message(msg)#1, msg2)
def message(self, message):
self.transport.write(message + '\n')
factory = Factory()
factory.protocol = IphoneChat #here1#
factory.clients = []
reactor.listenTCP(40, factory)
print "Iphone Chat server started"
reactor.run()
So the REAL issue is, I cannot connect clients to my server or something..... I'm sorry but I am very new to networking.
Any opinion will help.

If you've not established networking communication to your server from the internet before you'll want to establish that you can do that with simple test cases first, there's a lot that can stop traffic from the internet reaching your program on your server.
Is your server connected to the internet through a router? If so, can you communicate with the server from inside the local network (i.e. use the 192.168.xxx.xxx ip address), using something like netcat or telnet?
If that works you should try from outside the network (i.e. using the other ip address, from whatismyip.net or similar). If you've really no prior experience with networking you may have neglected to set up port forwarding, this is a setting on your router.
There are a lot of tutorials around teaching you how to set up a Ubuntu home server, I suggest learning how to host a (very) simple webpage as a means of learning how to network, this will help a lot with debugging a networked program like the one you're making.

Socket Server in Python

I am trying to set up a local server so that other PCs on the same local network can connect to it. When trying to do so, on the client side, I get the following error:
[Errno 10061] No connection could be made because the target machine actively refused it
I have been searching around for hours and still couldn't resolve this issue. I tried turning off my Firewall too, but nothing.
These are my server and client codes:
Server Code:
import socket
import threading
import SocketServer
import datetime
ver_codes = []
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
print threading.current_thread().isDaemon()
data = self.request.recv(1024)
command = data.split()[0]
if(command=="login"):
if(logged_in(data.split()[1])==False):
self.request.sendall(login(data.split()[1], data.split()[2]))
else:
self.request.sendall("already in")
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
sock.sendall(message)
response = sock.recv(1024)
print "Received: {}".format(response)
finally:
sock.close()
def logged_in(id_num):
for i in ver_codes:
if(i[0]==id_num):
return True
return False
def login(username, password):
login_file = open("Login.txt", "r")
match = login_file.readline()
while(match!="*"):
if(match.split()[0]==username):
if(match.split()[1]==password):
ver_codes.append([match.split()[0], encryption_code(match.split()[2])])
login_file.close()
return "{} {}".format(match.split()[2], encryption_code(match.split()[2]))
print "And Here"
match = login_file.readline()
return "Denied"
login_file.close()
def encryption_code(to_encrypt):
now = datetime.datetime.now()
return int(str(now.microsecond)) * int(to_encrypt)
if __name__ == "__main__":
HOST, PORT = "localhost", 7274
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
print server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = False
server_thread.start()
print "Server loop running in thread:", server_thread.name
Client Code:
import socket
import sys
HOST, PORT = "localhost", 7274
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
sock.connect((HOST, PORT))
sock.sendall("login mahdiNolan m1373")
received = sock.recv(1024)
finally:
sock.close()
I really appreciate any help you could give me!
Thanks A LOT beforehand!

Your issue is because you're listening on localhost - this will only accept connections from the local machine.
If you want to accept connections from anywhere, instead of "localhost" just pass the empty string "". This is equivalent to specifying INADDR_ANY to the C sockets API - see the ip man page for more information, or this page also looks like it has some useful explanation. In short, this means "accept connections on any local interface".
Instead of the empty string you can instead specify an IP address of a local interface to only accept connections on that interface - it's unlikely you need to do this unless you machine has multiple network cards inside it (e.g. acting as a gateway) and you only want to serve requests on one of the networks.
Also, on the client side you should use the actual address of the machine - replace "localhost" with the IP address or hostname of the server machine. For example, something like "192.168.0.99". If you want to find the IP address of the server under Windows, open a DOS window and run the ipconfig command, look for the line with IPv4 Address (assuming you've got an IPv4 network which is very likely).
The Windows firewall will also block the server from accepting connections as you've already found, but you shouldn't need to disable it - as soon as you run your server you should see a popup window where you can instruct it to accept connections (that was on Windows 7, it might be different on other versions). In any case, turning the software firewall off should allow everything to work, although whether that's a security risk is a matter outside of the scope of this question.