Develop Reference

Raw TCP Listen Socket on Cloud or Web Server

I have hardware that connects to raw TCP socket on any given IP and port combination. It then continually sends characters. The following piece of Python code may give you an idea of what the hardware does.
import socket
serverIP = '*server IP or domain*'
serverPort = 60000
Sock = socket(AF_INET, SOCK_STREAM)
Sock.connect((serverIP, serverPort))
while (1):
f = open ("send-data.txt","r")
while 1:
c = f.readline()
if not c:
break
Sock.send(c + '\n')
Sock.shutdown(0)
Sock.close()
When this code is run it exactly behaves like my hardware system. The send-data.txt file contains characters similar to what hardware sends.
I have written a socket server in Python using SocketServer library. It allows connections, receives character stream, and stores it into a local (newly created) file. Currently, I am running this code on my system, as localhost and it works. I would like to serve these files through a webpage.
I want to be able to do the same on remote server. As you can see, my hardware limits me to use only raw TCP sockets. From what I understand, I'll need low-level access to the server machine like IaaS. I tried pythonanywhere, but I guess they don't allow simple python sockets. Heroku also requires you to write a web app, and I don't know how to go about that or whether it'll work with my hardware.
What hosting/Cloud solution out there could act as above-mentioned socket server and also as HTTP server which would later serve these files and webpages.

If I understand your question correctly, you'd like to know which affordable hosting solution would allow you to communicate via arbitrary TCP sockets.
The answer is simple: Pretty much any VPS (Virtual Private Server) company or IaaS provider. Since you tagged your question with Amazon-EC2, yes they do too, but the learning curve to get your first instance running and the security groups (read: firewall rules, which live outside your VM) configured, is rather steep. That said, you do have a so-called "Free Tier" there for one year, which enables you to try out most of their services free-of-charge.
Other providers might be more suitable. (I'm not sure if it's allowed to suggest providers here, but you could for example look at Linode or Rackspace Cloud; they offer much less flexibility than EC2, but it's a whole lot easier to get started.)
As with any IaaS option, it would be beneficial to know Linux, networking and some security basics (at the very least) as you will be solely responsible for the things you create.
Talking about security...
If that piece of code you posted has a similarly rudimentary receiving end, you're setting yourself up for trouble as soon as it's out there in public, as the communication is done in plain text [*] and doesn't seem to require any kind of authentication. Anybody could probably telnet to the receiving end and just inject some lines of text?
(That's exactly why considerably sane PaaS providers often don't let you communicate over arbitrary ports and sockets :-) )
[*] I am guessing that, because you use readline. If any encryption was involved, you'd likely write/read in chunks of bytes.

How can I find the tcp address for another computer to transfer over ethernet?

I need to transfer data via pyzmq through two computers connected by an ethernet cable. I have already set up a script that runs on the same computer correctly, but I need to find the tcp address of the other computer in order to communicate. They both run Ubuntu 14.04. One of them should be a server processing requests while the other sends requests. How do I transfer data over tcp through ethernet? I simply need a way to find the address.
EDIT: (Clarification) I am running a behavioural study. I have a program called OpenSesame which runs in python and takes python scripts. I need a participant to be able to sit at a computer and be able to ask another person questions (specifically for help in a task). I need a server (using pyzmq preferably) to be connected by ethernet and communicate with that computer. It wrote a script. It works on the same computer, but not over ethernet. I need to find the address

Tcp is a protocol that uses an internet connection to Transfer data, through an IP address and specific port, you have to ensure that those ip directions are in the same range and uses the same port. For example: one pc 192.168.1.50 and another pc 192.168.1.60 in 502 port. The easiest way is using a Modem and set an statical address for each pc.
In the following link, you can find easily in which ip direction is connected your Computer. http://www.howtogeek.com/howto/17012/how-to-find-your-ip-address-in-ubuntu/

Maybe you could periodically send datagram message containing peer's ip address (or some other useful information) to broadcast address, to allow other peers to discover it. And after peer's address is dicovered you can estabish connection via ZeroMQ or other kind... connection. :)

Python Web Service vs Socket programming Client/Server Design

I need some help, i am on early design stage of a client server software and i don't know which of the 2 options (Web Service or Socket programming) are the right one for my software.
All programming is in python.
The layout:
PC will need to run a server service - this server will get commands from the local computer and will send them to the MiniPC.
MiniPC will need to run a client service - when it identify a command (method) he will go to hardware (connection by serial,usb.....), do something and return to the miniPC with result.
MiniPC get the Hardware result and sends it to the Logging server and to the Main PC
Notes:
PC can controls several MiniPC.
The amount of data in one hardware response can be up to 10Kb.
Commands from PC to MiniPC are small (strings)
Logging data can be up to 10Kb.
Questios:
What is you recomendation for protocol Web (http) or Socket programming?
Do you have any suggestions for the design?

You should be able to use socket programming for this. Setup a socket server at the PC and a client at MiniPC devices. The clients would wait for input (read from socket) from the PC and then send back the output that they would get back from hardware. In terms of design, I see two things. First, the socket server can run a select() to handle multiple clients. Second, you probably want to bump up the SO_SNDBUF socket option for MiniPC sockets and SO_RCVBUF for the server at PC to multiples of 10Kb. What is your argument for considering Web?

I'have done a similar project with ARM-based controllers instead on BeagleBone : feel free to ask me questions by commentaries.
Firstly, technically your BeagleBones are servers - since they ran a daemon service which is event triggered - and PC are clients. (but it is just pendantry)
Secondly, due to the limitations of embedded devices, I was not able to have an efficient Web server running on servers, so the choice was simple. I would advise you to stick with socket programming, but adding network services such as DCHP , support of TCP/UDP/UDP multicast, ping, echo, ...
Finally, the important question in terms of performance is the following :
what's the physical layer of communication ?
Ethernet ? Wifi ? Bluetooth/ZigBee ? I2C/CAN/... ?
I will guess it's Ethernet : IEEE 802.11 protocol doesn't scale well because of CSMA ( see here http://fr.wikipedia.org/wiki/CSMA ). If you want to have several devices (dozens), you will need switches/routers to encapsulate sub-networks to avoid network congestion.

Weird network behaviors with UDP sockets and python

I made some python script and ran it both on my computer and some distant shell (some website that provides shell access).
I'm using threads, pipes and UDP sockets to transfer data in a P2P fashion, so each script can both receive and send through the same socket. To test if this works, I open one terminal on my computer and one other terminal with ssh, connected to my shell. I make sure the script is the same on both machines, and feed it with an ip address.
Here is the script: http://codepad.org/V9Q1KcDT
(I don't know if I should paste it directly here or not)
My problem is this: strings I send seems to land something 20% of the time, sometimes often, sometimes not, and it seems to be random...
What am I doing wrong ?
Are UDP so unreliable ?
Are python thread+pipe+socket too slow ?
Could it be some kind of network problem with my shell provider ?
Is my program flawed ?
Are pipes a good solution to communicate with threads ?
I have no problem not using a shell and I have not tried, but it's useful for testing purposes.
BTW if i'm behing a router, how does the router knows where to send the packet if I'm not the only computer connected ? (I tried when I was the only one, it behaved identically).

User Datagram Protocol(UDP) could easily stand for Unreliable Datagram Protocol. UDP provides no guarantees regarding delivery so if you need reliability you have to implement it yourself by resending messages.
That said, threading and multiprocessing and remote machines and networks are a lot of variables to diagnose at once. I'd advise you to step back and try to boil the problem down to something simpler.
For instance:
first try TCP instead of UDP
instead of threads, run 2 processes (one doing nit_send, one doing nit_recv)
run them both on your local machine (use localhost: 127.0.0.1)
Once you have this basic test working then add features back in. For instance, switch to UDP. Once you've got your UDP issues sorted out locally try introducing remote machines or threads, etc.
Regarding the router question all machines connected to your router are most likely NAT'd so they all appear to have the same IP address (the IP address of your router) to machines on the internet. Look into NAT and port-forwarding if you want to route traffic to a specific machine on your local subnet.

python twisted stdio multiple connections to a server with a command prompt for interaction

I have written a simple twisted application that connects to a server that listens on 1 or more ports. The twisted app connects to this server and usually connects to a few of the open ports at a time. This server is a serial logger that connects to serial devices and provides the serial line information through a raw TCP Socket and I need to log all this data to disk.
My current app logs any received information to disk without issue.
What I now need to do but am unable to make progress on is add the ability to interact with my application through stdin. I need to be able to issue commands to the local application but also send text commands through the connected sockets.
I have a basic prompt using basic.LineReceiver and adding this to my reactor but can't figure out how to send the data to the server or even if this is the correct way of doing this.
A simplified example would be helpful to show what I need to do.
Thanks
J

To add an interactive console to your Twisted app, see this article -- it explains how to use twisted.internet.stdio for the purpose.