Develop Reference

How to use WHOIS queries from python script when port 43 is likely not accessible

Im very new to network programming and faced a following problem:
Im working on VMware CentOS7 virtual machine on Windows10 host.
My script should send WHOIS queries and parse their output (e.g. expiration date).
However, an attempt to send a query leads to a connection error:
>>>import whois
>>>whois.query('google.com')
WhoisCommandFailed: connect: Network is unreachable
I tried to whois from terminal, but error was the same.
When i tried to use whois directly from Windows, which hosts virtual machine, the error seemed to look same as well (connection timeout).
As i found out, it was most likely related to access to port 43. I created rules (for in and out) for Windows firewall for this port by a guide , but error still persisted.
It looks like access to this port was blocked by ISP (however ping command is working).
To sum up, I got two questions there:
1) (less important) How to check if port 43 is blocked by firewall either by ISP?
2) (most important) Is it possible somehow to reconfigure WHOIS for usage of another port (i.e. 23) for sending queries by Python script?
Unfortunately, ISP security policy doesn't allow them to open 43 port.

Mostly ISP doesn't block any port but yes, this is not 100% true.
Testing connection:
run tcpdump (install command: yum install tcpdump) command on CentOS: tcpdump -peni any tcp and port 43
You have to see lines with the following text: 192.168.1.1.57350 > 192.34.234.30.43 where 192.34.234.30 IP address means the remote whois server.
Try to telnet to remote server's TCP/43 port: telnet 192.34.234.30 43
You should see the following:
Trying 192.34.234.30...
Connected to 192.34.234.30.
Escape character is '^]'.
If you can`t see context like that and you get back prompt immediately you will a firewall rule somewhere what is block connection. I recommend to switch off firewall temporarily and test again.
You cannot change port number, because it is configured on the remote side, on the server.
Can CentOS7 server communicate towards the internet? In example can you install packages?
Is there any router between windows machine and ISP?

See task progress when using Luigi remotely (with ssh)

If using Luigi in a server I am connected to with ssh, is it possible to see the progress of tasks (as I can if I use luigi locally by looking up "localhost" in browser)?
Any help appreciated

Short answer: yes
When you run luigid "locally", there is a server that starts on your system, which you, as you mentioned, you can access at http://localhost:8082 (or whatever port you specify). To make that work on a remote server, all you need to do is run luigid in said server, then point your browser at, instead of http://localhost:8082, http://:8082 (or whatever port you configure luigid to listen on.

Programmatically get IP address of vm in Vagrant

I'm trying to create a rest api which can launch, provision and destroy vagrant boxes.
So far I've managed to do most of the easy tasks such as provisioning, parsing logs to determine if provisioning was successfull, starting, stopping, destroying vms and parsing logs to find when a port was conflicting and has been remapped.
However, I can't figure out how to get around finding the new ip the box received from the router's DHCP.
I could run
vagrant ssh -c ifconfig
and try to parse the ip address but if i don't know the interface name it won't work.
Is there a way to have vagrant report what is it's local ip? alternatively is there a way to find this ip otherwise?
I'm using python-vagrant to interface python with vagrant but at this point even executing shell commands through python would be acceptable.
Thanks

You can tell the Vagrant to use a specific IP instead on getting a random one by adding
Vagrant.configure('2') do |config|
config.vm.network :private_network, ip: '192.168.x.x'
end
You should already have the Vagrant.configure section and this will provision the box using the specific IP you define.

Python process suspends on SSH logout after nohup/screen

I have a remote server through Blue Host that's intended to run a server based on Twisted for Python. The only access I have to it is over SSH, so to keep Python running after I log out I tried using nohup python server.py & and screen -dm python server.py, getting the same results for each. Everything works fine until I log out of SSH - even though Python is running in the background as expected, once I've logged out, my client can no longer communicate with the server. The strange part is that if I log back in over SSH and check the running processes with ps aux, I see Python running and my client can successfully communicate with the server again. Even if I don't type anything at all once I log back in, everything works as expected. But, of course, as soon as I log back out, it's as if the server is gone.
I've contacted support for the hosting service in case this is some oddity on their end, but hopefully this is something that can be resolved on my end instead.
Edit: Looks like Blue Host doesn't want me doing server-y stuff without buying the VPS upgrade so it looks like that's the big problem.
Edit 2: Okay, so in case anybody ends up having a similar problem, here's what the main issue turned out to be. I was mistaken in my original description; I was able to connect to the server but I was getting kicked off immediately for what turned out to be a MySQL error. I guess trying to connect to a localhost database with no active connection somehow causes problems, so instead I changed the MySQL connection command to connect to my site's IP address instead, even though it was the same IP as the server. That seemed to do the trick in terms of my main issue.

Don't use this method to keep the server process running. Instead try using supervisor (apt-get install supervisor). It allows you to daemonize your process, and ability to stop/restart etc.
Here's a sample config entry (/etc/supervisor/supervisord.conf):
[program:my_server]
command=python /path/to/server/server.py
directory=/path/to/server/
autostart=true
autorestart=true
stdout_logfile=/var/log/server.log
stderr_logfile=/var/log/server_error.log
user=your_linux_user_name
After you edit your config, do
sudo service supervisor stop
sudo service supervisor start #need to do this - doing a `restart` doesn't reload the config file!
your server should now be running properly. You can manage its lifecycle via sudo supervisorctl

device behind firewall connect via ssh

There have been a few questions like this around the place but none have really answered my question specifically.(for example Connecting to device behind firewall )
What I want is a central server, that receives a heartbeat from multiple ( say 100's) embedded devices behind personal firewalls. These devices need to be able to do two things.
Grab new config from the server. I
suspect I can just do this via a
http get from the device to the
server and pull down some XML, then
reload its own config.
Open an ssh connection to the server
to allow an admin to login to the
command line of the device and do
maintenance and troubleshooting
remotely.ie device => server <= admin and admin can get to bash command line or equivalent.
the device is a low powered embedded device that will be running linux. A solution in python would be preferable (im thinking something with paramiko for the ssh) but im open to other solutions. The main thing is there is there will be no technical users in the private network, so it should be able to plug into a consumer grade ADSL modem, get a DHCP address and all this should work. I can preload the device with anything before hand, for example ssh certificates for passwordless ssh etc.
anybody got any idea's?
Cheers
Mark

You can setup ssh tunnel (from python script or from console):
ssh -NR10022:localhost:22 foo#mainserver.com
Then you can simply login to main server and then ssh bar#localhost -p 10022
You should have ssh keys, so you don't have to put password (google about "ssh without password").

A more elaborate method might be some type of firewall hole punching.
On second though, maybe this is not necessary, since there is only one firewall involved. The trick is to get your embedded device to initiate an outbound connection first.