Develop Reference

Creating an FTP server with python [duplicate]

Is it possible to have a one line command in python to do a simple ftp server? I'd like to be able to do this as quick and temporary way to transfer files to a linux box without having to install a ftp server. Preferably a way using built in python libraries so there's nothing extra to install.

Obligatory Twisted example:
twistd -n ftp
And probably useful:
twistd ftp --help
Usage: twistd [options] ftp [options].
WARNING: This FTP server is probably INSECURE do not use it.
Options:
-p, --port= set the port number [default: 2121]
-r, --root= define the root of the ftp-site. [default:
/usr/local/ftp]
--userAnonymous= Name of the anonymous user. [default: anonymous]
--password-file= username:password-style credentials database
--version
--help Display this help and exit.

Check out pyftpdlib from Giampaolo Rodola. It is one of the very best ftp servers out there for python. It's used in google's chromium (their browser) and bazaar (a version control system). It is the most complete implementation on Python for RFC-959 (aka: FTP server implementation spec).
To install:
pip3 install pyftpdlib
From the commandline:
python3 -m pyftpdlib
Alternatively 'my_server.py':
#!/usr/bin/env python3
from pyftpdlib import servers
from pyftpdlib.handlers import FTPHandler
address = ("0.0.0.0", 21) # listen on every IP on my machine on port 21
server = servers.FTPServer(address, FTPHandler)
server.serve_forever()
There's more examples on the website if you want something more complicated.
To get a list of command line options:
python3 -m pyftpdlib --help
Note, if you want to override or use a standard ftp port, you'll need admin privileges (e.g. sudo).

Why don't you instead use a one-line HTTP server?
python -m SimpleHTTPServer 8000
will serve the contents of the current working directory over HTTP on port 8000.
If you use Python 3, you should instead write
python3 -m http.server 8000
See the SimpleHTTPServer module docs for 2.x and the http.server docs for 3.x.
By the way, in both cases the port parameter is optional.

The answers above were all assuming your Python distribution would have some third-party libraries in order to achieve the "one liner python ftpd" goal, but that is not the case of what #zio was asking. Also, SimpleHTTPServer involves web broswer for downloading files, it's not quick enough.
Python can't do ftpd by itself, but you can use netcat, nc:
nc is basically a built-in tool from any UNIX-like systems (even embedded systems), so it's perfect for "quick and temporary way to transfer files".
Step 1, on the receiver side, run:
nc -l 12345 | tar -xf -
this will listen on port 12345, waiting for data.
Step 2, on the sender side:
tar -cf - ALL_FILES_YOU_WANT_TO_SEND ... | nc $RECEIVER_IP 12345
You can also put pv in the middle to monitor the progress of transferring:
tar -cf - ALL_FILES_YOU_WANT_TO_SEND ...| pv | nc $RECEIVER_IP 12345
After the transferring is finished, both sides of nc will quit automatically, and job done.

For pyftpdlib users. I found this on the pyftpdlib website. This creates anonymous ftp with write access to your filesystem so please use with due care. More features are available under the hood for better security so just go look:
sudo pip3 install pyftpdlib
python3 -m pyftpdlib -w
## updated for python3 Feb14:2020
Might be helpful for those that tried using the deprecated method above.
sudo python -m pyftpdlib.ftpserver

apt-get install python3-pip
pip3 install pyftpdlib
python3 -m pyftpdlib -p 21 -w --user=username --password=password
-w = write permission
-p = desired port
--user = give your username
--password = give your password

Install:
pip install twisted
Then the code:
from twisted.protocols.ftp import FTPFactory, FTPRealm
from twisted.cred.portal import Portal
from twisted.cred.checkers import AllowAnonymousAccess, FilePasswordDB
from twisted.internet import reactor
reactor.listenTCP(21, FTPFactory(Portal(FTPRealm('./'), [AllowAnonymousAccess()])))
reactor.run()
Get deeper:
http://twistedmatrix.com/documents/current/core/examples/

The simpler solution will be to user pyftpd library. This library allows you to spin Python FTP server in one line. It doesn’t come installed by default though, but we can install it using simple apt command
apt-get install python-pyftpdlib
now from the directory you want to serve just run the pythod module
python -m pyftpdlib -p 21

I dont know about a one-line FTP server, but if you do
python -m SimpleHTTPServer
It'll run an HTTP server on 0.0.0.0:8000, serving files out of the current directory. If you're looking for a way to quickly get files off a linux box with a web browser, you cant beat it.

launch tensorboard in colab

i try launch tensorboard on colab, my code:
LOG_DIR = model_dir
get_ipython().system_raw(
'tensorboard --logdir {} --host 0.0.0.0 --port 6060 &'
.format(LOG_DIR)
)
get_ipython().system_raw('./ngrok http 6060 &')
! curl -s http://localhost:4040/api/tunnels | python3 -c \
"import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"
two days ago everything worked, but now such an error:
error

The error linked suggests the port you're using is 6006, but the code example you gave above has the port as 6060. So may just be a typo there.
It's also possible you want a TCP tunnel, not an HTTP tunnel.
In either case, might I suggest trying something like pyngrok to programmatically manage your ngrok tunnel for you? Full disclosure, I am the developer of it. Here are the docs if you're interest.

Running HTML file in localhost:8080

I want to run an HTML file in localhost:8080, I'm using the command:
python3 -m http.server
Problem is when I try to open localhost:8080 it downloads the HTML file instead of displaying it.

Problem is when I try to open localhost:8080 it downloads the HTML file instead of displaying it.
You want to open http://localhost:8000 instead.
When you use the command you mentioned, python3 -m http.server, it defaults to port 8000, as explained in its startup output:
$ python3 -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
We don't know what different server you have running on port 8080, but apparently it doesn't put Content-type: text/html in its output headers.
viewing webserver http headers
It's easy to view those headers, e.g. with wget use the -S switch.

You need to add an option that'll put your website on port 8080 because the http.server command defaults to port 8000.
You can do this using:
python3 -m http.server 8080
Then when you go to 0.0.0.0:8080 it should show you your webpage instead of a download prompt.
Also, you might have another instance of http.server running on port 8080.
You can find the PID of this task using:
ps -A | grep "python3"
Which should show something that looks like this:
Then you could kill it using:
kill <PID-FOR-PYTHON3-INSTANCE>
Or in my case the task that's running on port 8080 is:
kill 6856
Or, if you don't mind, just kill all Python3 tasks using:
killall python3
Which in my case would kill both Python3 tasks.
WARNING: be very, VERY careful before running the killall command, because this command will NOT save your work.
UPDATE: that blurry section in the picture is my username, I wasn't sure if it would be against the rules to include it.
Good luck.

Nvidia Theano docker image not available

Trying to run docker command :
nvidia-docker run -d -p 8888:8888 -e PASSWORD="123abcChangeThis" theano_secure start-notebook.sh
# Then open your browser at http://HOST:8888
taken from https://github.com/nouiz/Theano-Docker
returns error :
Error: image library/theano_secure:latest not found
Appears the theano_secure image is not currently available ?
Searching for theano_secure :
$ nvidia-docker search theano_secure:latest
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
The return of this command is empty so image is not available ?
If so is there an alternative Theano docker image from nvidia ?
Update :
building from source :
docker build -t theano_secure -f Dockerfile.0.8.X.jupyter.cuda.secure .
returns :
Err http://developer.download.nvidia.com Release.gpg
Unable to connect to developer.download.nvidia.com:http: [IP: 184.24.98.231 80]
and :
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease
Manually checking URL's : http://developer.download.nvidia.com & http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease are both not available. Should I build with alternative docker file ?
Update 2 :
I think this error is occurring as http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease does not exist. However http://archive.ubuntu.com/ubuntu/dists/trusty/Release does exist.
Can docker be modified to use http://archive.ubuntu.com/ubuntu/dists/trusty/Release instead of http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease ?
OS version :
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.4 LTS
Release: 14.04
Codename: trusty
Update 3 :
"you are supposed to docker build first", before nvidia-docker run" I did try
docker build -t theano_secure -f Dockerfile.0.8.X.jupyter.cuda.secure .
which returns :
Err http://developer.download.nvidia.com Release.gpg Unable to connect to developer.download.nvidia.com:http: [IP: 184.24.98.231 80]
I can pull image docker pull kaixhin/theano but this does not run via Jupyter notebook in same way as nvidia-docker run -it -p 8888:8888 tensorflow/tensorflow:latest-gpu documented at https://hub.docker.com/r/tensorflow/tensorflow/ . There does not appear to be a docker Jupyter Theano container available.
How to expose the docker instance kaixhin/theano via Jupyter notebook ?
I tried : nvidia-docker run -d -p 8893:8893 -v --name theano2 kaixhin/theano start-notebook.sh but receive error :
docker: Error response from daemon: invalid header field value "oci runtime error: container_linux.go:247:
starting container process caused \"exec: \\\"start-notebook.sh\\\": executable file not found in $PATH\"\n".
Modification of kaixhin/theano docker container in order expose it via Jupyter notebook ?

Error: image library/theano_secure:latest not found
Because theano_secure doesn't like ubuntu,centos, it is not official repository on docker hub, so you need to build it by yourself.
Err http://developer.download.nvidia.com Release.gpg Unable to connect to developer.download.nvidia.com:http: [IP: 184.24.98.231 80]
Please check your internet connection first, telnet 184.24.98.231 80.
Maybe you are in a limited network place, try behind a proxy to do this again. You may want take a look about how to build image behind a proxy.

From what I understand of the nouiz/Theano-Docker README, you are supposed to docker build first, before nvidia-docker run.
But since the build is tricky, I would try instead docker pull kaixhin/theano (from kaixhin/cuda-theano/), much more recent (3 days ago), which is based on theano Dockerfile.
That image does rely on CUDAand needs to be run on an Ubuntu host OS with NVIDIA Docker installed. The driver requirements can be found on the NVIDIA Docker wiki.

One line ftp server in python

Is it possible to have a one line command in python to do a simple ftp server? I'd like to be able to do this as quick and temporary way to transfer files to a linux box without having to install a ftp server. Preferably a way using built in python libraries so there's nothing extra to install.

Obligatory Twisted example:
twistd -n ftp
And probably useful:
twistd ftp --help
Usage: twistd [options] ftp [options].
WARNING: This FTP server is probably INSECURE do not use it.
Options:
-p, --port= set the port number [default: 2121]
-r, --root= define the root of the ftp-site. [default:
/usr/local/ftp]
--userAnonymous= Name of the anonymous user. [default: anonymous]
--password-file= username:password-style credentials database
--version
--help Display this help and exit.

Check out pyftpdlib from Giampaolo Rodola. It is one of the very best ftp servers out there for python. It's used in google's chromium (their browser) and bazaar (a version control system). It is the most complete implementation on Python for RFC-959 (aka: FTP server implementation spec).
To install:
pip3 install pyftpdlib
From the commandline:
python3 -m pyftpdlib
Alternatively 'my_server.py':
#!/usr/bin/env python3
from pyftpdlib import servers
from pyftpdlib.handlers import FTPHandler
address = ("0.0.0.0", 21) # listen on every IP on my machine on port 21
server = servers.FTPServer(address, FTPHandler)
server.serve_forever()
There's more examples on the website if you want something more complicated.
To get a list of command line options:
python3 -m pyftpdlib --help
Note, if you want to override or use a standard ftp port, you'll need admin privileges (e.g. sudo).

Why don't you instead use a one-line HTTP server?
python -m SimpleHTTPServer 8000
will serve the contents of the current working directory over HTTP on port 8000.
If you use Python 3, you should instead write
python3 -m http.server 8000
See the SimpleHTTPServer module docs for 2.x and the http.server docs for 3.x.
By the way, in both cases the port parameter is optional.

The answers above were all assuming your Python distribution would have some third-party libraries in order to achieve the "one liner python ftpd" goal, but that is not the case of what #zio was asking. Also, SimpleHTTPServer involves web broswer for downloading files, it's not quick enough.
Python can't do ftpd by itself, but you can use netcat, nc:
nc is basically a built-in tool from any UNIX-like systems (even embedded systems), so it's perfect for "quick and temporary way to transfer files".
Step 1, on the receiver side, run:
nc -l 12345 | tar -xf -
this will listen on port 12345, waiting for data.
Step 2, on the sender side:
tar -cf - ALL_FILES_YOU_WANT_TO_SEND ... | nc $RECEIVER_IP 12345
You can also put pv in the middle to monitor the progress of transferring:
tar -cf - ALL_FILES_YOU_WANT_TO_SEND ...| pv | nc $RECEIVER_IP 12345
After the transferring is finished, both sides of nc will quit automatically, and job done.

For pyftpdlib users. I found this on the pyftpdlib website. This creates anonymous ftp with write access to your filesystem so please use with due care. More features are available under the hood for better security so just go look:
sudo pip3 install pyftpdlib
python3 -m pyftpdlib -w
## updated for python3 Feb14:2020
Might be helpful for those that tried using the deprecated method above.
sudo python -m pyftpdlib.ftpserver

apt-get install python3-pip
pip3 install pyftpdlib
python3 -m pyftpdlib -p 21 -w --user=username --password=password
-w = write permission
-p = desired port
--user = give your username
--password = give your password

Install:
pip install twisted
Then the code:
from twisted.protocols.ftp import FTPFactory, FTPRealm
from twisted.cred.portal import Portal
from twisted.cred.checkers import AllowAnonymousAccess, FilePasswordDB
from twisted.internet import reactor
reactor.listenTCP(21, FTPFactory(Portal(FTPRealm('./'), [AllowAnonymousAccess()])))
reactor.run()
Get deeper:
http://twistedmatrix.com/documents/current/core/examples/

The simpler solution will be to user pyftpd library. This library allows you to spin Python FTP server in one line. It doesn’t come installed by default though, but we can install it using simple apt command
apt-get install python-pyftpdlib
now from the directory you want to serve just run the pythod module
python -m pyftpdlib -p 21

I dont know about a one-line FTP server, but if you do
python -m SimpleHTTPServer
It'll run an HTTP server on 0.0.0.0:8000, serving files out of the current directory. If you're looking for a way to quickly get files off a linux box with a web browser, you cant beat it.