I need to connect from a windows server 2008 in a secure network to an ubuntu box and write and read files easily from python code. I want to avoid samba or ftp, so I am considering NFS and my question is, if pynfs works stable on windows (if at all, or does it work on linux only?)
I found the source and some forks on github
I am also unsure about the state of the project, it is not in pypi, it cannot be installed over pip, so I wonder if this is a maintaned and updated project with a future. It would be great to hear from someone who has some production experience with it, I am using python 2.7
I would prefer pynfs had some modern infrastructure around it.
I went with samba this time.
pynfs is a test suite and not ment to run as nfs server in production
Related
I have the last six months been working on a Python GUI application that I will use at work. Specifically my GUI will run on a couple of super computer clusters that I use for work.
However, I am mostly developing the software at my personal computer, and here I do not have direct access to the commands that my GUI will call, since the GUI will use subprocess to call commands that only are available on the computing cluster.
So, in order to efficiently develop the program, I often have to copy the directory containing all files related to the GUI, to the cluster. Then I test my current version there, locate all my bugs, fix them by editing the files on the cluster, and finally copy back all files to my computer, overwriting the old version.
This just seems like a bad way of doing it, but I have to be able to test my software in the environment it is made for in order to find my bugs.
Surely this is a common problem in software development... What do actual programmers do (as opposed to hobby programmers such as myself)?
Edit:
Examples of commands that are only available on the computing cluster, that I make heavy use of, are squeue, sacct, and scontrol (SLURM related commands).
Edit2:
I could mention that I tested using ssh connections with Python, but it slowed down the commands significantly, having to establish the ssh connection for each command I wanted. Unless I could set of a lasting ssh session, as in logging in when opening my program, I don't think the ssh-ing will work.
Explore the concepts that make Vagrant a popular choice for developers
Vagrant is a tool for building and managing virtual machine
environments in a single workflow. With an easy-to-use workflow and
focus on automation, Vagrant lowers development environment setup
time, increases production parity, and makes the "works on my machine"
excuse a relic of the past.
Your use case is covered by a couple of vagrant boxes that create a slurm cluster for development purposes. A good starting point might be
Example slurm cluster on your laptop (multiple VMs via vagrant)
If you understand and can setup your development environment with tools like Vagrant, you might explore next which options modern code editors or integrated development environments (IDE) offer for remote development. Remote development covers some other use cases, that might fit into your developer toolbox as well.
A "good enough", free and open source code editor for Python development is Visual Studio Code. According to the docs it has powerful features for remote development.
Visual Studio Code Remote Development allows you to use a container, remote machine, or the Windows Subsystem for Linux (WSL) as a full-featured development environment.
Read the docs
VS Code Remote Development
I've been wanting to run my own server for a while and I figured that running one for my django website would be a good start. What do you recommend I use for this? I've been trying to use a Ubuntu Virtual Machine to run it on one of my old laptops that I don't really use anymore until I can buy a dedicated server.
Should I run it from a Virtual Machine? If so, would Ubuntu be best? That appears to be the case, but I want to be sure before I invest in anything. I want to be able to access the website from other computers, just like any other website. Am I going about this wrong? If so, what can you suggest me?
Yes, you will need a static IP address.
If this is your first experiment, my advice would be:
1) Use an old, dedicated PC with no other stuff on it. Unless you do it just right, you should presume hackers could get anything on the disk...
2) Why make life complex with layer after layer of software? Install Ubuntu and run a standard server under a Unix OS
3) Be very careful about the rest of your attached network. Even if the PC is dedicated, unless you properly managed port forwarding, etc., ALL of your computers could be susceptible to attack.
An old friend of mine discovered, back in the Napster peer-to-peer days, that he could basically go and read EVERYTHING on the hard drives of most people who had set up Napster on their computer.
It really depends on your requirements. Will you be accessing the website externally (making it public) or locally? Running Django from your laptop can work but if you are planning to make it public, you will need an external IP to point your domain to. Unless you have a business account, ISPs usually don't give static IPs to individual customers. Ubuntu would be a wise choice and you can run conda or virtualenv easily.
VPS are quite cheap these days. You can look into AWS free tier that provides you with 500 hours/month on a micro server.
If you are planning to access your website internally then you don't need anything other than your laptop or perhaps raspberry pi. If you are trying to make it available for everyone on the external network, VPS would be the best bet.
Currently, and this changes often, I like to either setup a local development environment using virtualenv (to install dependencies) and Ngrok (to expose machine to an external address) or C9.io. If you want further info about setup, I'm happy to provide.
As already stated Ubantu is a good choice but there is also Debian. I use Debian because I started off working with a colleague who was already using it and I find it very good. I began with an old, disused desktop PC which I nuked and turned into a proper linux server. For development I didn't need a very high spec machine. (Think it has 1 GB ram) I have it set up in my flat and my domestic internet connection is fine for most of my needs. Note: It isn't necessary to have a static IP address for development, although it is preferable if you already have one. As an alternative you can use a service such as dnydns.org where you can set up virtual domain names that point to your domestic dynamic IP address. Most routers these days have facilities within them for updating services like dyndns.org with your new dynamic IP address or you can install a plug-in to your server that will do this for you. All my projects have their own virtualenvs and I have VNCServer installed so I can access my server and work from anywhere where I have an internet connection. I've been running this way for the past three years with some household name clients and haven't had any issues at all.
When it comes to production you can simply use any of the many VPS services that are out there. Amazon has already been mentioned. Someone recommended creating a droplet at DigitalOcean.com as I was wanting to host django applications and I find them to be very good and cost effective. Anyway just my 2 cents worth...hope it helps
I have a server that I'd like to use to maintain persistent connections with a set of devices, just so that they can pass simple messages back and forth. It's a trivial task, but selecting a server-side platform has been suprrisingly difficult (especially since I have no administrative privileges - it's a dedicated commercial server).
My best idea so far is to write a TCP server in Python. The Twisted platform seems suitable for the task, and has a lot of good reviews. However, my server has Python 2.7 but not Twisted, and the admins have been reluctant to install it for me.
Is there any way that I can just upload a Twisted package to the server and reference it in my libraries without installing it as a framework?
I'm not sure what you mean by "installing it as a framework". If you are using an OS X server hosting environment, then maybe you're talking about Framework with a Capital F. However, OS X server hosting isn't a very common environment so I'm guessing that's not it.
If you just want to know how to install a Python library in your home directory, then the general answer is:
$ python setup.py install --user
This Just Works™ on Python 2.7 (assuming the package uses distutils, which Twisted does, and you unpack the source .tar.gz and change your working directory to the directory that is the root of the contents of that .tar.gz), so you should be done after that.
Use virtualenv to create your private Python libraries installation.
the more I read the worse it gets... I am starting out with Python and I cannot make my
mind up on how to set up my dev environment. I want to use Python and Django to build web applications.
Ideally, I'd love to use and IDE on Win7 (which would help with tooltips and help about methods, classes, etc) and have the web app run on a virtual linux machine (need apache+mysql). I have downloaded the turnkey linux appliance for Django and it seems to work fine.
So, in the end, it is unclear to me if people here are recommending to edit my code on the same machine where the app runs. I'd prefer to code on the Win7 machine and then publish the app/files on the fly to the linux virtual box, then accessing the app via the browser.
This is the setup for my current php project at work and I think it works perfectly.
Please clarify if people normally code and run their web apps all on one machine only or not.
Thank you!
I don't know how it can be unclear to you "if people here are recommending to edit my code on the same machine where the app runs". The easy answer is no, no way, never, ever. There can be no ambiguity about that.
Edit Ah, apologies for the misunderstanding - you clarify in the comments that you're not talking about your production environment. In that case, yes it is a perfectly good idea - even preferable - to edit on the same machine as your development app is running. There's no reason not to, and it makes life a whole lot easier.
Note you shouldn't really use Apache in development: it requires a lot of configuration, and doesn't automatically reload after code changes without even more configuration. Use the development server. And in case you were concerned, all of this runs perfectly well on a Windows machine.
I agree with the comment. Personally, I use Aptana Studio (http://www.aptana.com/) alongside GIT to have local version control (integrates well into Aptana). From there on, it is easy to either deploy locally or push the changes to a remote GIT repo.
I am working on a project where I am quite comfortable running linux, virtualenv, pip, manage.py runserver, git and so on for back-end development. I work with a front-end developer who needs to collaborate remotely, currently via a Dropbox synced copy of the codebase (also in a git branch) on Windows. A development server on my side lets the developer see their changes semi-live.
Although this has served us fairly well so far, has anyone come across a similar working arrangement with a better setup for collaboration?
I'm mindful that the source control learning curve and environmental management overhead is potentially significant and somewhat unnecessary for front-end work (as long as I commit from time to time). I'm considering a VM based setup such as BitNami's DjangoStack so that the front-end dev has their own server setup, but I thought I'd ask about other experiences.
I would recommend vagrant not only for quick development setups (which it excels at), but also for sharing VM configurations as you can publish your own vagrant file which your designer uses.
It relies on VirtualBox Sun Oracle's open source hypervisor and is available for free on all major platforms.
I have been in a very similar situation before Rog, where the backend was a Ruby on Rails setup running on *nix, and the frontend guy needed windows. We initially set up a Windows-Apache-MySql+git+RoR (using Cygwin and other tools) but eventually installing our app libraries and gems became a pain on the windows setup (anytime we would introduce a new gem (or app in django terms) the setup would break on windows). In the end we finally made the front-end guy work on *nix setup.
andLinux is extremely useful in these situations, it lets your run a seamless install of linux withing a windows 2000 setup, so the front end guy can still use windows tool. It is not like a dual boot, but here both the OS are running at the same time. Have a look into it.