I'm having some problem making a python file run everytime the AWS server boots.
I am trying to run a python file to start a web server on Amazon Webservice EC2 server.
But I am limited to edit systemd folder and other folders such as init.d
Is there anything wrong?
Sorry I don't really understand EC2's OS, it seems a lot of methods are not working on it.
What I usually do via ssh to start my server is:
python hello.py
Can anyone tell me how to run this file automatically every time system reboots?
It depends on your linux OS but you are on the right track (init.d). This is exactly where you'd want to run arbitrary shell scripts on start up.
Here is a great HOWTO and explanation:
https://www.tldp.org/HOWTO/HighQuality-Apps-HOWTO/boot.html
and another stack overflow specific to running a python script:
Run Python script at startup in Ubuntu
if you want to share you linux OS I can be more specific.
EDIT: This may help, looks like they have some sort of launch wizard:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
When you launch an instance in Amazon EC2, you have the option of
passing user data to the instance that can be used to perform common
automated configuration tasks and even run scripts after the instance
starts. You can pass two types of user data to Amazon EC2: shell
scripts and cloud-init directives. You can also pass this data into
the launch wizard as plain text, as a file (this is useful for
launching instances using the command line tools), or as
base64-encoded text (for API calls).
Related
I want to know if there's a way to have windows server 2019 automatically launch django's web server. I also want the launch to be performed at startup and by SYSTEM.
I tried using batch scripts that launch manage.py from venv's python interpreter. When I launch the batch manually (i.e. double click) it works fine and dandy. But it appears that SYSTEM fails in running the script correctly when planning the task.
I made SYSTEM launch another script at startup (a simple python script that creates a txt file from within its own venv) and it works.
If the Django launch sceipt is launched by USER then it works.
The problem is with the launching of django with SYSTEM. I've also tried streamlit and the result is the same.
Do you have any Ideas?
Sample batch script:
cd path\of\managepyfile\
C:\path_to_venv\Scripts\python -m manage.py runserver
We run a similar application (not python) but an application that uses a web server.
We have it setup as a task in task scheduler that when the server starts up, it runs the powershell script that executes a command to start the web server.
Link to setup
However, you could use a web server like IIS and deploy the files to the www folder in the cdrive and run the site as an IIS service.
Setting it up on IIS was a little tricky if you've never used IIS before. Happy to help out as we have deployed our test access tool for one of our apps this way.
So there are variants of this question - but none quite hit the nail on the head.
I want to run spyder and do interactive analysis on a server. I have two servers , neither have spyder. They both have python (linux server) but I dont have sudo rights to install packages I need.
In short the use case is: open spyder on local machine. Do something (need help here) to use the servers computation power , and then return results to local machine.
Update:
I have updated python with my packages on one server. Now to figure out the kernel name and link to spyder.
Leaving previous version of question up, as that is still useful.
The docker process is a little intimidating as does paramiko. What are my options?
(Spyder maintainer here) What you need to do is to create an Spyder kernel in your remote server and connect through SSH to it. That's the only facility we provide to do what you want.
You can find the precise instructions to do that in our docs.
I did a long search for something like this in my past job, when we wanted to quickly iterate on code which had to run across many workers in a cluster. All the commercial and open source task-queue projects that I found were based on running fixed code with arbitrary inputs, rather than running arbitrary code.
I'd also be interested to see if there's something out there that I missed. But in my case, I ended up building my own solution (unfortunately not open source).
My solution was:
1) I made a Redis queue where each task consisted of a zip file with a bash setup script (for pip installs, etc), a "payload" Python script to run, and a pickle file with input data.
2) The "payload" Python script would read in the pickle file or other files contained in the zip file. It would output a file named output.zip.
3) The task worker was a Python script (running on the remote machine, listening to the Redis queue) that would would unzip the file, run the bash setup script, then run the Python script. When the script exited, the worker would upload output.zip.
There were various optimizations, like the worker wouldn't run the same bash setup script twice in a row (it remembered the SHA1 hash of the most recent setup script). So, anyway, in the worst case you could do that. It was a week or two of work to setup.
Edit:
A second (much more manual) option, if you just need to run on one remote machine, is to use sshfs to mount the remote filesystem locally, so you can quickly edit the files in Spyder. Then keep an ssh window open to the remote machine, and run Python from the command line to test-run the scripts on that machine. (That's my standard setup for developing Raspberry Pi programs.)
I'm having troubles to run a python script in DigitalOcean.
I have two doubts.
How to upload the scripy.py to DigitalOcean droplet.
How to run the script.
I'm able to access to the console, but further that I don't know what to do and i can't find any specific information on internet.
I'm running a Ubuntu 14.4 Droplet through web.
Ok first, in order to upload any file to your droplet you can user the command scp
scp foobar.txt your_username#remotehost.edu:/some/remote/directory
Here is a related question that shows you how to use scp from Windows.
Then in the console setup in the remote host check if you can run the command python. If you do not have it, just follow the steps in the documentation and you will have python running inside your remote machine.
If you put a Python script on the server and ssh in, you can run it from the command line. For instance,
python yourFantasticScript.py
If you want a level of automation to triggering the script to run, you will need to learn more about automation scheduling and server technologies.
I have coded a Python Script for Twitter Automation using Tweepy. Now, when i run on my own Linux Machine as python file.py The file runs successfully and it keeps on running because i have specified repeated Tasks inside the Script and I also don't want to stop the script either. But as it is on my Local Machine, the script might get stopped when my Internet Connection is off or at Night. So i couldn't keep running the Script Whole day on my PC..
So is there any way or website or Method where i could deploy my Script and make it Execute forever there ? I have heard about CRON JOBS before in Cpanel which can Help repeated Tasks but here in my case i want to keep running my Script on the Machine till i don't close the script .
Are their any such solutions. Because most of twitter bots i see are running forever, meaning their Script is getting executed somewhere 24x7 . This is what i want to know, How is that Task possible?
As mentioned by Jon and Vincent, it's better to run the code from a cloud service. But either way, I think what you're looking for is what to put into the terminal to run the code even after you close the terminal. This is what worked for me:
nohup python code.py &
You can add a systemd .service file, which can have the added benefit of:
logging (compressed logs at a central place, or over network to a log server)
disallowing access to /tmp and /home-directories
restarting the service if it fails
starting the service at boot
setting capabilities (ref setcap/getcap), disallowing file access if the process only needs network access, for instance
I'm looking to use a local webserver to run a series of python scripts for the user. For various unavoidable reasons, the python script must run locally, not on a server. As a result, I'll be using HTML+browser as the UI, which I'm comfortable with, for the front end.
I've been looking, therefore, for a lightweight web server that can execute python scripts, sitting in the background on a machine, ideally as a Windows service. Security and extensibility are not high priorities as it's all running internally on a small network.
Should I run a native python webserver as a Windows service (in which case, how)? Or is it just as easy to install Apache onto the user's machine and run as CGI? Since this is all local, performance is not an issue either.
Or am I missing something obvious?
Don't waste a lot of time creating Windows service.
Don't waste a lot of time on Windows Apache.
Just make a Python service that responds to HTTP requests.
Look at https://docs.python.org/2/library/basehttpserver.html
https://docs.python.org/3/library/http.server.html for version 3
Python offers an HTTP server that you can extend with your server-side methods.
Look at http://docs.python.org/library/wsgiref.html
Python offers a WSGI reference implementation that makes your server easy and standards-compliant.
Also http://fragments.turtlemeat.com/pythonwebserver.php
"I'm trying to avoid making the user run python stuff from the command prompt."
I don't see how clicking a web page is any different from clicking desktop icons.
Starting a web server based on Python is relatively easy, once you have the web server. First, build the server. Later, you can make sure the server starts. Let's look at some ways.
Your user can't use a random browser to open your local page. They need a bookmark to launch "localhost:8000/myspecialserverinsteadofthedestop/" That bookmark can be a .BAT file that (1) runs the server, (2) runs firefox with the proper initial URL.
You can put the server in the user's start-this menu.
You can make your Python program a windows "service".
Best way is to make your own local server by using command prompt.
Make a new folder say Project
Make a new folder inside project & name it as "cgi-bin"(without quotes)
Paste your .py file inside the cgi-bin folder
Open cmd and change to the directory from which you want to run the server and type "python -m CGIHTTPServer"(without quotes)
Minimize the cmd window & open your browser and type "localhost:8000/cgi-bin/yourpythonfilename.py"(without quotes).
The wasiest step would be navigate to folder where your files are located and running http.server module
cd /yourapp
python3 -m http.server
the you should see something like this in console
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
Running a native python webserver as a windows service should be a no brainer. Check out the documentation for writing windows services (win32api, ActiveState python) in python and also the documentation for subclassing BaseHttpServer and SimpleHttpServer.
BTW: I had a similar question on stackoverflow: How to stop BaseHTTPServer.serve_forever() in a BaseHTTPRequestHandler subclass?
Basically, you subclass BaseHTTPServer (you have to anyway...) and then... but just read the accepted answer - it set me on the right track!