Deploying a Python Script on a Server (CentOS): Where to start? - python

I'm new to Python (relatively new to programing in general) and I have created a small python script that scrape some data off of a site once a week and stores it to a local database (I'm trying to do some statistical analysis on downloaded music). I've tested it on my Mac and would like to put it up onto my server (VPS with WiredTree running CentOS 5), but I have no idea where to start.
I tried Googling for it, but apparently I'm using the wrong terms as "deploying" means to create an executable file. The only thing that seems to make sense is to set it up inside Django, but I think that might be overkill. I don't know...
EDIT: More clarity

You should look into cron for this, which will allow you to schedule the execution of your Python script.
If you aren't sure how to make your Python script executable, add a shebang to the top of the script, and then add execute permissions to the script using chmod.

Copy script to server
test script manually on server
set cron, "crontab -e" to a value that will test it soon
once you've debugged issues set cron to the appropriate time.

Sounds like a job for Cron?
Cron is a scheduler that provides a way to run certain scripts (apps, etc.) at certain times.
Here is a short tutorial that explains how to set up cron.
See this for more general cron information.
Edit:
Also, since you are using CentOS: if you end up having issues with your script later on... it could partly be caused by SELinux. There are ways to disable SELinux on your server (if you have enough access permissions.) But... there are arguments against disabling SELinux, as well.

Related

Creating WebUI for sysadmin scripts

Yes, my question may sound very vague, incomplete. Beginner to webui developement.
Looking for An interactive WebUI for SysAdmin tasks:
What I have now?
a lot of Ansible tasks, playbooks, bash scripts, python scripts for sysadmin tasks.
Some of them can be stored locally(in laptop or in any location)
Some of them can be placed only where it is kept and need to run from there.
What I am trying to achieve?
I would like to add a WebUI application to support these scripts. Don't want to edit the scripts and provide the values everytime through SSH and that usual execution like ./script.sh & python mypy.py.
Once of the Example task:
Have backups stored in S3 and onpremise locations. Importantly, Database and some miscellaneous things. Currently I am running some bash scripts, python scripts to restore them. (need to modify some variable depends on the selected backup)
currently it shows what we have in S3 and we need to choose a backup date and list of archives and modify the restore script.
So I was thinking, why not read from s3 and other backups locations, list items as checkboxes, execute will run the current scripts and everyone is happy.
while googling currently available choices,
https://github.com/bugy/script-server >> but this application couldn't parse from my output and use it somewhere else.
So, what should I be learning, what all things I will require to implement this solution (pythonic way preferred)? Maybe this could be a big ask.
I was thinking about Django or any php based framework?
I would like that to be a easy development. :D
Removing Flask, Django from my list.
Going ahead with python streamlit. Meet the requirements and easy to implement with existing python scripts.

Running python script infinitely on a server

I am building a python script which needs to run infinitely on a server. It will access a Microsoft Exchange server and read mails, process them and trigger automated voice calls.
I have successfully implemented the automated call action. Presently the script runs on my PC. I have three questions.
For running the script on a server instead of PC, does the syntax of the code other than connecting to the server needs to change? I mean, the parts where I'm reading mails and triggering calls, does that need to be changed? Or can the same script run on a server? If it does need change, can somebody please attach what changes need to be done.
Since I need to run the script on a server, and access a Microsoft Exchange server, can the script be run on the Exchange server itself? If yes, please attach helpful resources.
The script does not take any input as such, but it accesses a couple of files that need to edited manually from time to time. How should I achieve that?
The distinction between PC and Server doesn't matter. Your script will require a set of resources and may make assumptions about the OS it's running on. Those are the things that matter. As long as the required resources are there, it should run fine. For example, if your script requires Python 3.6+ to run, then you must have Python 3.6+ installed on either the Server/PC. If you are using a particular python package, then it should be installed. If you make assumptions about where files are located on disk, those paths either need to be OS independent, or match the OS of the Server/PC, and those files need to be there. But the syntax of the python shouldn't change.
If your goal is to run the python server as a service on the server, then more information about what type of server (windows/linux) is required. Assuming you are considering running it on an exchange server, I suppose it's most likely you'll want to run on Windows. This has been asked and answered here. In relation to your code, you will want to make sure your script can be handled as a library, and you won't want to call sys.exit inside your code, but should rely on exceptions to pass up errors. My preferred pattern is something like
def main(argv=None):
# parse arguments if you have them and run the script
if __name__ == '__main__':
main()
Then in your service you can import and call main(...) without running another executable.
See #1. Whether it can run on that server depends on whether all of the required resources and files are available there. There is possibly a question of whether you would WANT to run the script on your exchange server. That answer depends on the load the script takes, how busy/active your server is, Whether you want the extra software installed on your server, etc.
Your best solution here will depend on your situation. If you can login and edit the files, then maybe that's what you do. If you want to edit them on your PC and then push them up, then there are solutions for that. All depends on what makes sense for your project/situation.

Can I use Apache, cgi-bin, and Python?

My question is more theoretical than anything.
What I have:
Linux machine
A python script that runs an entire suite of tests.
The same script prints out all of the information to the terminal.
Currently, the way we run our software suite is via SSH to the machine, running python3 MyScript.py and then checking the output.
My goal is to create an interface that would work on a Linux/Windows machine and tablet, and would be able to execute the script, as well as visually display progress of it.
I figured, the best approach (also possibly the easiest?) is to use a web server. Web Interface is fairly universal regardless of platform it is being accessed from. And the backend is as simple as apt get install apache2 and voila! Done!
However, I have never had experience setting up cgi-bin, etc. My question is: is it actually feasible? Am I able to have an HTML button that when clicked will execute MyScript.py server-side and perhaps even parse through whatever the script is outputting to the terminal. That way I would know exactly what the script is doing, and its progress.
Any help regarding this approach (or technologies I can use) is greatly appreciated!

Deploying a Python Script to Windows and Linux

I have a python server that I need to run in both a Linux and Windows environment, and my question is about deployment. What is the best method for deploying the solution instead of just double clicking on the file and running it?
Since I use the server_forever() on the server, I can just run the script from command line, but this keeps the python window open. If I log off the machine, naturally the process will stop. So what is the best method for deploying a python script that needs to keep running if the user is logged in or off a machine.
Since I am going to be using multiple environment, Linux and Windows, can you please be specific in what OS you are talking about?
For windows, I was thinking of running the script 'At Startup' using the Windows scheduler. But I wanted to see if anyone had a better option. For linux, I really don't know what to create. I am assuming a CRON job?
Deployment does refer to coding, so using serve_forever() on a multiprocessing job manager keeps the python window open upon execution. Is there a way to hide this window through code? Would you recommend using a conversion tool like py2exe instead?
This is the subject matter of a whole library of books, so I will just give an introduction here :-)
You can basically start scripts directly and then have multiple options to do this in a way that they keep running in the background.
If you have certain functionality that needs to run on regular moments, you would do this by scheduling it:
Windows: Windows Scheduler or specific scheduling tools
Linux: Cron
If your problem is that you want to start a script without it closing on you while SSH'ing into Linux, you want to look into the "screen" or "tmux" tools.
If you want to have it started automatically this could be done by using the "At Startup" as you point out and Linux has similar functionalities, but the preferred and more robust way would be to set up a service that is better integrated with the OS.
Windows: Windows Service
Linux: Daemon
Even more capabilities can be yielded by using an application server such a Django
Tomcat (see comment) is an option, but definitely not the standard one; you'll have a hard time finding support both from Tomcat people running Python or Python people running their stuff on Tomcat. That being said, I imagine you could enable CGI and have it run a Python command with your script.
Yet, instead of just starting a Python script I would strongly encourage you to have a look at different Python options that are probably available for your specific use case. From lightweight web solutions like Flask over a versatile networking engine like Twisted to a full blown web framework like Django.
They all have rather well-thought-out deployment solutions available. Look up WSGI for more background.

Reboot windows machines at a certain time of day and automatically login with Python

I know how to reboot machines remotely, so that's the easy part. However, the complexity of the issue is trying to setup the following. I'd like to control machines on a network for after-hours use such that when users logoff and go home, or shutdown their computers, whatever, python or some combination of python + windows could restart their machines (for cleanliness) and automatically login, running a process for the night, then in the morning, stop said process and restart the machine so the user could easily login like normal.
I've looked around, haven't had too terribly much luck, though it looks like one could do it with a changing of the registry. That sounds like a rough idea though, modifying the registry on a per-day basis. Is there an easier way?
You probably want to consider running whatever program you're considering as a Windows service, unless you absolute need a desktop. There are a couple of questions concerning that, e.g. here and here, as well as recipes on Active State. That involves no real need to start up or login to the computer.
There's also always the option of scheduled tasks and what not. That can actually be done programmatically through Python, e.g., as in this blog post.
As for powering on powered off computers, while I've never done anything with it, I know Windows supports Wake-on-LAN functionality, and there seem to be some good resources, including, again, a recipe on ActiveState.
If you need a desktop to run your program, I don't think you have any choice but to mess with the registry to permit autologins, as I don't believe the Window's GINA is scriptable in any way shape or form.
I can't think of any way to do strictly what you want off the top of my head other than the registry, at least not without even more drastic measures. But doing this registry modification isn't a big deal; just change the autologon username/password and reboot the computer. To have the computer reboot when the user logs off, give them a "logoff" option that actually reboots rather than logging off; I've seen other places do that.
(edit)FYI: for registry edits, Windows has a REG command that will be useful if you decide to go with that route.(/edit)
Also, what kind of process are you trying to run? If it's not a GUI app that needs your interaction, you don't have to go through any great pains; just run the app remotely. At my work, we use psexec to do it very simply, and I've also created C++ programs that run code remotely. It's not that difficult, the way I do it is to have C++ call the WinAPI function to remotely register a service on the remote PC and start it, the service then does whatever I want (itself, or as a staging point to launch other things), then unregisters itself. I have only used Python for simple webpage stuff, so I'm not sure what kind of support it has for accessing the DLLs required, but if it can do that, you can still use Python here.
Or even better yet, if you don't need to do this remotely but just want it done every night, you can just use the Windows scheduler to run whatever application you want run during the night. You can even do this programmatically as there are a couple Windows commands for that: one is the "at" command, and I don't recall right now what the other is but just a little Googling should find it for you.
Thanks for the responses. To be more clear on what I'm doing, I have a program that automatically starts on bootup, so getting logged in would be preferred. I'm coding a manager for a render-farm for work which will take all the machines that our guys use during the day and turn them into render servers at night (or whenever they log off for a period of time, for example).
I'm not sure if I necessarily require a GUI app, but the computer would need to boot and login to launch a server application that does the rendering, and I'm not certain if that can be done without logging in. What i'm needing to run is Autodesk's Backburner Server.exe
Maybe that can be run without needing to be logged in specifically, but I'm unfamiliar with doing things of that nature.

Categories

Resources