I am building a python script which needs to run infinitely on a server. It will access a Microsoft Exchange server and read mails, process them and trigger automated voice calls.
I have successfully implemented the automated call action. Presently the script runs on my PC. I have three questions.
For running the script on a server instead of PC, does the syntax of the code other than connecting to the server needs to change? I mean, the parts where I'm reading mails and triggering calls, does that need to be changed? Or can the same script run on a server? If it does need change, can somebody please attach what changes need to be done.
Since I need to run the script on a server, and access a Microsoft Exchange server, can the script be run on the Exchange server itself? If yes, please attach helpful resources.
The script does not take any input as such, but it accesses a couple of files that need to edited manually from time to time. How should I achieve that?
The distinction between PC and Server doesn't matter. Your script will require a set of resources and may make assumptions about the OS it's running on. Those are the things that matter. As long as the required resources are there, it should run fine. For example, if your script requires Python 3.6+ to run, then you must have Python 3.6+ installed on either the Server/PC. If you are using a particular python package, then it should be installed. If you make assumptions about where files are located on disk, those paths either need to be OS independent, or match the OS of the Server/PC, and those files need to be there. But the syntax of the python shouldn't change.
If your goal is to run the python server as a service on the server, then more information about what type of server (windows/linux) is required. Assuming you are considering running it on an exchange server, I suppose it's most likely you'll want to run on Windows. This has been asked and answered here. In relation to your code, you will want to make sure your script can be handled as a library, and you won't want to call sys.exit inside your code, but should rely on exceptions to pass up errors. My preferred pattern is something like
def main(argv=None):
# parse arguments if you have them and run the script
if __name__ == '__main__':
main()
Then in your service you can import and call main(...) without running another executable.
See #1. Whether it can run on that server depends on whether all of the required resources and files are available there. There is possibly a question of whether you would WANT to run the script on your exchange server. That answer depends on the load the script takes, how busy/active your server is, Whether you want the extra software installed on your server, etc.
Your best solution here will depend on your situation. If you can login and edit the files, then maybe that's what you do. If you want to edit them on your PC and then push them up, then there are solutions for that. All depends on what makes sense for your project/situation.
Related
With this question I would like to gain some insights/verify that I'm on the right track with my thinking.
The request is as follows: I would like to create a database on a server. This database should be updated periodically by adding information that is present in a certain folder, on a different computer. Both the server and the computer will be within the same network (I may be running into some firewall issues).
So the method I am thinking of using is as follows. Create a tunnel between the two systems. I will run a script that periodically (hourly or daily) searches through the specified directory, convert the files to data and add it to the database. I am planning to use python, which I am fairly familiar with.
Note: I dont think I will be able to install python on the pc with the files.
Is this at all doable? Is my approach solid? Please let me know if additional information is required.
Create a tunnel between the two systems.
If you mean setup the firewall between the two machines to allow connection, then yeah. Just open the postgresql port. Check postgresql.conf for the port number in case it isn't the default. Also put the correct permissions in pg_hba.conf so the computer's ip can connect to it.
I will run a script that periodically (hourly or daily) searches through the specified directory, convert the files to data and add it to the database. I am planning to use python, which I am fairly familiar with.
Yeah, that's pretty standard. No problem.
Note: I dont think I will be able to install python on the pc with the files.
On Windows you can install anaconda for all users or just the current user. The latter doesn't require admin privileges, so that may help.
If you can't install python, then you can use some python tools to turn your python program into an executable that contains all the libraries, so you just have to drop that into a folder on the computer and execute it.
If you absolutely cannot install anything or execute any program, then you'll have to create a scheduled task to copy the data to a computer that has python over the network, and run the python script there, but that's extra complication.
If the source computer is automatically backed up to a server, you can also use the backup as a data source, but there will be a delay depending on how often it runs.
I want to create web form that stays on forever on a single computer. Users can come to the computer fill out the form and submit it. After submitting, it will record the responses in an excel file and send emails. The next user can then come and fill out a new form automatically. I was planning on using Flask for this task since it is simple to create, but since I am not doing this on some production server, I will just have it running locally in development on the single computer.
I have never seen anyone do something like this with Flask so I was wondering if my idea is possible or if I should avoid it. I am also new to web development so I was wondering what problems there could be with keeping a flask application stay on 24/7 on a local development computer.
Thanks
There is nothing wrong with doing this in principle however, it is likely not the best solution for the time-to-reward payoff.
First, to answer your question, this could easily be done, even for a beginner, completing this in a few hours with minimal Python and HTML experience could definitely be done. Your app could crash in the background for many reasons (running out of space, bad memory addresses, etc) but most likely you will be fine.
As for specifically building it, it is all possible, there are libraries you can use to add the results to an excel file, or you can easily just append to a CSV (which is what I would recommend). Creating and sending an email, similarly is relatively simple, but again, doing it without python would be much easier.
If you are not set on flask/python, you could check out Google Forms but if you are set on python, or want to use it as a learning experience, it can definitely be done.
Your idea is possible and while there are many ways to do this kind of thing, what you are suggesting is not necessarily to be avoided.
All apps that run on a computer over a long period of time start a process and keep it going until closed. That is essentially what you are doing.
Having done this myself (and still currently doing it) at my business, I can say that it works great.
The only caveat is that to ensure that it will always be available, you need to have the process monitored by some tool to make sure that it gets restarted if it ever closes due to a variety of reasons.
In linux, supervisor is a great tool for doing that. In windows you could register it as a service. But you could also just create an easy way to restart and make it easy for the user to do so if it is down when they need it.
Yes, this could be done. It's very similar to the applications that run on the servers in data centers.
To keep the application running forever or restarting it after your system starts you'll need to use a system manager similar to systemd in Unix. You could use NSSM - the Non-Sucking Service Manager
or Service Control to monitor your application and restart it if it crashes. This will also have to be enabled on startup.
Other than this, you could use Waitres to serve your Flask application. Waitress is a WSGI web server with which you can easily configure the number of threads and workers to enable serving multiple users at the same time.
In a production environment, it's always suggested to use a web server interface like Gunicorn or Waitress.
My client has provided me with a Python console application which performs some work and writes the result into a .txt file. My task is to write a Windows service which reads that particular .txt file and performs further actions.
I used C# on .NET to write the service. My solution contains 3 projects:
The logic layer project.
The Windows service layer project.
A test app layer project, used for debugging and other purposes.
Both the Windows service layer and test app layer are using the logic layer for core functionality. When I run the application through the test layer, everything works perfectly, but whenever I try to run the application through the service, the Python standalone application that the service launches doesn't write any output files. I could see that the Python app runs in the task manager, but there's no output anywhere. I believe the Python code is crashing but I couldn't get the exact reason.
I've tried the following ways to debug the issue:
Searched the Windows and System32 directories for any related output files, just to consider the possibility of the service having these directories as the default working directory.
Used absolute paths in the service code to make sure that the Python part is not writing output files to some unknown location.
Had the client implement passing the output directory to the Python code through command line arguments.
Wrote a mock console app in C# which writes a file, tried to call it through the service, but it worked fine and wrote the file as expected.
Suspected the standard IO could be causing the Python application to crash and thus used the standard IO in my mock program, but it worked without any issues.
Tried giving a long task to the Python code, which should've taken about 30 minutes to execute completely, but the Python script ran and closed immediately, which essentially is reliable proof of the theory that it crashes at some point.
Tried running the service with my unelevated Windows user instead of the Local System pseudouser.
Tried configuring the service to be able to interact with the desktop.
I am all out of ideas here. Any direction I should also search in?
Just a note, if that matters: I am using System.Diagnostics.Process to launch the Python script.
If it works from your test app, it sounds like a permissions issue. What security context / user is the windows service running as, and does that user have permission to write to the filesystem where you are expecting it? Have you tried using a full path to the output file to be sure where it is expected?
I'd be inclined to write a tiny python app that just saves "hello world" to a file, and get that to work from a windows service, then build up from there.
Thanks to the help from timhowarduk, I finally was able to get to the root cause of the problem. The python script was looking for a configuration file, and when it was running from the Windows Service, it was looking for that config file in System32.
All the windows services are run from System32.
The above caused the python script to search in System32 since it was running as part of the windows service. I guess I might just ask the client to edit the python script to read config from the windows service application directory.
I'm new to Python (relatively new to programing in general) and I have created a small python script that scrape some data off of a site once a week and stores it to a local database (I'm trying to do some statistical analysis on downloaded music). I've tested it on my Mac and would like to put it up onto my server (VPS with WiredTree running CentOS 5), but I have no idea where to start.
I tried Googling for it, but apparently I'm using the wrong terms as "deploying" means to create an executable file. The only thing that seems to make sense is to set it up inside Django, but I think that might be overkill. I don't know...
EDIT: More clarity
You should look into cron for this, which will allow you to schedule the execution of your Python script.
If you aren't sure how to make your Python script executable, add a shebang to the top of the script, and then add execute permissions to the script using chmod.
Copy script to server
test script manually on server
set cron, "crontab -e" to a value that will test it soon
once you've debugged issues set cron to the appropriate time.
Sounds like a job for Cron?
Cron is a scheduler that provides a way to run certain scripts (apps, etc.) at certain times.
Here is a short tutorial that explains how to set up cron.
See this for more general cron information.
Edit:
Also, since you are using CentOS: if you end up having issues with your script later on... it could partly be caused by SELinux. There are ways to disable SELinux on your server (if you have enough access permissions.) But... there are arguments against disabling SELinux, as well.
I know how to reboot machines remotely, so that's the easy part. However, the complexity of the issue is trying to setup the following. I'd like to control machines on a network for after-hours use such that when users logoff and go home, or shutdown their computers, whatever, python or some combination of python + windows could restart their machines (for cleanliness) and automatically login, running a process for the night, then in the morning, stop said process and restart the machine so the user could easily login like normal.
I've looked around, haven't had too terribly much luck, though it looks like one could do it with a changing of the registry. That sounds like a rough idea though, modifying the registry on a per-day basis. Is there an easier way?
You probably want to consider running whatever program you're considering as a Windows service, unless you absolute need a desktop. There are a couple of questions concerning that, e.g. here and here, as well as recipes on Active State. That involves no real need to start up or login to the computer.
There's also always the option of scheduled tasks and what not. That can actually be done programmatically through Python, e.g., as in this blog post.
As for powering on powered off computers, while I've never done anything with it, I know Windows supports Wake-on-LAN functionality, and there seem to be some good resources, including, again, a recipe on ActiveState.
If you need a desktop to run your program, I don't think you have any choice but to mess with the registry to permit autologins, as I don't believe the Window's GINA is scriptable in any way shape or form.
I can't think of any way to do strictly what you want off the top of my head other than the registry, at least not without even more drastic measures. But doing this registry modification isn't a big deal; just change the autologon username/password and reboot the computer. To have the computer reboot when the user logs off, give them a "logoff" option that actually reboots rather than logging off; I've seen other places do that.
(edit)FYI: for registry edits, Windows has a REG command that will be useful if you decide to go with that route.(/edit)
Also, what kind of process are you trying to run? If it's not a GUI app that needs your interaction, you don't have to go through any great pains; just run the app remotely. At my work, we use psexec to do it very simply, and I've also created C++ programs that run code remotely. It's not that difficult, the way I do it is to have C++ call the WinAPI function to remotely register a service on the remote PC and start it, the service then does whatever I want (itself, or as a staging point to launch other things), then unregisters itself. I have only used Python for simple webpage stuff, so I'm not sure what kind of support it has for accessing the DLLs required, but if it can do that, you can still use Python here.
Or even better yet, if you don't need to do this remotely but just want it done every night, you can just use the Windows scheduler to run whatever application you want run during the night. You can even do this programmatically as there are a couple Windows commands for that: one is the "at" command, and I don't recall right now what the other is but just a little Googling should find it for you.
Thanks for the responses. To be more clear on what I'm doing, I have a program that automatically starts on bootup, so getting logged in would be preferred. I'm coding a manager for a render-farm for work which will take all the machines that our guys use during the day and turn them into render servers at night (or whenever they log off for a period of time, for example).
I'm not sure if I necessarily require a GUI app, but the computer would need to boot and login to launch a server application that does the rendering, and I'm not certain if that can be done without logging in. What i'm needing to run is Autodesk's Backburner Server.exe
Maybe that can be run without needing to be logged in specifically, but I'm unfamiliar with doing things of that nature.