I made a new anaconda installation and happily fired up a Flask program but was promptly overwhelmed. Hundreds and thousands of "detected change in .... reloading". That was three days ago. by yesterday it seemed to have calmed down a bit and would believe it was detecting a change about once every minute. It has kept up that rate since. What I assume is happening is that it is slowly but surely making some kind of map of my python environment.
Does this ever end? Is there anything I can do to speed it up?
Thanks, Tunneller
Try using debug=False in the run statement. For example:
app.run(host='0.0.0.0', debug=False)
This will stop the flask app from detecting changes to files it is using and updating itself.
Related
Okay, so basically I am creating a website. The data I need to display on this website is delivered twice daily, where I need to read the delivered data from a file and store this new data in the database (instead of the old data).
I have created the python functions to do this. However, I would like to know, what would be the best way to run this script, while my flask application is running? This may be a very simple answer, but I have seen some answers saying to incorporate the script into the website design (however these answers didn't explain how), and others saying to run it separately. The script needs to run automatically throughout the day with no monitoring or input from me.
TIA
Generally it's a really bad idea to put a webserver to handle such tasks, that is the flask application in your case. There are many reasons for it so just to name a few:
Python's Achilles heel - GIL.
Sharing system resources of the application between users and other operations.
Crashes - it happens, it could be unlikely but it does. And if you are not careful, the web application goes down along with it.
So with that in mind I'd advise you to ditch this idea and use crontabs. Basically write a script that does whatever transformations or operations it needs to do and create a cron job at a desired time.
I just recently learned Python and now I thought about making a very simple online game for fun. I don't know what the game should be but I thought I should be able to somehow communicate with others playing the game.
I figured I could save actions I'm doing in the game in a file on a Webserver. The game should then run some function for the other players if the file gets changed. Now my question: Is there any way to watch that file for changes? I tried watchdog but as far as I found it only works on local files. I could probably use a loop with urllib2 checking the content but I'm afraid that's not very elegant or fast.
Im using Python 2.7.13 on a Windows 10 computer.
Thanks in advance for your answer :)
(disclaimer, I agree with #roganjosh in the comments, I don't think this is the right way to approach your problem, but just to answer your question)
In order to monitor changes in a webserver, unless they provide some other API, usually you do implement a loop but instead of getting the whole file all the time, you only send a "HEAD" request. If you see updates in the file then you fetch it.
See this answer for more information: How do you send a HEAD HTTP request in Python 2?
I'm a fairly new to web development so this might actually be normal behavior - but when I make logic changes in my views, it can take about an hour for those changes to show up on my production site.
The changes are instant if I fire up the localhost. Server is Windows IIS 7.5. HTML, CSS, and JS changes show up instantly, it's the code in the view that takes a while to filter through. Any ideas on what is causing this and how to fix it?
Have you tried doing a manual reboot of the application pool where the site is sitting in IIS? Documentation might not be exact for the version but it should explain it well enough to give you an idea about what's going:
https://technet.microsoft.com/en-us/library/cc753179(v=ws.10).aspx
Basically, if you have the application pool recycle every 3 hours, when you make a change it could take up to 3 hours for the change to take effect. You also don't want it recycling every 5 minutes either. But you can do a manual recycle if you really want to see your changes.
I have a recurring problem with my hadoop cluster in that occasionally a functioning code stops seeing python modules that are in the proper location. I'm looking for tips from someone who might have faced the same problem.
When I first started programming and a code stopped working I asked a question here on SO and someone told me to just go to bed and in the morning it should work, or some other "you're a dummy, you must have changed something" kind of comment.
I run the code several times, it works, I go to sleep, in the morning I try to run it again and it fails. Sometimes I kill jobs with CTRL+C, and sometimes I use CTRL+Z. But this just takes up resources and doesn't cause any other issue besides that - the code still runs. I have yet to have see this problem right after the code works. This usually happens the morning after, when I come into work after the code worked when I left 10 hours ago. Restarting the cluster typically solves the issue
I'm currently checking to see if the cluster restarts itself for some reason, or if some part of it is failing, but so far the ambari screens show everything green. I'm not sure if there is some automated maintenance or something that is known to screw things up.
Still working my way through the elephant book, sorry if this topic is clearly addressed on page XXXX, I just haven't made it to that page yet.
I looked through all the error logs, but the only meaningful thing I see is in stderr:
File "/data5/hadoop/yarn/local/usercache/melvyn/appcache/application_1470668235545_0029/container_e80_1470668235545_0029_01_000002/format_text.py", line 3, in <module>
from formatting_functions import *
ImportError: No module named formatting_functions
So we solved the problem. The issue is particular to our set up. We have all of our datanodes nfs mounted. Occasionally a node fails, and someone has to bring it back up and remount it.
Our script specifies the path to libraries like:'
pig -Dmapred.child.env="PYTHONPATH=$path_to_mnt$hdfs_library_path" ...
so pig couldn't find the libraries, because $path_to_mnt was invalid for one of the nodes.
After a whole load of hard work I've eventually got a hello world flask app running on Windows Azure, the app is built locally and runs fine, deploying it to Azure is a nightmare though. So I've sort of got two questions here.
I can't seem to get a stack trace at all, I've tried setting things in web.config, but the documentation on how to use all this stuff is just apawling, all I can find is just literally badly written blog posts dotted around one of microsoft's millions of blogs. Which doesn't even help me to fix my problem.
The second question relates to the first one, due to some horrible debugging methods (taking my application apart and commenting things out) I feel like it could be pymongo causing this, I've built it without the C extensions and it's in my site-packages and it works on my local machine. However without a stack trace I've just no idea how to fix this without wanting to pull my hair out.
Can anyone shed some light on this? Really disappointing because the rest of azure isn't too bad, theres far better website hosting alternatives out there like heroku which are literally 10 command setups. I've been working on this all day so far..
Solved
For those who are interested I ended up solving this problem my manually adding error handling into my flask application completely bypassing the IIS settings and windows azure configs - far too complicated with no documentation at all.
from werkzeug.debug import get_current_traceback
#app.errorhandler(500)
def internal_server_error(e):
base = os.path.dirname(os.path.abspath(__file__))
f = open('%s/logs/error.log' % (base), 'a')
track = get_current_traceback(skip=1, show_hidden_frames=True, ignore_system_exceptions=False)
track.log(f)
f.close()
return 'An error has occured', 500