After a whole load of hard work I've eventually got a hello world flask app running on Windows Azure, the app is built locally and runs fine, deploying it to Azure is a nightmare though. So I've sort of got two questions here.
I can't seem to get a stack trace at all, I've tried setting things in web.config, but the documentation on how to use all this stuff is just apawling, all I can find is just literally badly written blog posts dotted around one of microsoft's millions of blogs. Which doesn't even help me to fix my problem.
The second question relates to the first one, due to some horrible debugging methods (taking my application apart and commenting things out) I feel like it could be pymongo causing this, I've built it without the C extensions and it's in my site-packages and it works on my local machine. However without a stack trace I've just no idea how to fix this without wanting to pull my hair out.
Can anyone shed some light on this? Really disappointing because the rest of azure isn't too bad, theres far better website hosting alternatives out there like heroku which are literally 10 command setups. I've been working on this all day so far..
Solved
For those who are interested I ended up solving this problem my manually adding error handling into my flask application completely bypassing the IIS settings and windows azure configs - far too complicated with no documentation at all.
from werkzeug.debug import get_current_traceback
#app.errorhandler(500)
def internal_server_error(e):
base = os.path.dirname(os.path.abspath(__file__))
f = open('%s/logs/error.log' % (base), 'a')
track = get_current_traceback(skip=1, show_hidden_frames=True, ignore_system_exceptions=False)
track.log(f)
f.close()
return 'An error has occured', 500
Related
First I'd like to say that I know similar questions about calling Python code in Flutter have been asked before, but I think this particular case has some challenges.
Some notes about the app I'm aiming for:
Basically a note taking app, records a lecture or meeting or whatever and transcribes the text for you, with a few extra features thrown in. I'd like to have all speech being processed locally both to ensure it works offline and reduce the app's dependence on cloud services.
I'm trying to use Whisper, a new speech to text software that processes everything locally, which is a necessity for my app. I know I could make a Flutter plugin but I'm not sure if that's the best route to go about this for a few reasons:
I haven't done it before, so it would be quite a time investment to do this and just hope it works out.
One of the ways I've seen of doing this involves sending data over http between Python and Flutter, but Whisper would need a continuous stream of audio to work properly which I'm not sure this approach is suited for.
I'd really like to have 1 codebase that runs on any device.
I'd be fine with the app only working on pc for now, but I'd like to also have it working on Android and maybe IOS if reasonably possible. Any other routes I can take towards development are great too but I'd really like to stick with Flutter for this app if I can.
Just found that one: https://github.com/azkadev/whisper_dart
Did not tried it until now but seems to be worth the try.
I have a recurring problem with my hadoop cluster in that occasionally a functioning code stops seeing python modules that are in the proper location. I'm looking for tips from someone who might have faced the same problem.
When I first started programming and a code stopped working I asked a question here on SO and someone told me to just go to bed and in the morning it should work, or some other "you're a dummy, you must have changed something" kind of comment.
I run the code several times, it works, I go to sleep, in the morning I try to run it again and it fails. Sometimes I kill jobs with CTRL+C, and sometimes I use CTRL+Z. But this just takes up resources and doesn't cause any other issue besides that - the code still runs. I have yet to have see this problem right after the code works. This usually happens the morning after, when I come into work after the code worked when I left 10 hours ago. Restarting the cluster typically solves the issue
I'm currently checking to see if the cluster restarts itself for some reason, or if some part of it is failing, but so far the ambari screens show everything green. I'm not sure if there is some automated maintenance or something that is known to screw things up.
Still working my way through the elephant book, sorry if this topic is clearly addressed on page XXXX, I just haven't made it to that page yet.
I looked through all the error logs, but the only meaningful thing I see is in stderr:
File "/data5/hadoop/yarn/local/usercache/melvyn/appcache/application_1470668235545_0029/container_e80_1470668235545_0029_01_000002/format_text.py", line 3, in <module>
from formatting_functions import *
ImportError: No module named formatting_functions
So we solved the problem. The issue is particular to our set up. We have all of our datanodes nfs mounted. Occasionally a node fails, and someone has to bring it back up and remount it.
Our script specifies the path to libraries like:'
pig -Dmapred.child.env="PYTHONPATH=$path_to_mnt$hdfs_library_path" ...
so pig couldn't find the libraries, because $path_to_mnt was invalid for one of the nodes.
I'm currently working on a Google App Engine (Python) project which primarily uses Google Cloud SQL (with SQL Alchemy) for back-end data storage.
Most of the time everything works perfectly well. However, occasionally "something" goes haywire and we start getting bizarre exceptions. For example:
AttributeError: 'ColumnProperty' object has no attribute 'strategy'
AttributeError: 'RelationshipProperty' object has no attribute 'strategy'
We think this might be related to the spinning up of a new GAE instance, but we can't really be sure.
With all that being said, my question is this. What are some strategies that my team and I can use to track down this issue?
Keep in mind that the application is running on Google App Engine so that might limit our options a bit.
Update: Owen Nelson's comment below is right on. We've added threading.RLock as suggested by Google. However we are still seeing this issue, but much less often.
I want to be clear, to this point we've been unable to reproduce this issue in our local environment. We are pretty sure this has something to do with dynamic instances spinning up and that isn't something that we can really do in development.
From what I can understand, your application have problems only in production mode.
Try to reproduce the bug in dev mode
The best possible solution would be to be able to reproduce that bug in development mode. To do that, you could try to run a batch of unittest with LOTS of data. (See how to do local test on appengine).
If that doesn't work...
Turn on appstats to get more information on the handler
You can turn on appstats to try and get information on which handler is currently causing the problem. Appstats normally gives you information on datastore, this is not relevant in our case, but you can get information from the requests in general (such as response time)
Identify the handler and wrap it in a beautiful try catch
Once you identify the source of the problem or from where it is raised, you can surround it with a try..catch.. With that you can get more information on the current execution Trace and hopefully solve your problem
I'm a beginner level user of Django and Python right now, and so far anything I do locally has immediately worked on my hosting once uploaded. My hosting is provided by Hostmonster.
However, I've just installed PyEnchant. All I use it for is basic spell checking and suggesting new words. Also, 'string' is always a string of words separated by '+'.
from enchant import Dict
def spellcheck(string):
spellcheck = Dict("en-GB")
suggestedword = []
for word in string.split('+'):
if len(word) > 2 and not spellcheck.check(word):
suggestedword.append(spellcheck.suggest(word)[0])
else:
suggestedword.append(word)
return suggestedword
Locally, using the Django dev server, all works fine. On my host I get:
Django Version: 1.4
Exception Type: MemoryError
Exception Location: /home/user/python/lib/python2.7/ctypes/__init__.py in _reset_cache, line 279
It seems to be throwing the error a few steps after 'from enchant import Dict'.
I'm guessing the dictionary is too large to store in temporary memory?
Any idea how to get around this? Please go easy on me if I'm either asking something very stupid, or in a very stupid way :).
If I'm leaving out any vital data, it's because I don't know it's important, so feel free to tell me what other information would help solve this (if it can be solved on a shared host).
Thanks in advance for any help!
EDIT1:
Using SSH, I can import and use PyEnchant:
>>> import enchant
>>> spellcheck = enchant.Dict("en-GB")
>>> spellcheck.suggest('nmae')
['name', 'mane']
Which makes me even more confused, as I have had no luck avoiding 'MemoryError' when I use it as above in my question.
EDIT2:
Still not able to figure this out. If I do 'import enchant' in any module, it seems to cause the MemoryError, yet I am able to use 'import enchant' via remote shell and the python interpreter.
EDIT3:
Still, after a few days of googling and trying things out, I can't get this MemoryError to go away. Has anyone seen this before with 'PyEnchant'? I'm thinking my host is perhaps not giving enough ram to load the PyEnchant import? Is there any way to change how memory is used by a module?
I have just had the same problem after moving my Django installation. The problem was httpd (Apache) access to the database. In my case it was Selinux but I assume that general UNIX type file permissions would cause a similar problem. In this instance it worked fine on the Django server but not on my local Apache when trying out a viable production setup.
Does your host use Linux?
Could you run Apache to help determine the problem?
I'm serving a Django app behind IIS 6. I'm wondering if I can restart IIS 6 within Python/Django and what one of the best ways to do would be.
Help would be great!
Besides what's already suggested, you can also use WMI via either the Win32_Service or the IIsWebService class, which inherits from it. There is a Python WMI wrapper available, which is based on pywin32.
UPDATE: A quick test of the following worked for me.
import wmi
c = wmi.WMI()
for service in c.Win32_Service(Name="W3SVC"):
result, = service.StopService()
I didn't test the next piece of code, but something like this should also work:
for service in c.IIsWebService():
result, = service.StopService()
You can see the documentation for the return values from the StopService and StartService methods.
The following post shows how to control Windows services from Python: http://fuzzytolerance.info/code/using-python-to-manage-windows-services/
You should be able that to restart the IIS web publishing service (known as 'w3svc')
I think that you can execute an iisreset via a commandline. I've never tried that with Django but it should work and be quite simple to implement.