Develop Reference

PRAW Errors updating to new Python Distro

So I am trying to create a bot that cross posts from a sub (r/pics) to (r/polpics) using a bit of code from u/GoldenSights. I upgraded to a new python distro and I get a ton of errors, I don't even know where to begin. Here is the code (formatting off, error lines bold):
Traceback (most recent call last):
File "C:\Users\tonyc\AppData\Local\Programs\Python\Python36-32\Lib\site-
packages\praw\subdump.py", line 84, in <module>
r = praw.Reddit(USERAGENT)
File "C:\Users\tonyc\AppData\Local\Programs\Python\Python36-32\lib\site-
packages\praw\reddit.py", line 150, in __init__
raise ClientException(required_message.format(attribute))
praw.exceptions.ClientException: Required configuration setting 'client_id'
missing.
This setting can be provided in a praw.ini file, as a keyword argument to the `Reddit` class constructor, or as an environment variable.
This seems to be related to USERAGENT setting. I don't think I have that configured right.
USERAGENT = ""
# This is a short description of what the bot does. For example
"/u/GoldenSights' Newsletter bot"
SUBREDDIT = "pics"
# This is the sub or list of subs to scan for new posts.
# For a single sub, use "sub1".
# For multiple subs, use "sub1+sub2+sub3+...".
# For all use "all"
KEYWORDS = ["It looks like this post is about US Politics."]
# Any comment containing these words will be saved.
KEYDOMAINS = []
# If non-empty, linkposts must have these strings in their URL
This is the error line:
print('Logging in')
r = praw.Reddit(USERAGENT) <--here, this is error line 84
r.set_oauth_app_info(APP_ID, APP_SECRET, APP_URI)
r.refresh_access_information(APP_REFRESH)
Also in Reddit.py :
raise ClientException(required_message.format(attribute)) <--- error
praw.exceptions.ClientException: Required configuration setting 'client_id'
missing.
This setting can be provided in a praw.ini file, as a keyword argument to
the `Reddit` class constructor, or as an environment variable.

Firstly, you're going to want to have your API credentials stored externally in your praw.ini file. This makes things a lot more secure, and looks like it might go some way to fixing your issue. Here's what a completed praw.ini file looks like, including the useragent, so try to replicate this.
[DEFAULT]
# A boolean to indicate whether or not to check for package updates.
check_for_updates=True
# Object to kind mappings
comment_kind=t1
message_kind=t4
redditor_kind=t2
submission_kind=t3
subreddit_kind=t5
# The URL prefix for OAuth-related requests.
oauth_url=https://oauth.reddit.com
# The URL prefix for regular requests.
reddit_url=https://www.reddit.com
# The URL prefix for short URLs.
short_url=https://redd.it
[appname]
client_id=IE*******T14_w
client_secret=SW***********************CLY
password=******************
username=appname
user_agent=web:appname:1.0.0 (by /u/username)
Let me know how things go after you sort this out.

How to tell if there's a render error when manually rendering a Django template?

I'm using the Django template system for some form emails, with the templates editable by the end user.
A template's render method returns the rendered content and silently passes over invalid variables and tags. Is there a reasonable way to tell if there was an error during rendering?
I've considering setting settings.TEMPLATE_STRING_IF_INVALID to a unique string and then testing for the presence of this string but that would affect normal template rendering, which isn't acceptable. I've scanned Django's source code in the hope there is a "render invalid variable/tag" method I can override cleanly to raise an exception, but no such luck.
Any ideas?

Sadly enough hacking around the problem like that is the best solution available. It should be noted that the variable is called TEMPLATE_STRING_IF_INVALID btw.
Alternatively I would recommend using Jinja2 together with Coffin, that makes debugging a lot easier as well and Jinja2 actually does give proper stacktraces for errors like these.

Perhaps you could write the templates in Jinja2 instead. It's a bit more powerful than Django's templating language, but can throw exceptions on non-existent variables - and the syntax is very similar.
It also provides similar render etc functions to do just what you're asking, without affecting any of the rest of your Django project.
For example:
>>> from jinja2 import Environment, StrictUndefined
>>> env = Environment(undefined=StrictUndefined)
>>> template = env.from_string('Say hello to {{name}}')
>>>
>>> template.render()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/jinja2/environment.py", line 969, in render
return self.environment.handle_exception(exc_info, True)
File "/Library/Python/2.7/site-packages/jinja2/environment.py", line 742, in handle_exception
reraise(exc_type, exc_value, tb)
File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'name' is undefined
>>>
>>> template.render({'name': 'Ben'})
u'Say hello to Ben'
>>>
>>> template.render(name='Ben')
u'Say hello to Ben'

django test file download - "ValueError: I/O operation on closed file"

I have code for a view which serves a file download, and it works fine in the browser. Now I am trying to write a test for it, using the internal django Client.get:
response = self.client.get("/compile-book/", {'id': book.id})
self.assertEqual(response.status_code, 200)
self.assertEquals(response.get('Content-Disposition'),
"attachment; filename=book.zip")
so far so good. Now I would like to test if the downloaded file is the one I expect it to download. So I start by saying:
f = cStringIO.StringIO(response.content)
Now my test runner responds with:
Traceback (most recent call last):
File ".../tests.py", line 154, in test_download
f = cStringIO.StringIO(response.content)
File "/home/epub/projects/epub-env/lib/python2.7/site-packages/django/http/response.py", line 282, in content
self._consume_content()
File "/home/epub/projects/epub-env/lib/python2.7/site-packages/django/http/response.py", line 278, in _consume_content
self.content = b''.join(self.make_bytes(e) for e in self._container)
File "/home/epub/projects/epub-env/lib/python2.7/site-packages/django/http/response.py", line 278, in <genexpr>
self.content = b''.join(self.make_bytes(e) for e in self._container)
File "/usr/lib/python2.7/wsgiref/util.py", line 30, in next
data = self.filelike.read(self.blksize)
ValueError: I/O operation on closed file
Even when I do simply: self.assertIsNotNone(response.content) I get the same ValueError
The only topic on the entire internet (including django docs) I could find about testing downloads was this stackoverflow topic: Django Unit Test for testing a file download. Trying that solution led to these results. It is old and rare enough for me to open a new question.
Anybody knows how the testing of downloads is supposed to be handled in Django? (btw, running django 1.5 on python 2.7)

This works for us. We return rest_framework.response.Response but it should work with regular Django responses, as well.
import io
response = self.client.get(download_url, {'id': archive_id})
downloaded_file = io.BytesIO(b"".join(response.streaming_content))
Note:
streaming_content is only available for StreamingHttpResponse (also Django 1.10):
https://docs.djangoproject.com/en/1.10/ref/request-response/#django.http.StreamingHttpResponse.streaming_content

I had some file download code and a corresponding test that worked with Django 1.4. The test failed when I upgraded to Django 1.5 (with the same ValueError: I/O operation on closed file error that you encountered).
I fixed it by changing my non-test code to use a StreamingHttpResponse instead of a standard HttpResponse. My test code used response.content so I first migrated to CompatibleStreamingHttpResponse, then changed my test code code to use response.streaming_content instead to allow me to drop CompatibleStreamingHttpResponse in favour of StreamingHttpResponse.

Why do I getting 'FileField' object has no attribute 'put'?

Following http://mongoengine.org/docs/v0.4/guide/gridfs.html documentation about mongoengine FileField I did the following:
In my model
files = ListField(FileField())
In my test code
# Create an entry
photo = FileField()
f = open('/home/foo/marmot.jpg', 'r')
photo.put(f, content_type='image/jpeg')
entry.files = [photo,]
Trying to follow the doc, however i get an error:
Traceback (most recent call last):
File "/home/bar/tests.py", line 76, in test_MongoDGACLogook_creation
photo.put(f, content_type='image/jpeg')
AttributeError: 'FileField' object has no attribute 'put'
Am I missing something obvious ?
Thanks

This isn't obvious at all IMO, but if you look at the Mongoengine code you'll see that the put method is defined in the GridFSProxy class, which is accessed via a descriptor in FileField (the __get__ and __set__ methods).
Looking at the code and the examples in the docs, it appears the only way to access or use a FileField is through the descriptor....so, collection.file_field.
Given all this, I don't think it's possible to have a list of file fields using the Mongoengine API as it exists now.

f = mongoengine.fields.GridFSProxy()
to_read = open('/home/.../marmot.jpg', 'r')
f.put(to_read, filename=os.path.basename(to_read.name))
to_read.close()

If you are uploading multiples files and trying to save it a ListField(FileField())
<input type='file' name='myfiles' multiple="">
files = []
for f in request.FILES.getlist('myfiles'):
mf = mongoengine.fields.GridFSProxy()
mf.put(f, filename=f.name)
files.append(mf)
entry.files = files
entry.save()

I had exactly the same problem. As suggested by #KoppeKTop on GitHub in this post, I finally extended my model (Pet) using an EmbeddedDocument like this:
class OneImage(mongoengine.EmbeddedDocument):
element = ImageField()
class Pet(mongoengine.Document):
photos = EmbeddedDocumentListField(OneImage)
# ...more fields... #
I can then add a new image using
i = OneImage()
i.element.put(form.photo.data.stream)
entry.photos.append(i)
entry.save()
This may not always be the best option, but personally I prefer it because I can work with models as usual without having to work with proxies. And I can also save other photo metadata in the future, if I need to.

How to enable i18n from within setup_app in websetup.py ? (formatted resend)

From within the setup_app function (websetup.py) of a pylons i18n application, which is making use of a db, I was trying to initiate multilingual content to be inserted into the db.
To do so the idea was something like:
#necessary imports here
def setup_app(command, conf, vars):
....
for lang in langs:
set_lang(lang)
content=model.Content()
content.content=_('content')
Session.add(content)
Session.commit()
Unfortunately it seems that it doesn't work. the set_lang code line is firing an exception as follows:
File ".. i18n/translation.py", line 179, in set_lang
translator = _get_translator(lang, **kwargs)
File ".. i18n/translation.py", line 160, in _get_translator
localedir = os.path.join(rootdir, 'i18n')
File ".. /posixpath.py", line 67, in join
elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'
Actually I'm even not sure it could be possible launching i18n mechanisms from within this setup_app function without an active request object.
Anyone has tried some trick on a similar story ?

Apologies, I'm not familiar with i18n together with Pylons...
That said, you need to track down what 'path' is, and what its relative to. The error is because path is expected to be a string, but instead is set to None... causing the exception because the code is attempting a string operation 'path.endswith()' but path is None.