I have a config.ini file that looks like this
[REDDIT]
client_id = 'myclientid23jd934g'
client_secret = 'myclientsecretjf30gj5g'
password = 'mypassword'
user_agent = 'myuseragent'
username = 'myusername'
When I try to use reddit's API praw like this:
import configparser
import praw
class redditImageScraper:
def __init__(self, sub, limit):
config = configparser.ConfigParser()
config.read('config.ini')
self.sub = sub
self.limit = limit
self.reddit = praw.Reddit(client_id=config.get('REDDIT','client_id'),
client_secret=config.get('REDDIT','client_secret'),
password=config.get('REDDIT','password'),
user_agent=config.get('REDDIT','user_agent'),
username=config.get('REDDIT','username'))
def get_content(self):
submissions = self.reddit.subreddit(self.sub).hot(limit=self.limit)
for submission in submissions:
print(submission.id)
def main():
scraper = redditImageScraper('aww', 25)
scraper.get_content()
if __name__ == '__main__':
main()
I get this traceback
Traceback (most recent call last):
File "config.py", line 30, in <module>
main()
File "config.py", line 27, in main
scraper.get_content()
File "config.py", line 22, in get_content
for submission in submissions:
File "C:\Users\Evan\Anaconda3\lib\site-packages\praw\models\listing\generator.py", line 61, in __next__
self._next_batch()
File "C:\Users\Evan\Anaconda3\lib\site-packages\praw\models\listing\generator.py", line 71, in _next_batch
self._listing = self._reddit.get(self.url, params=self.params)
File "C:\Users\Evan\Anaconda3\lib\site-packages\praw\reddit.py", line 454, in get
data = self.request("GET", path, params=params)
File "C:\Users\Evan\Anaconda3\lib\site-packages\praw\reddit.py", line 627, in request
method, path, data=data, files=files, params=params
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\sessions.py", line 185, in request
params=params, url=url)
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\sessions.py", line 116, in _request_with_retries
data, files, json, method, params, retries, url)
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\sessions.py", line 101, in _make_request
params=params)
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\rate_limit.py", line 35, in call
kwargs['headers'] = set_header_callback()
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\sessions.py", line 145, in _set_header_callback
self._authorizer.refresh()
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\auth.py", line 328, in refresh
password=self._password)
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\auth.py", line 138, in _request_token
response = self._authenticator._post(url, **data)
File "C:\Users\Evan\Anaconda3\lib\site-packages\prawcore\auth.py", line 31, in _post
raise ResponseException(response)
prawcore.exceptions.ResponseException: received 401 HTTP response
However when I manually insert the credentials, my code runs exactly as expected. Also, if I run the line
print(config.get('REDDIT', 'client_id'))
I get the output 'myclientid23jd934g' as expected.
Is there some reason that praw won't allow me to pass my credentials using configparser?
Double check what your inputs to praw.Reddit are:
kwargs = dict(client_id=config.get('REDDIT','client_id'),
client_secret=config.get('REDDIT','client_secret'),
password=config.get('REDDIT','password'),
user_agent=config.get('REDDIT','user_agent'),
username=config.get('REDDIT','username')))
print(kwargs)
praw.Reddit(**kwargs)
You're overcomplicating configuration here — PRAW will take care of this for you.
If you rename config.ini to praw.ini, you can replace your whole initialization with just
self.reddit = praw.Reddit('REDDIT')
This is because PRAW will look for a praw.ini file and parse it for you. If you want to give the section a more descriptive name, make sure to update it in the praw.ini as well as in the single parameter passed to Reddit (which specifies the section of the file to use).
See https://praw.readthedocs.io/en/latest/getting_started/configuration/prawini.html.
As this page notes, values like username and password should not have quotation marks around them. For example,
password=mypassword
is correct, but
password="mypassword"
is incorrect.
Related
So I'm trying to program a Reddit reply bot to simply moderating and i got pretty far into it but then when testing the code python gave me a long error that I don't understand. I haven't tried fixing it much because my skill on python is very limited so I have no idea what to do.
import praw
userAgent = 'Recomend Bot 0.1'
cID = 'rz8Gh2k8RS-NRA'
cSC= '9FR8Balfkd0OcgiKVosMSqAP2YM'
userN = ''
userP =''
numFound = 0
reddit = praw.Reddit(user_agent=userAgent, client_id=cID, client_secret=cSC, username=userN, password=userP)
subreddit = reddit.subreddit('empfehlen_testen')
bot_phrase = 'Test Reply 177013'
keywords = {'test', 'Test', 'recomendation'}
for submission in subreddit.new(limit=10):
n_title = submission.title.lower()
for i in keywords:
if i in n_title:
numFound = numFound + 1
print('Bot replying to: ')
print("Title: ", submission.title)
print("Text: ", submission.selftext)
print("Score: ", submission.score)
print("---------------------------------")
print('Bot saying: ', bot_phrase)
print()
submission.reply(bot_phrase)
if numFound == 0:
print()
print("Sorry, didn't find any posts with those keywords, try again!")
#credit for code goes to Phrynk for code all I did was get it to work on my coumputer
That got working somehow and then this error message popped up
Traceback (most recent call last):
File "C:\Users\Dillon\Desktop\RedditBot\reddit_bot.py", line 26, in <module>
for submission in subreddit.new(limit=10): #this views the top 10 posts in that subbreddit
File "C:\Users\Dillon\Desktop\RedditBot\praw\models\listing\generator.py", line 52, in __next__
self._next_batch()
File "C:\Users\Dillon\Desktop\RedditBot\praw\models\listing\generator.py", line 62, in _next_batch
self._listing = self._reddit.get(self.url, params=self.params)
File "C:\Users\Dillon\Desktop\RedditBot\praw\reddit.py", line 446, in get
data = self.request("GET", path, params=params)
File "C:\Users\Dillon\Desktop\RedditBot\praw\reddit.py", line 581, in request
method, path, data=data, files=files, params=params
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\sessions.py", line 185, in request
params=params, url=url)
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\sessions.py", line 116, in _request_with_retries
data, files, json, method, params, retries, url)
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\sessions.py", line 101, in _make_request
params=params)
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\rate_limit.py", line 35, in call
kwargs['headers'] = set_header_callback()
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\sessions.py", line 145, in _set_header_callback
self._authorizer.refresh()
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\auth.py", line 328, in refresh
password=self._password)
File "C:\Users\Dillon\AppData\Local\Programs\Python\Python37-32\lib\site-packages\prawcore\auth.py", line 142, in _request_token
payload.get('error_description'))
prawcore.exceptions.OAuthException: invalid_grant error processing request
prawcore.exceptions.OAuthException: invalid_grant error processing request
means there was a problem authenticating the user.
Remember that the username is your reddit's account name, not the bot's name.
I'm struggling with getting for example line info via getLine using pattern instead of uuid. According to getLine AXL schema for CUCM 11.5 it should be possible to choose either uuid or pattern as lookup criteria. The python (2.7) module that I'm using is suds-jurko 0.6.
Base code is as follows:
from suds.client import Client
import ssl
wsdl = 'file:///C:/Users/xyz/Documents/axlplugin/schema/current/AXLAPI.wsdl'
location = 'https://10.10.20.1/axl/'
username = '***'
password = '***'
ssl._create_default_https_context = ssl._create_unverified_context
client = Client(wsdl, location=location, username=username, password=password)
First of all, the proof that the pattern I want to use as a lookup string for getLine actually exists:
>>> line2 = client.service.listLine({'pattern': '1018'}, returnedTags={'description': ''})
>>> line2
(reply){
return =
(return){
line[] =
(LLine){
_uuid = "{1EC56035-6B5D-283A-4DF0-EFEFA01FCEFF}"
description = None
},
}
}
Then, I try getLine using uuid which works well:
>>> line5 = client.service.getLine('1EC56035-6B5D-283A-4DF0-EFEFA01FCEFF')
>>> line5
(reply){
return =
(return){
line =
(RLine){
_uuid = "{1EC56035-6B5D-283A-4DF0-EFEFA01FCEFF}"
pattern = "1018"
description = None
usage = "Device"
routePartitionName = ""
aarNeighborhoodName = ""
----Rest omitted for brevity---
Now, if I try to use pattern, not uuid, I have following error:
line6 = client.service.getLine({'pattern': '1018'}, {'routePartitionName': ''})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\suds\client.py", line 521, in __call__
return client.invoke(args, kwargs)
File "C:\Python27\lib\site-packages\suds\client.py", line 576, in invoke
soapenv = binding.get_message(self.method, args, kwargs)
File "C:\Python27\lib\site-packages\suds\bindings\binding.py", line 109, in get_message
content = self.bodycontent(method, args, kwargs)
File "C:\Python27\lib\site-packages\suds\bindings\document.py", line 95, in bodycontent
add_param, self.options().extraArgumentErrors)
File "C:\Python27\lib\site-packages\suds\argparser.py", line 83, in parse_args
return arg_parser(args, kwargs, extra_parameter_errors)
File "C:\Python27\lib\site-packages\suds\argparser.py", line 108, in __call__
self.__process_parameters()
File "C:\Python27\lib\site-packages\suds\argparser.py", line 299, in __process_parameters
self.__process_parameter(*pdef)
File "C:\Python27\lib\site-packages\suds\argparser.py", line 294, in __process_parameter
self.__in_choice_context(), value)
File "C:\Python27\lib\site-packages\suds\bindings\document.py", line 86, in add_param
p = self.mkparam(method, pdef, value)
File "C:\Python27\lib\site-packages\suds\bindings\document.py", line 130, in mkparam
return Binding.mkparam(self, method, pdef, object)
File "C:\Python27\lib\site-packages\suds\bindings\binding.py", line 225, in mkparam
return marshaller.process(content)
File "C:\Python27\lib\site-packages\suds\mx\core.py", line 59, in process
self.append(document, content)
File "C:\Python27\lib\site-packages\suds\mx\core.py", line 72, in append
self.appender.append(parent, content)
File "C:\Python27\lib\site-packages\suds\mx\appender.py", line 88, in append
appender.append(parent, content)
File "C:\Python27\lib\site-packages\suds\mx\appender.py", line 229, in append
Appender.append(self, child, cont)
File "C:\Python27\lib\site-packages\suds\mx\appender.py", line 168, in append
self.marshaller.append(parent, content)
File "C:\Python27\lib\site-packages\suds\mx\core.py", line 71, in append
if self.start(content):
File "C:\Python27\lib\site-packages\suds\mx\literal.py", line 86, in start
raise TypeNotFound(content.tag)
suds.TypeNotFound: Type not found: 'pattern'
Trying different syntax also gives error:
line6 = client.service.getLine('1018')
No handlers could be found for logger "suds.client"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\suds\client.py", line 521, in __call__
return client.invoke(args, kwargs)
File "C:\Python27\lib\site-packages\suds\client.py", line 581, in invoke
result = self.send(soapenv)
File "C:\Python27\lib\site-packages\suds\client.py", line 619, in send
description=tostr(e), original_soapenv=original_soapenv)
File "C:\Python27\lib\site-packages\suds\client.py", line 670, in process_reply
raise WebFault(fault, replyroot)
suds.WebFault: Server raised fault: 'Item not valid: The specified Line was not found'
Am I using incorrect syntax or it's actually not possible to use pattern however schema file states differently?
Additionally, I'm wondering how to extract only uuid value from for example listLine method, is it possible via python axl?
In Suds, if memory serves me correctly, you can use keyword arguments for get and update requests.
For example:
resp = client.service.getLine(routePartitionName='PT-ONCLUSTER', pattern='\+49301234567')
if that doesn't work, try:
resp = client.service.getLine({'routePartitionName': 'PT-ONCLUSTER', 'pattern': '\+49301234567'})
Or better yet, check out python-zeep instead. It's much faster and better maintained than suds. Some example code with Zeep:
# -*- coding: utf-8 -*-
from zeep import Client
from zeep.cache import SqliteCache
from zeep.transports import Transport
from zeep.exceptions import Fault
from zeep.plugins import HistoryPlugin
from requests import Session
from requests.auth import HTTPBasicAuth
from urllib3 import disable_warnings
from urllib3.exceptions import InsecureRequestWarning
from lxml import etree
disable_warnings(InsecureRequestWarning)
username = 'admin'
password = 'password'
# If you're not disabling SSL verification, host should be the FQDN of the server rather than IP
host = '10.1.1.1'
wsdl = 'file://C:/path/to/wsdl/AXLAPI.wsdl'
location = 'https://{host}:8443/axl/'.format(host=host)
binding = "{http://www.cisco.com/AXLAPIService/}AXLAPIBinding"
# Create a custom session to disable Certificate verification.
# In production you shouldn't do this,
# but for testing it saves having to have the certificate in the trusted store.
session = Session()
session.verify = False
session.auth = HTTPBasicAuth(username, password)
transport = Transport(cache=SqliteCache(), session=session, timeout=20)
history = HistoryPlugin()
client = Client(wsdl=wsdl, transport=transport, plugins=[history])
service = client.create_service(binding, location)
def show_history():
for item in [history.last_sent, history.last_received]:
print(etree.tostring(item["envelope"], encoding="unicode", pretty_print=True))
try:
resp = service.getLine(pattern='1018', routePartitionName='')
except Fault:
show_history()
I use reddit API praw and psraw to extract comments from a subreddit, however, I got two errors today after running a few loops:
JSON object decoded error or empty -> ValueError, even I catch exception in my code, still doesnt work.
http request
example:
Traceback (most recent call last):
File "C:/Users/.../subreddit psraw.py", line 20, in <module>
for comment in submission.comments:
File "C:\Python27\lib\site-packages\praw\models\reddit\base.py", line 31, in __getattr__
self._fetch()
File "C:\Python27\lib\site-packages\praw\models\reddit\submission.py", line 142, in _fetch
'sort': self.comment_sort})
File "C:\Python27\lib\site-packages\praw\reddit.py", line 367, in get
data = self.request('GET', path, params=params)
File "C:\Python27\lib\site-packages\praw\reddit.py", line 451, in request
params=params)
File "C:\Python27\lib\site-packages\prawcore\sessions.py", line 174, in request
params=params, url=url)
File "C:\Python27\lib\site-packages\prawcore\sessions.py", line 108, in _request_with_retries
data, files, json, method, params, retries, url)
File "C:\Python27\lib\site-packages\prawcore\sessions.py", line 93, in _make_request
params=params)
File "C:\Python27\lib\site-packages\prawcore\rate_limit.py", line 33, in call
response = request_function(*args, **kwargs)
File "C:\Python27\lib\site-packages\prawcore\requestor.py", line 49, in request
raise RequestException(exc, args, kwargs)
prawcore.exceptions.RequestException: error with request
HTTPSConnectionPool(host='oauth.reddit.com', port=443): Read timed out. (read timeout=16.0)
Since a subreddit contains 10k+ comments, is there a way to solve such issue? is it because reddit website has some problems today?
My code:
import praw, datetime, os, psraw
reddit = praw.Reddit('bot1')
subreddit = reddit.subreddit('example')
for submission in psraw.submission_search(reddit, subreddit='example', limit=1000000):
try:
#get comments
for comment in submission.comments:
subid = submission.id
comid = comment.id
com_body = comment.body.encode('utf-8').replace("\n", " ")
com_date = datetime.datetime.utcfromtimestamp(comment.created_utc)
string_com = '"{0}", "{1}", "{2}"\n'
formatted_string_com = string_com.format(comid, com_body, com_date)
indexFile_comment = open('path' + subid + '.txt', 'a+')
indexFile_comment.write(formatted_string_com)
except ValueError:
print ("error")
pass
continue
except AttributeError:
print ("error")
pass
continue
I'm trying to list the projects (but the same thing occurs whether I try to do that, or attempt to list the users, so let's just generalise the error to any API call). Every time I do this, I get the following HTTP 401 code:
File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/cherrypy/lib/jsontools.py", line 61, in json_handler
value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "/var/www/frontend/controllers/api/user.py", line 63, in PUT
print keystoneClient.projects.list()
File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner
return wrapped(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneclient/v3/projects.py", line 107, in list
**kwargs)
File "/usr/lib/python2.7/site-packages/keystoneclient/base.py", line 75, in func
return f(*args, **new_kwargs)
File "/usr/lib/python2.7/site-packages/keystoneclient/base.py", line 383, in list
self.collection_key)
File "/usr/lib/python2.7/site-packages/keystoneclient/base.py", line 124, in _list
resp, body = self.client.get(url, **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 173, in get
return self.request(url, 'GET', **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 331, in request
resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 98, in request
return self.session.request(url, method, **kwargs)
File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner
return wrapped(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 387, in request
auth_headers = self.get_auth_headers(auth)
File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 647, in get_auth_headers
return auth.get_headers(self, **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/plugin.py", line 84, in get_headers
token = self.get_token(session)
File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 90, in get_token
return self.get_access(session).auth_token
File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 136, in get_access
self.auth_ref = self.get_auth_ref(session)
File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 167, in get_auth_ref
authenticated=False, log=False, **rkwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 595, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner
return wrapped(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 484, in request
raise exceptions.from_response(resp, method, url)
Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-39d02130-6f47-4cae-bc30-0b645296752e)
Code:
import cherrypy
import ldap
from keystoneauth1 import loading
from keystoneauth1 import session as session
from keystoneclient.v3 import client as client
from keystoneauth1.identity import v3
import json
# Storing username and password in a cherrypy session
cherrypy.session['username'] = data.get("username")
cherrypy.session['password'] = data.get("password").replace(" ","%20")
# Setting up KeyStone Client
auth = v3.Password(
auth_url = KEYSTONE_URL,
username = cherrypy.session['username'],
password = cherrypy.session['password'],
user_domain_name="default",
domain_name = "default"
)
sess = session.Session(auth=auth, verify='/etc/ssl/certs/ca-bundle.crt')
keystoneClient = client.Client(session=sess)
# Getting list of projects
print keystoneClient.projects.list()
This occurs whilst using a normal user's credentials. If I login using admin credentials, there is no error and the projects are listed. I would like to know:
Why is the overall error occurring? (from what I've read, it seems like it has to reauthenticate the credentials and it doesn't like it?)
Why is it not happening when I use admin credentials but with a 'normal' users credentials?
How do I solve this?
To answer the first question, you need a project scoped token since the unscoped token doesn't have any role associated with it. You cannot do any operation with the unscoped token. Here is a way to get the project scoped token:
auth = v3.Password(auth_url=auth_url,
username=username,
password=password,
user_domain_name="default",
project_name=project_name,
project_domain_name="default")
sess = session.Session(auth=auth, verify='/etc/ssl/certs/ca-bundle.crt')
keystoneClient = client.Client(session=sess)
And for a normal user, you cannot get back the entire list of projects in the provider and this is why it works fine with using admin credentials but not for that of a normal user.
In order to solve it, you need to know the user_id of that normal user. One way is to use the admin credentials to get a keystone client and call
keystone.users.get(user="username")
Or just use the OpenStack Dashboard and go to the identity dashboard. There is a panel called User and you can see the user_id from there.
Once you have the user_id, you can do:
keystoneClient.projects.list(user=user_id)
HTH.
I'm trying to write a redditbot; I decided to start with a simple one, to make sure I was doing things properly, and I got a RequestException.
my code (bot.py):
import praw
for s in praw.Reddit('bot1').subreddit("learnpython").hot(limit=5):
print s.title
my praw.ini file:
# The URL prefix for OAuth-related requests.
oauth_url=https://oauth.reddit.com
# The URL prefix for regular requests.
reddit_url=https://www.reddit.com
# The URL prefix for short URLs.
short_url=https://redd.it
[bot1]
client_id=HIDDEN
client_secret=HIDDEN
password=HIDDEN
username=HIDDEN
user_agent=ILovePythonBot0.1
(where HIDDEN replaces the actual id, secret, password and username.)
my Traceback:
Traceback (most recent call last):
File "bot.py", line 3, in <module>
for s in praw.Reddit('bot1').subreddit("learnpython").hot(limit=5):
File "/usr/local/lib/python2.7/dist-packages/praw/models/listing/generator.py", line 79, in next
return self.__next__()
File "/usr/local/lib/python2.7/dist-packages/praw/models/listing/generator.py", line 52, in __next__
self._next_batch()
File "/usr/local/lib/python2.7/dist-packages/praw/models/listing/generator.py", line 62, in _next_batch
self._listing = self._reddit.get(self.url, params=self.params)
File "/usr/local/lib/python2.7/dist-packages/praw/reddit.py", line 322, in get
data = self.request('GET', path, params=params)
File "/usr/local/lib/python2.7/dist-packages/praw/reddit.py", line 406, in request
params=params)
File "/usr/local/lib/python2.7/dist-packages/prawcore/sessions.py", line 131, in request
params=params, url=url)
File "/usr/local/lib/python2.7/dist-packages/prawcore/sessions.py", line 70, in _request_with_retries
params=params)
File "/usr/local/lib/python2.7/dist-packages/prawcore/rate_limit.py", line 28, in call
response = request_function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/prawcore/requestor.py", line 48, in request
raise RequestException(exc, args, kwargs)
prawcore.exceptions.RequestException: error with request request() got an unexpected keyword argument 'json'
Any help would be appreciated. PS, I am using Python 2.7., on Ubuntu 14.04. Please ask me for any other information you may need.
The way i see it, it seems you have a problem with your request to Reddit API. Maybe try changing the user-agent in your in-file configuration. According to PRAW basic configuration Options the user-agent should follow a format <platform>:<app ID>:<version string> (by /u/<reddit username>) . Try that see what happens.