print user_dic[id] displays the right result PersonA. This is when I input the id manually.
user_stream = {u'2331449': u'PersonB', u'17800013': u'PersonA'}
user_dic= {}
for item in user_stream:
user_dic[item['id']] = item['name']
id = '17800013'
print user_dic[id] #returns the right value
However, when I try to put the user_id through a for loop that iterates through json I get an error: KeyError at 17800013 for the line name = user_dic[user_id]. I don't understand why the user_dic[id] works when manually inputting the id, but user_dic[user_id] doesn't work when going through the for loop even though the input is the same.
#right fql query
fql_query = "SELECT created_time, post_id, actor_id, type, updated_time, attachment FROM stream WHERE post_id in (select post_id from stream where ('video') in attachment AND source_id IN ( SELECT uid2 FROM friend WHERE uid1=me()) limit 100)"
fql_var = "https://api.facebook.com/method/fql.query?access_token=" + token['access_token'] + "&query=" + fql_query + "&format=json"
data = urllib.urlopen(fql_var)
fb_stream = json.loads(data.read())
fb_feed = []
for post in fb_stream:
user_id = post["actor_id"]
name = user_dic[user_id] #this is the line giving me trouble
title = post["attachment"]["name"]
link = post["attachment"]["href"]
video_id = link[link.find('v=')+2 : link.find('v=')+13]
fb_feed.append([user_id, name, title, video_id])
There is no need for user_dic. What you are doing in first part is just a redundant work and you are also doing it wrong. Your user_stream is already in a form how you wanted it. Your first part should contain this line:
user_stream = {u'2331449': u'PersonB', u'17800013': u'PersonA'}
And in second part (at line where you are facing problem) you should do:
name = user_stream[user_id]
If you think that you will face KeyError then dict has a method .get, which returns None if the Key is not found. You can specify your value instead of None to return if there is KeyError
name = user_stream.get('user_id')
#returns None by default
name = user_stream.get('user_id', '')
#returns empty string now
#on both cases exception will not raised
Related
I got the following problem. I'm trying to pull the specific field, in the "warnings" array, which has the given UID. I can't seem to figure out why it's not working.
The output (Everything prints out successfully): https://i.imgur.com/ZslJ0rV.png\
My MongoDB structure: https://i.imgur.com/3bRegAD.png
client = pymongo.MongoClient("")
database = client["LateNight"]
ModlogsCollection = database["modlogs"]
theUID = "63TF-lYv0-72m7-9f4I"
theGuild = 1063516188988153896
all_mod_docs = ModlogsCollection.find({"_id": str(theGuild)})
all_uids = []
for doc in all_mod_docs:
doc_keys = [key for key in doc.keys() if key != "_id"]
for key in doc_keys:
sub_doc = doc[key]
if warnings := sub_doc.get("warnings"):
for warning in warnings:
if warning["UID"] == theUID:
print(warning)
print("Warning")
result = ModlogsCollection.update_one(
{"_id": str(theGuild)},
{"$pull": {
"warnings": {"UID": theUID}
}}
)
print(result)
print(result.modified_count)
as you yourself said you try to "extract the specific field, in the warnings table that has the UID given". Before recovering the UID value you must specify the index 0. Afterwards you get a dictionary that will have the keys:
moderator, reason, time and UID
I'm using Python3 with BeautifulSoup. I want to scrape data for a few employees from a site, depending on their ID number.
My code:
for UID in range(201810000,201810020):
ID = UID
print(ID)
#scrapped Data
ZeroDay = s.post("https://site/Add_StudantRow.php",data={"SID":ID})
ZeroDay_content = bs(ZeroDay.content,"html.parser", from_encoding='windows-1256')
std_ID = ZeroDay_content.find("input", {"name":"SID[]"})["value"]
std_name = ZeroDay_content.find("input", {"name":"Name[]"})["value"]
std_major_= ZeroDay_content.select_one("option[selected]", {"name":"Qualifications[]"})["value"]
std_major = ZeroDay_content.find("input", {"name":"Specialization[]"})["value"]
std_social= ZeroDay_content.select_one("select[name='MILITARY_STATUS[]'] option[selected]")["value"]
std_ID_num= ZeroDay_content.find("input", {"name":"ID_Number[]"})["value"]
std_gender= ZeroDay_content.select_one("select[name='Gender[]'] option[selected]")["value"]
print(std_ID,std_name,std_gender,std_major,std_major_,std_ID_num,std_social)
After I ran my code, this error appeared:
std_ID = ZeroDay_content.find("input", {"name":"SID[]"})["value"]
TypeError: 'NoneType' object is not subscriptable
I assigned a range for their ID's from 201810000 to 201810020 but not all the IDs are valid. I mean maybe 201810015 not valid and 201810018 valid.
Note: when I put a valid ID in UID the error did not appear, possibly because when the ID returns a null value the error appears, but how can I do a range of IDs in this case?
As not all of your UID values return a valid page, you would just need to first test for the presence of a required tag. As you are looking for form elements, I assume there will be an enclosing <form> tag you could test for first.
For example:
for UID in range(201810000, 201810020):
ID = UID
print(ID)
ZeroDay = s.post("https://site/Add_StudantRow.php", data={"SID":ID})
ZeroDay_content = bs(ZeroDay.content, "html.parser", from_encoding='windows-1256')
if ZeroDay_content.find("form", <xxxxxxx>):
std_ID = ZeroDay_content.find("input", {"name":"SID[]"})["value"]
std_name = ZeroDay_content.find("input", {"name":"Name[]"})["value"]
std_major_= ZeroDay_content.select_one("option[selected]", {"name":"Qualifications[]"})["value"]
std_major = ZeroDay_content.find("input", {"name":"Specialization[]"})["value"]
std_social= ZeroDay_content.select_one("select[name='MILITARY_STATUS[]'] option[selected]")["value"]
std_ID_num= ZeroDay_content.find("input", {"name":"ID_Number[]"})["value"]
std_gender= ZeroDay_content.select_one("select[name='Gender[]'] option[selected]")["value"]
print(std_ID, std_name, std_gender, std_major, std_major_, std_ID_num,s td_social)
Where <xxxxx> would be suitable attributes to search for.
The error you are getting is because your first .find() call is returning None to indicate that the item is not present. You then use ["value"] on None which gives the error without first testing if you have found the required item.
I resolve this by add an IF statement and use content-length as a thing to determine that the request was made or not, because i have noticed that the content-length is less than 170 if the request is return nothing and more 170 if return any thing .
I'm adding articles into a database. That worked fine, except, while the database doesn't let me create duplicates during a session, once I log out and log back in, it does. I didn't want duplicates, so I added these new lines:
maybe_existing_article = Article.query.filter_by(url=article.url)
if (maybe_existing_article):
article = maybe_existing_article.url
return "exists"
But that didn't work, I get "print("article.id=" + str(article.id))
NameError: name 'article' is not defined".
Here is the relevant code, including the new lines.
##############################################
#app.route('/bootstrap', methods=['GET', 'POST'])
def bootstrap():
posted = 1
print ("bootstrap")
global article
if request.method == 'POST':
if not request.form['title'] or not request.form['url'] or not request.form['image_url'] or not request.form['snippet']:
flash('Please enter all the fields', 'error')
else:
article = Article(request.form['title'], request.form['url'], request.form['image_url'],
request.form['snippet'])
maybe_existing_article = Article.query.filter_by(url=article.url)
if (maybe_existing_article):
article = maybe_existing_article.url
return "exists"
else:
db.session.add(article)
try:
db.session.commit()
except exc.SQLAlchemyError:
flash('Article url already exists, failed to post new article')
posted = 0
#return render_template('/error.html', article_url=article.url)
article_list = Article.query.filter_by(url=article.url)
if posted == 1:
flash('Record was successfully added')
else:
db.session.rollback()
article_list = Article.query.filter_by(url=article.url)
article=article_list[0]
print ("article.id=" + str(article.id))
import json
print("a")
return json.dumps(article.id)
else:
print("article.id=" + str(article.id))
urlNumber = str(article.id)
message = {'greeting':urlNumber}
return jsonify(message) # serialize and use JSON headers
And here's the create_tables.py:
article_table = """CREATE TABLE IF NOT EXISTS article (
id INTEGER PRIMARY KEY,
title TEXT NOT NULL,
url TEXT NOT NULL,
image_url TEXT NOT NULL,
snippet TEXT NOT NULL,
date_upload TEXT DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT article_unq UNIQUE (url)
);"""
You can use an Unique constraint at the table that you want and catch the error when you try to insert a row that was already inserted. In SQLite, you can't add this constraint after the Table is created, so you would need to create the table again.
You can also add an Unique Index to the table that works almost the same:
CREATE UNIQUE INDEX index_name ON table_name(column_name);
If you can't do that, you can always send a SELECT to that table with the ID you are trying to insert and them return an error to the user if it was already inserted, although this is hardly recommended.
I need to loop through commits and get name, date, and messages info from
GitHub API.
https://api.github.com/repos/droptable461/Project-Project-Management/commits
I have many different things but I keep getting stuck at string indices must be integers error:
def git():
#name , date , message
#https://api.github.com/repos/droptable461/Project-Project-Management/commits
#commit { author { name and date
#commit { message
#with urlopen('https://api.github.com/repos/droptable461/Project Project-Management/commits') as response:
#source = response.read()
#data = json.loads(source)
#state = []
#for state in data['committer']:
#state.append(state['name'])
#print(state)
link = 'https://api.github.com/repos/droptable461/Project-Project-Management/events'
r = requests.get('https://api.github.com/repos/droptable461/Project-Project-Management/commits')
#print(r)
#one = r['commit']
#print(one)
for item in r.json():
for c in item['commit']['committer']:
print(c['name'],c['date'])
return 'suc'
Need to get person who did the commit, date and their message.
item['commit']['committer'] is a dictionary object, and therefore the line:
for c in item['commit']['committer']: is transiting dictionary keys.
Since you are calling [] on a string (the dictionary key), you are getting the error.
Instead that code should look more like:
def git():
link = 'https://api.github.com/repos/droptable461/Project-Project-Management/events'
r = requests.get('https://api.github.com/repos/droptable461/Project-Project-Management/commits')
for item in r.json():
for key in item['commit']['committer']:
print(item['commit']['committer']['name'])
print(item['commit']['committer']['date'])
print(item['commit']['message'])
return 'suc'
I'm having trouble changing a key value to a dictionary value
def get(self):
#Get all the Subjects
subjects = ndb.gql('SELECT name,order FROM Subject ORDER BY order ASC')
values = {'subjects':subjects}
#Get all the Contents
for subject in subjects:
contents = ndb.gql('SELECT * FROM Content WHERE ANCESTOR IS :1 ORDER BY order ASC',subject.key)
values[subject.name] = contents #***HERE is the issue***
Rather than getting a dictionary
value = {key:value}
I'm trying to get
value = {{key:value}:value}
Thanks in advance for any suggestions!
EDIT:
When I try
values['subject':subject.name] = contents
I get the error
TypeError: unhashable type
Solved: with a different approach:
def get(self):
#Get all the Subjects
subjects = ndb.gql('SELECT name,order FROM Subject ORDER BY order ASC')
values = {'subjects':subjects}
#Get all the Contents
values['contents'] = []
for subject in subjects:
#Formatting HTML output
subjectAll = subject.name + ' ' + subject.order
contents = ndb.gql('SELECT name,order FROM Content WHERE ANCESTOR IS :1 ORDER BY order ASC',subject.key)
values['contents'].append(subjectAll)
for content in contents:
#Formatting HTML output
contentAll = content.name + ' ' + content.order
values['contents'].append(contentAll)