I'm using a standard try/except syntax for skipping rows in a csv file that aren't streaming properly and therefore can't be downloaded. My code:
for row in list_reader:
media_id = row['mediaId']
filename = row['mediaId']
saveFile = media.get_item(media_id)
stream_url = saveFile['streams'][0]['streamLocation']
try:
r = requests.get(stream_url, allow_redirects=True)
with open(os.path.join('./media', filename), 'wb') as ofile:
ofile.write(r.content)
counter += 1
except:
IndexError
print "error"
However after downloading a number of files the problem row comes up, the error is not handled and I get the error:
Traceback (most recent call last):
File "downloadmedia.py", line 28, in <module>
stream_url = saveFile['streams'][0]['streamLocation']
IndexError: list index out of range
I've tried an if/else syntax instead, using the length of the stream_url variable, but this gives the same error. Can someone explain why the error handling doesn't work?
As stated in the comments, your try/except is in the wrong place. Through the error you provided, you can see that the index error occurs at the line stream_url = saveFile['streams'][0]['streamLocation']
You need to make sure the try/except is covering this line to prevent this.
for row in list_reader:
try:
media_id = row['mediaId']
filename = row['mediaId']
saveFile = media.get_item(media_id)
stream_url = saveFile['streams'][0]['streamLocation']
r = requests.get(stream_url, allow_redirects=True)
with open(os.path.join('./media', filename), 'wb') as ofile:
ofile.write(r.content)
counter += 1
except IndexError:
print "error"
Related
I'm following this tutorial from the website: https://towardsdatascience.com/creating-the-twitter-sentiment-analysis-program-in-python-with-naive-bayes-classification-672e5589a7ed
Everything is good so far but I keep getting an error when trying to run this code.
def buildTrainingSet(corpusFile, tweetDataFile):
import csv
import time
corpus = []
with open(corpusFile,'rb') as csvfile:
lineReader = csv.reader(csvfile,delimiter=',', quotechar="\"")
for row in lineReader:
corpus.append({"tweet_id":row[2], "label":row[1], "topic":row[0]})
rate_limit = 180
sleep_time = 900/180
trainingDataSet = []
for tweet in corpus:
try:
status = twitter_api.GetStatus(tweet["tweet_id"])
print("Tweet fetched" + status.text)
tweet["text"] = status.text
trainingDataSet.append(tweet)
time.sleep(sleep_time)
except:
continue
# now we write them to the empty CSV file
with open(tweetDataFile,'wb') as csvfile:
linewriter = csv.writer(csvfile,delimiter=',',quotechar="\"")
for tweet in trainingDataSet:
try:
linewriter.writerow([tweet["tweet_id"], tweet["text"], tweet["label"], tweet["topic"]])
except Exception as e:
print(e)
return trainingDataSet
#================
corpusFile = "C:\Users\Vilma\Documents\CIS450\group prjt/corpus.csv"
tweetDataFile = "C:\Users\Vilma\Documents\CIS450\group prjt/tweetDataFile.csv"
trainingData = buildTrainingSet (corpusFile, tweetDataFile)
I keep getting this error:
File "<ipython-input-33-54fea359e8f9>", line 1
corpusFile = "C:\Users\Vilma\Documents\CIS450\group prjt/corpus.csv"
^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
I even tried putting r' in front of C:\Users\Vilma\Documents\CIS450\group prjt/corpus.csvbut I still keeping getting error.
update: Fixed error, I put code as
corpusFile = r'C:\Users\Vilma\Documents\CIS450\group prjt\corpus.csv'
tweetDataFile = r'C:\Users\Vilma\Documents\CIS450\group prjt\tweetDataFile.csv'
However, a new error pops up:
File "<ipython-input-41-f44768dabc6e>", line 7, in buildTrainingSet
with open(corpusFile,'rb') as csvfile:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Vilma\\Documents\\CIS450\\group prjt\\corpus.csv'
Try correcting your file path.
corpusFile = "C:\Users\Vilma\Documents\CIS450\group prjt/corpus.csv"
Should be:
corpusFile = "C:\Users\Vilma\Documents\CIS450\group prjt\corpus.csv"
Hope this helps!
You can use:
corpusFile = r"C:\Users\Vilma\Documents\CIS450\group prjt\corpus.csv"
If you are not finding the file, please make sure the file exists in the folder.
I'm having problem with my app in python my app is for downloading videos from the web at a specified time. my program name is tidopy.py
but I get this Error:
Traceback (most recent call last):
File "tidopy.py", line 29, in
file.write(data)
TypeError: argument 1 must be string or buffer, not instance
I have problem with this part:
while (coun > x):
file = open(namelist[x], 'wb')
file.write(urllib2.urlopen(addresslist[x])).read()
file.close()
x = x + 1
x is a variable for the number of videos.
namelist is a list for the name of videos.
addresslist is a list for the address of web videos
How can I fix it?
please help.
Here is a simple code to perform a download from a list.
import requests
import shutil
namelist = [...]
addresslist = [...]
for k, x in enumerate(namelist):
r = requests.get(x, stream=True)
if r.ok:
with open(addresslist[k], 'wb') as f:
r.raw.decode_content = True
shutil.copyfileobj(r.raw, f)
I am currently working on a project for which I need to download a few thousand citations from PubMed. I am currently using BioPython and have written this code:
from Bio import Entrez
from Bio import Medline
from pandas import *
from sys import argv
import os
Entrez.email = "my_email"
df = read_csv("my_file_path")
i=0
for index, row in df.iterrows():
print (row.id)
handle = Entrez.efetch(db="pubmed",rettype="medline",retmode="text", id=row.id)
records = Medline.parse(handle)
for record in records:
try:
abstract = str(record["AB"])
except:
abstract = "none"
try:
title = str(record["TI"])
except:
title = "none"
try:
mesh = str(record["MH"])
except:
mesh = "none"
path = 'my_file_path'
filename= str(row.id) + '.txt'
filename = os.path.join(path, filename)
file = open(filename, "w")
output = "title: "+str(title) + "\n\n" + "abstract: "+str(abstract) + "\n\n" + "mesh: "+str(mesh) + "\n\n"
file.write(output)
file.close()
print (i)
i=i+1
However, I receive the following error when this code is run:
Traceback (most recent call last):
File "my_file_path", line 13, in <module>
handle = Entrez.efetch(db="pubmed",rettype="medline",retmode="text", id=row.id)
File "/.../anaconda/lib/python3.5/site-packages/biopython-1.68-py3.5-macosx-10.6-x86_64.egg/Bio/Entrez/__init__.py", line 176, in efetch
if ids.count(",") >= 200:
AttributeError: 'numpy.int64' object has no attribute 'count'
Here are the first few columns of the CSV file:
id
10029645
10073846
10078088
10080457
10088066
...
Your error is at
handle = Entrez.efetch(db="pubmed",rettype="medline",retmode="text", id=row.id)
From the documentation
id
UID list. Either a single UID or a comma-delimited list of UIDs
From the examples I see, id is a string, not a numpy.int64 out of a pandas dataframe. You should convert that row.id to a string
I wrote a function. Now I keep getting syntax errors within the try statement. I don't know if its the code I wrote or the try statement
Function:
def connector (links):
for links in infile:
avenues = links.rstrip()
words = []
dct = {}
cord = []
There is more to the code but the error keeps occurring in the try statement, where it says except, any ideas?
try:
infile = open("routes.txt", "r")
links = inf.readlines()
Connector(links)
except LookupError as exceptObj:
print("Error:", str(exceptObj))
connector should be lowercase
You indented wrong
try:
infile = open("routes.txt", "r")
links = inf.readlines()
connector(links)
except LookupError as exceptObj:
print("Error:", str(exceptObj))
This produces and error:
ValueError: Expecting value: line 1 column 1 (char 0)
Here is my code:
...
print("Your phonebook contains the following entries:")
for name, number in phoneBook.items():
print("%s - %s" % (name, number))
while not created:
if not os.path.isfile('phonebook.json'):
with open('phonebook.json', 'wb') as f:
try:
f.write('{}')
except TypeError:
{}
created = True
print('New phonebook created!')
else:
print('Phonebook found!')
created = True
with open('phonebook.json', 'r') as f:
try:
phoneBook_Ori = json.load(f)
phoneBook_Upd = dict(phoneBook_Ori.items() + phoneBook.items())
phoneBook_Ori.write(phoneBook_Upd)
except EOFError:
{}
if EOFError:
with open('phonebook.json', 'w') as f:
json.dump(phoneBook, f)
else:
with open('phonebook.json', 'w') as f:
json.dump(phoneBook_Ori, f)
Has anyone got an idea of how to fix this?
I have also previously asked a question on this code here
I copy pasted your code in the python 2.x interpreter.
I received a ValueError regarding the phonebook.json file. I created a dummy file with:
{'sean':'310'}
My error reads:
ValueError: Expecting property name: line 1 column 2
This was the only way I was able to receive a ValueError.
Therefore, I believe your issue lies in the way the json is written in phonebook.json. Can you post its contents or a subset?
Also, using phoneBook_Ori.write() seems very questionable, as the json module has no method called write(), and the return on json.load(), if used on json objects, is a dictionary, which also cannot write(). You would probably want to use json.dump().
read more at:
https://docs.python.org/2/library/json.html
Anyway, I hope I was helpful.
I was getting this error whilst using json.load(var) with var containing an empty JSON response from a REST API call.
In your case, the JSON response (phonebook.json) must have records. This will fix the error.