I'm trying to test this code out: https://github.com/shanepm/500px-Bot/blob/master/500px.py
Completely new to Phyton, but have programming skills.
Running on Windows 10.
I get this error msg:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The error is pointed to be at pendingFollowList = json.loads(f.read()) and on the rest of the json.loads...calls
Have tried to pass one more argument on the line above:
with open('pendingUsers.txt', 'r', **encoding='utf-8'**) as f:
This did not help.
Have also checked Encoding on .txt files as well.
Anybody that knows what to do??
Thanks in advance!
Alen
What I would do is instead of using a .txt file, use a .json file because you're using json. I've used
import json
with open("filename.json") as f:
Json_content = json.load(f)
And then if you want to access one of the values of your Json you would Json_content["key_in_json_file"]
all!
It helped out to change the extension.
My main issue was that my input files where blank and not json-formatted correctly.
Thanks to all for the contribution.
BR
Alen
Related
I'm relatively new to python / using JSONs and I'm having an issue.
I'm trying to add 200 of my tweets to a json file. here is my code that does this :
def process_or_store(tweet):
with open('baileighsTweets.json', 'a') as f:
f.write(json.dumps(tweet._json, indent=4))
f.close()
for tweet in tweepy.Cursor(api.user_timeline).items(200):
process_or_store(tweet)
This code runs fine, and adds my tweets to a json file, with each tweet being a json object. however, an error occurs with one of my objects in the json file :
picture with the error
same code on a different line, no error
it appears to be a very basic syntax issue but I'm confused about why it happened - my code adds to the json file, I don't do it manually, so I don't understand why I received an 'end of file expected' error, and I don't how to fix it.
Thanks in advance for your help/suggestions guys!
It would have been good if you had attached the json file along the question. I feel there might be syntax issue like missing "," or the "}" braces. You can use https://jsonlint.com/ to validate your json and try to understand where the real issue is. Hope this helps you.
I have big size of json file to parse with python, however it's incomplete (i.e., missing parentheses in the end). The json file consist of one big json object which contains json objects inside. All json object in outer json object is complete, just finishing parenthese are missing.
for example, its structure is like this.
{bigger_json_head:value, another_key:[{small_complete_json1},{small_complete_json2}, ...,{small_complete_json_n},
So final "]}" are missing. however, each small json forms a single row so when I tried to print each line of the json file I have, I get each json object as a single string.
so I've tried to use:
with open("file.json","r",encoding="UTF-8") as f:
for line in f.readlines()
line_arr.append(line)
I expected to have a list with line of json object as its element
and then I tried below after the process:
for json_line in line_arr:
try:
json_str = json.loads(json_line)
print(json_str)
except json.decoder.JSONDecodeError:
continue
I expected from this code block, except first and last string, this code would print json string to console. However, it printed nothing and just got decode error.
Is there anyone who solved similar problem? please help. Thank you
If the faulty json file only miss the final "]}", then you can actually fix it before parse it.
Here is an example code to illustrate:
with open("file.json","r",encoding="UTF-8") as f:
faulty_json_str = f.read()
fixed_json_str = faulty_json_str + ']}'
json_obj = json.loads(fixed_json_str)
import ast
dict_from_file=[]
with open('4.txt', 'r') as inf:
dict_from_file = ast.literal_eval(inf.read())
File "<unknown>", line 1
["hello":"work", "please":"work"]
^
SyntaxError: invalid character in identifier
Hi Everyone! The above is my code and my error. I have a really complicated 40MB data file in the form of a dictionary to work on, but couldn't get that import to work so tried a simple one.
I'm using the latest Jupyter notebook from the latest version of Anaconda, on Windows 10. My dictionary is a txt file created using windows notepad. The complicated dictionary was originally a JSON file that I changed into a txt file thinking it would be easier but I may be wrong.
I think the above error is an encoding issue? But not sure how to fix it.
Thanks!
If you are the owner/write of the file (dict formated), save as json
import json
#To write
your_dict = {.....}
with open("file_name.txt", "w+") as f:
f.write(json.dumps(your_dict)
#To read
with open("file_name.txt") as f:
read_dict = json.load(f)
This is possibly a python 3 "feature".
This code removes the unwanted characters at the start of the input file and returns the input data as type string.
with open('4.txt', 'r',,encoding="utf-8-sig") as inf:
dict_from_file = ast.literal_eval(inf.read())
This removes the strange characters put at the beginning of read data.
i have a code that updates CSVs from a server. it gets data using:
a = urllib.urlopen(url)
data = a.read().strip()
then i append the data to the csv by
f = open(filename+".csv", "ab")
f.write(ndata)
f.close()
the problem is that randomly, a line in the csv gets written like this (or gets a line break somewhere along the csv):
2,,,,,
015-04-21 13:00:00,18,998,50,31,2293
instead of its usual form:
2015-04-21 13:00:00,6,1007,29,25,2394
2015-04-21 13:00:00,7,1004,47,26,2522
i tried printing my data in shell after the program ran, and it would show that the broken csv entry actually appears to be normal.
hope you guys can help me out. thanks.
running python 2.7.9 on win8.1
What actions are performed on your "ndata" variable ?
You should use the csv module to manage CSV files : https://docs.python.org/2/library/csv.html
Edit after comment :
If you do not want to use the "csv" module I linked to you, instead of
a = urllib.urlopen(url)
data = a.read().strip()
ndata = data.split('\n')
f.write('\n'.join(ndata[1:]))
you should do this :
a = urllib.urlopen(url)
f.writelines(a.readlines()[1:])
I don't see any reason explaining your randomly unwanted "\n" if you are sure that your incoming data is correct. Do you manage very long lines ?
I recommand you to use the csv module to read your input : you'll be sure to have a valid CSV content if your input is correct.
I just tried to update a program i wrote and i needed to add another pickle file. So i created the blank .pkl and then use this command to open it(just as i did with all my others):
with open('tryagain.pkl', 'r') as input:
self.open_multi_clock = pickle.load(input)
only this time around i keep getting this really weird error for no obvious reason,
cPickle.UnpicklingError: invalid load key, 'Γ'.
The pickle file does contain the necessary information to be loaded, it is an exact match to other blank .pkl's that i have and they load fine. I don't know what that last key is in the error but i suspect that could give me some incite if i know what it means.
So have have figured out the solution to this problem, and i thought I'd take the time to list some examples of what to do and what not to do when using pickle files. Firstly, the solution to this was to simply just make a plain old .txt file and dump the pickle data to it.
If you are under the impression that you have to actually make a new file and save it with a .pkl ending you would be wrong. I was creating my .pkl's with notepad++ and saving them as .pkl's. Now from my experience this does work sometimes and sometimes it doesn't, if your semi-new to programming this may cause a fair amount of confusion as it did for me. All that being said, i recommend just using plain old .txt files. It's the information stored inside the file not necessarily the extension that is important here.
#Notice file hasn't been pickled.
#What not to do. No need to name the file .pkl yourself.
with open('tryagain.pkl', 'r') as input:
self.open_multi_clock = pickle.load(input)
The proper way:
#Pickle your new file
with open(filename, 'wb') as output:
pickle.dump(obj, output, -1)
#Now open with the original .txt ext. DONT RENAME.
with open('tryagain.txt', 'r') as input:
self.open_multi_clock = pickle.load(input)
Gonna guess the pickled data is throwing off portability by the outputted characters. I'd suggest base64 encoding the pickled data before writing it to file. What what I ran:
import base64
import pickle
value_p = pickle.dumps("abdfg")
value_p_b64 = base64.b64encode(value_p)
f = file("output.pkl", "w+")
f.write(value_p_b64)
f.close()
for line in open("output.pkl", 'r'):
readable += pickle.loads(base64.b64decode(line))
>>> readable
'abdfg'