I'm trying to write a dictionary to a file.
and when I try to read it back, it returns nothing, as if the file is empty.
I saw the file and there is something written into it.
I think that the way I write is not the best and that's why it does the problem.
my expected outcome is to read the file, get back the dictionary, and write to it. This is how I write:
# this function gets the db_name, creating a text file with the db_name
# and creating a empty dictionary with the key that is the table_name.
def __createTheDb(self,dirPath, db_name, table_name):
global db_file
os.chdir(dirPath) #gets into the directory
print(os.getcwd(), "from create the db")
db_file = open(db_name + ".txt", "w")
temp_dict = dict.fromkeys(f"{table_name}".split())
db_file.write(str(temp_dict))
return db_file
How I write the file:
def writeFile(self, db_name, table_name, key, value):
global db_file, my_json #gets the global variable
print(type(db_file))
file = open(db_name,"r").read()
print(file) #-> the problem is that this prints nothing.
I tried to read it with json.load(db_file), but it says it's unreadable.
The problem ->
return loads(fp.read(),
io.UnsupportedOperation: not readable
I just want to convert the text file to dictionary so I can set it's key...
try loading it with
with open("./path/file.json", "r") as f:
loadedDic = json.load(f)
then loadedDic will be equal to the dictionary IF the file you loaded is just a dictionary. If it has a list of dictionaries, or a dictionary of dictionaries, etc you'll need to parse it the way you normally parse lists/dicts/etc.
Related
I have created a JSON file after scraping data online with the following simplified code:
for item in range(items_to_scrape)
az_text = []
for n in range(first_web_page, last_web_page):
reviews_html = requests.get(page_link)
tree = fromstring(reviews_html.text)
page_link = base_url + str(n)
review_text_tags = tree.xpath(xpath_1)
for r_text in review_text_tags:
review_text = r_text.text
az_text.append(review_text)
az_reviews = {}
az_reviews[item] = az_text
with open('data.json', 'w') as outfile:
json.dump(az_reviews , outfile)
There might be a better way to create a JSON file with the first key equal to the item number and the second key equal to the list of reviews for that item, however I am currently stuck at opening the JSON file to see the items have been already scraped.
The structure of the JSON file looks like this:
{
"asin": "0439785960",
"reviews": [
"Don’t miss this one!",
"Came in great condition, one of my favorites in the HP series!",
"Don’t know how these books are so good and I’ve never read them until now. Whether you’ve watched the movies or not, read these books"
]
}
The unsuccessful attempt that seems to be closer to the solution is the following:
import json
from pprint import pprint
json_data = open('data.json', 'r').read()
json1_file = json.loads(json_data)
print(type(json1_file))
print(json1_file["asin"])
It returns a string that replicates exactly the result of the print() function I used during the scraping process to check what the JSON file was going to be look like, but I can't access the asins or reviews using json1_file["asin"] or json1_file["reviews"] since the file read is a string and not a dictionary.
TypeError: string indices must be integers
Using the json.load() function I still print the right content, but I have cannot figure out how to access the dictionary-like object from the JSON file to iterate through keys and values.
The following code prints the content of the file, but raises an error (AttributeError: '_io.TextIOWrapper' object has no attribute 'items') when I try to iterate through keys and values:
with open('data.json', 'r') as content:
print(json.load(content))
for key, value in content.items():
print(key, value)
What is wrong with the code above and what should be adjusted to load the file into a dictionary?
string indices must be integers
You're writing out the data as a string, not a dictionary. Remove the dumps, and only dump
with open('data.json', 'w') as outfile:
json.dump(az_reviews, outfile, indent=2, ensure_ascii=False)
what should be adjusted to load the file into a dictionary?
Once you're parsing a JSON object, and not a string, then nothing except maybe not using reads, then loads and rather only json.load
Another problem seems to be that you're overwriting the file on every loop iteration
Instead, you probably want to open one file then loop and write to it afterwards
data = {}
for item in range(items_to_scrape):
pass # add to data
# put all data in one file
with open('data.json', 'w') as f:
json.dump(data, f)
In this scenario, I suggest that you store the asin as a key, with the reviews as values
asin = "123456" # some scraped value
data[asin] = reviews
Or write a unique file for each scrape, which you then must loop over to read them all.
for item in range(items_to_scrape):
data = {}
# add to data
with open('data{}.json'.format(item), 'w') as f:
json.dump(data, f)
I'm storing a dictionary on a JSON file so I can easily access and change the data between sessions. This is how I'm trying to load the file into a variable, but after "pc_data = " I'm getting 'Expression Expected' from PyCharm. How am I able to load the file into a variable and then continue the function after variable assignment? Code below:
pc_data = with open(lib_dir+'player_characters.txt', 'r') as json_data:
json.load(json_data)
yield json_data
Your use of with is incorrect. with does not return a value, so you cannot assign it.
Try this:
with open(lib_dir+'player_characters.txt', 'r') as json_data:
pc_data = json.load(json_data)
yield pc_data # <- I'm guessing you want to yield parsed json here?
Scenario is i need to convert dictionary object as json and write to a file . New Dictionary objects would be sent on every write_to_file() method call and i have to append Json to the file .Following is the code
def write_to_file(self, dict=None):
f = open("/Users/xyz/Desktop/file.json", "w+")
if json.load(f)!= None:
data = json.load(f)
data.update(dict)
f = open("/Users/xyz/Desktop/file.json", "w+")
f.write(json.dumps(data))
else:
f = open("/Users/xyz/Desktop/file.json", "w+")
f.write(json.dumps(dict)
Getting this error "No JSON object could be decoded" and Json is not written to the file. Can anyone help ?
this looks overcomplex and highly buggy. Opening the file several times, in w+ mode, and reading it twice won't get you nowhere but will create an empty file that json won't be able to read.
I would test if the file exists, if so I'm reading it (else create an empty dict).
this default None argument makes no sense. You have to pass a dictionary or the update method won't work. Well, we can skip the update if the object is "falsy".
don't use dict as a variable name
in the end, overwrite the file with a new version of your data (w+ and r+ should be reserved to fixed size/binary files, not text/json/xml files)
Like this:
def write_to_file(self, new_data=None):
# define filename to avoid copy/paste
filename = "/Users/xyz/Desktop/file.json"
data = {} # in case the file doesn't exist yet
if os.path.exists(filename):
with open(filename) as f:
data = json.load(f)
# update data with new_data if non-None/empty
if new_data:
data.update(new_data)
# write the updated dictionary, create file if
# didn't exist
with open(filename,"w") as f:
json.dump(data,f)
i'm writing a script in Python doing a while true cicle, how can I make my script take the same file abc123.json for each cicle and modify some variables in it?
If I understand your question correctly, you want to read a file named abc123.json somewhere on a local hard drive that is accessible via path and modify a value for a key (or more) for that json file, then re-write it.
I'm pasting an example of some code I used a while ago in hopes it helps
import json
from collections import OrderedDict
from os import path
def change_json(file_location, data):
with open(file_location, 'r+') as json_file:
# I use OrderedDict to keep the same order of key/values in the source file
json_from_file = json.load(json_file, object_pairs_hook=OrderedDict)
for key in json_from_file:
# make modifications here
json_from_file[key] = data[key]
print(json_from_file)
# rewind to top of the file
json_file.seek(0)
# sort_keys keeps the same order of the dict keys to put back to the file
json.dump(json_from_file, json_file, indent=4, sort_keys=False)
# just in case your new data is smaller than the older
json_file.truncate()
# File name
file_to_change = 'abc123.json'
# File Path (if file is not in script's current working directory. Note the windows style of paths
path_to_file = 'C:\\test'
# here we build the full file path
file_full_path = path.join(path_to_file, file_to_change)
#Simple json that matches what I want to change in the file
json_data = {'key1': 'value 1'}
while 1:
change_json(file_full_path, json_data)
# let's check if we changed that value now
with open(file_full_path, 'r') as f:
if 'value 1' in json.load(f)['key1']:
print('yay')
break
else:
print('nay')
# do some other stuff
Observation: the code above assumes that both your file and the json_data share the same keys. If they dont, your function will need to figure out how to match keys between data structures.
I have a file called usernames.py that may contain a list or does exist at all:
usernames.py
['user1', 'user2', 'user3']
In Python I now want to read this file if it exists and append to the list a new user or create a list with that user i.e. ['user3']
This is what I have tried:
with open(path + 'usernames.py', 'w+') as file:
file_string = host_file.read()
file_string.append(instance)
file.write(file_string)
This gives me an error unresolved 'append'. How can I achieve this? Python does not know it is a list and if the file does not exist even worst as I have nothing to convert to a list.
Try this:
import os
filename = 'data'
if os.path.isfile(filename):
with open(filename, 'r') as f:
l = eval(f.readline())
else:
l = []
l.append(instance)
with open(filename, 'w') as f:
f.write(str(l))
BUT this is quite unsafe if you don't know where the file is from as it could include any code to do anything!
It would be better not to use a python file for persistence -- what happens if someone slips you a usernames.py that has exploit code in it? Consider a csv file or a pickle, or just a text file with one user per line.
That said, if you don't open it as a python file, something like this should work:
from os.path import join
with open( join(path, 'usernames.py'), 'r+') as file:
file_string = file.read()
file_string = file_string.strip().strip('[').strip(']')
file_data = [ name.strip().strip('"').strip("'") for name in file_string.split(',' )]
file_data.append( instance )
file.fseek(0)
file.write(str(file_data))
If usernames contain commas or end in quotes, you have to be more careful.