I am trying to set up GPT-2 encoding using the following tutorial:
https://medium.com/#ngwaifoong92/beginners-guide-to-retrain-gpt-2-117m-to-generate-custom-text-content-8bb5363d8b7f
I am trying to enter
python encode.py giselle.txt giselle.npz
in the command prompt but it keeps giving me a FileNotFoundError for an encoder.json file that is already in the path it tries to retrieve it from.
How may I go about fixing this?
I have tried editing the encode.py file to specify the exact path to the encoder.json file but it still gives FileNotFoundError
def get_encoder(model_name):
with open(os.path.join('models', model_name, 'encoder.json'), 'r') as f:
encoder = json.load(f)
with open(os.path.join('models', model_name, 'vocab.bpe'), 'r', encoding="utf-8") as f:
bpe_data = f.read()
I expected the giselle.txt to be encoded to the giselle.npz file but all that is sent in the command prompt is
File "C:\projectgiselle\gpt-2-finetuning\src\encoder.py", line 109, in get_encoder
with open(os.path.join('models', model_name, 'encoder.json'), 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'models\\117M\\encoder.json'
I had this same error!
The tutorial is geared towards Windows users and has you copy train.py and encode.py into the /src/ directory for 'simplicity', so that you don't have to modify the PYTHONPATH. But moving these might violate the code's expectations of where certain scripts are. The solution is simply to move the expect files (encoder.json) correspondingly.
I was able to successfully continue the tutorial by copying and pasting the entire /model/ folder into /src/, so that /src/model/. Then you can run encode.py from /src/ as it instructs.
Related
I'm trying to read in a .txt file on chromebook to manipulate the data that is there. My code to source the file is:
def readInFile():
arr_intValues = []
myFile = open("#MY FILE HERE", "r")
#you need to change the directory
for myLine in myFile:
arr_intValues.append(int(myLine))
return arr_intValues
myNewList = readInFile()
print(myNewList)
Trouble is, i cannot find out where the source of the file is for this code. If i drop the file into a chrome tab, it reads:
file:///media/fuse/crostini_9549350604ce9beeb7d5a9a66d5bf09733144d34_termina_penguin/RandomNum.txt
Meanwhile the file location that "Get info" returns is:
My files/Linux files/RandomNum.txt
Both of these options fail if I attempt to open it and print it in my code.
How do I find the correct file directory?
I attempted to find the directory of a .txt file on using Chrome OS and have not been successful.
If you're running the program on Linux, you'll need to specify the Linux path.
I am trying to create a new file and write on it . but when I am doing something like this I am getting file not found error. seems I am missing something which I can't figure out .
data = {
"table_name": "name",
}
file_path = '/tmp/myfile.json'
with open(file_path + '.json', 'w') as f:
json.dump(data, f, indent=2)
[ERROR] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/myfile.json'
are you trying to run this on Windows by any chance? Or a *nix system where you don't have access to /tmp?
For me, this script runs fine on a Ubuntu and MacOS (although the created file is /tmp/myfile.json.json instead of /tmp/myfile.json because of the + '.json' in your script), but not on Windows.
well python tries to find the file in the cuurent working directory, check if the cwd is what you think it is
import pathlib
print(pathlib.Path().absolute())
u can do this to check the cwd
Issue: Unable to save file in directory (/root/Notion/Image) when using Cron schedule
This is what my code is trying to do:
Check email
Download image attachment
Store in a directory - root/Notion/Image
Retrieve file path
The script is working when I run it manually in Google Cloud terminal. The problem is when I try to schedule it on Cron, it's unable to access the folder to save the file locally.
This is the error when the script failed and require permission:
Traceback (most recent call last):
File "Notion/test.py", line 121, in <module>
path = get_attachments(email.message_from_bytes(msg[0][1]))
File "Notion/test.py", line 47, in get_attachments
with open(filePath, 'wb') as f:
PermissionError: [Errno 13] Permission denied: '/root/Notion/Image/3.jpeg'
This is the code to retrieve attachment from email
def get_attachments(msg):
for part in msg.walk():
if part.get_content_maintype()=='multipart':
continue
if part.get('Content-Disposition') is None:
continue
fileName = part.get_filename()
if bool(fileName):
filePath = os.path.join(attachment_dir, fileName)
with open(filePath, 'wb') as f:
f.write(part.get_payload(decode=True))
return str(filePath)
Resolved:
The problem is that I shouldn't use root directory since it requires permission. I've changed it to home directory instead.
attachment_dir = '/home/dev_thomas_yang/folder_name/folder_name'
For people who needs to check their home direction, simply run this script.
from pathlib import Path
home= str(Path.home())
print(home)
Thanks Triplee for the patience to breakdown my issue despite my sloppy ways of presenting it!
The easiest fix hands down is to change the code so it doesn't try to write to /root. Have it write to the invoking user's home directory instead.
Your question doesn't show the relevant parts of the code, but just change attachment_dir so it's not an absolute path. Maybe separately take care of creating the directory if it doesn't already exist.
import pathlib
# ...
attachment_dir = pathlib.Path("cron/whatever/attachments").mkdir(parents=True, exist_ok=True)
# ...
for loop in circumstances:
get_attachments(something)
A better design altogether would be to have get_attachments accept the directory name as a parameter, so you can make this configurable from the code which calls it. Global variables are a nuisance and cause hard-to-debug problems because they hide information which is important for understanding the code, and tricky to change when you try to debug that code and don't know which parts of the code depend on the old value.
I created a .exe file with Pyinstaller. The program should create a data file with the results at runtime. It works in the Python interpreter mode when I execute python test.py but it doesn't work in the .exe mode. It simply doesn't create a file and responds with an error.
Which flags should I add to the command python -m PyInstaller --hidden-import=timeit --hidden-import=bisect -F test.py to make it work?
The exception with this setup is:
Error: [Errno 2] No such file or directory: "C:\Users\Admin\AppData\Local\Temp\..."
Where the aforementioned directory is temporary and I don't have access to it.
The piece of code which is supposed to write a file is:
def write_file(data, path, encoding='utf-8'):
'''Creates report files.'''
try:
if config.python_version == 2 and type(data) in (list, str):
f = io.open(path, 'wb')
else:
f = io.open(path, 'w', encoding=encoding, newline='')
if type(data) is list:
writer = csv.writer(f)
writer.writerows(data)
else:
f.write(data)
f.close()
console(u'Report file: ' + path)
except IOError as e:
console(e, level=Level.error)
I assume there should be a setting to point to the place where the file should be saved.
I've checked here https://pyinstaller.readthedocs.io/en/stable/spec-files.html#adding-files-to-the-bundle but without success. I couldn't use the listed flags properly, nothing seemed to work.
How can I specify the place where the file would be saved?
The issue isn't with pyinstaller, it's with however you're creating your file.
You may be using some environment variable when running your python script from the command line that isn't set when you run your Exe
I have created a simple example of a program that creates a data file in the directory from which it is called:
#myscript.py
f = open("Test.txt", "w")
print("Hello World!", file=f)
Then I generate the Exe with Pyinstaller:
pyinstaller -F myscript.py
Copy the exe anywhere and you can create Test.txt, if you have permissions in that folder.
Double click myscript.exe
Test.txt appears in the same folder as myscript.exe
Hello World!
I am trying to read content of a file on my work network from my work network. I copy and pasted a code snippet from a google search and modified it to the below. Why might I still be getting [Errno 2] (I have changed some of the path names for this question board)
The file path in my file explorer shows that "> This PC > word common common" and I don't have "This PC" in my path. I tried adding that into the place I would think it goes in the string. That didn't solve it.
I tried making sure I have matching capitalization. That didn't solve it.
I tried renaming the file to have a *.txt on the end. That didn't solve it.
I tried the different variations of // and / and \ with and without the r predecessor and while that did eliminate the first error I was getting. It didn't help this error.
(Looking at the code errors in the right gutter is says my line length is greater than the PEP8 standard. While I doubt that is the root of my problem, if you can throw in the 'right' wrap method for a file path that long that would be helpful.)
myfile = open("z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246", "rt") # open lorem.txt for reading text
contents = myfile.read() # read the entire file into a string
myfile.close() # close the file
print(contents) # print contents
Full Error Copy:
C:\Users\e087680\PycharmProjects\FailureCompiling\venv\Scripts\python.exe C:/Users/e087680/PycharmProjects/FailureCompiling/FirstScriptAttempt.py
Traceback (most recent call last):
File "C:/Users/e087680/PycharmProjects/FailureCompiling/FirstScriptAttempt.py", line 1, in
myfile = open("z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246", "rt") # open lorem.txt for reading text
FileNotFoundError: [Errno 2] No such file or directory: 'z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246'
EDIT
DEBUG EFFORTS
working to figure out how to change directory. Just in case that is the problem. Tested this code bit
import os
path = "z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246"
os.chdir(path)
isExist = os.path.exists(path)
print(isExist)
Received this error
C:\Users\e087680\PycharmProjects\FailureCompiling\venv\Scripts\python.exe C:/Users/e087680/PycharmProjects/FailureCompiling/ScriptDebugJunkFile.py
Traceback (most recent call last):
File "C:/Users/e087680/PycharmProjects/FailureCompiling/ScriptDebugJunkFile.py", line 5, in <module>
os.chdir(path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246'
My intention for adding the picture below is to show how File Explorer displays the file path for my file
FileExplorerPathScreenShot
EDIT
I think this confirms that my 'OS' doesn't have my file.
from os import path
path.exists("PCC_ESS_FC_Full_Report_65000122-1_R0016_962019_9246")
def main():
print ("File exists:" + str(path.exists('PCC_ESS_FC_Full_Report_65000122-1_R0016_962019_9246')))
if __name__== "__main__":
main()
Output
File exists: False
I thought OS was a standard variable for Operating system. Now I'm not sure.
EDIT
Using Cmd in DOS, I confirmed that my path for the z: is correct
EDIT - Success
I ran
import os
print( os.listdir("z:/"))
Confirmed I don't need the monster string of folders.
Confirmed, although explorer doesn't show it, it is a *.txt file
Once I implemented these two items the first code worked fine.
Thank you #Furas
To open and read a file specify the filename in your path:
myfile = open("U:/matrix_neo/word common common/hello world.txt", "rt") # open file
contents = myfile.read() # read the entire file into a string
myfile.close() # close the file
print(contents) # print contents
The U: is a mapped drive in my network.
I did not find any issue with your change dir example. I used a path on my U: path again and it returned True.
import os
path = "U:/matrix_neo/word common common"
os.chdir(path)
isExist = os.path.exists(path)
print(isExist)
The check the attributes on the directory that you are trying to read from. Also try to copy the file to a local drive for a test and see if you can read the file and also check if it exists.
This is an alternative to the above and uses your path to make sure that the long file path works:
import os
mypath = "z:/abcdefg/abc123_proj2/word_general/word common common/Users/Mariee/Python/abc_abc_ab_Full_Report_12345-1_R9999_962019_9246"
myfile = 'whatever is your filename.txt'
if not os.path.isdir(mypath):
os.makedirs (mypath)
file_path = os.path.join(mypath, myfile)
print(file_path)
if os.path.exists(file_path) is True:
with open(file_path) as filein:
contents = filein.read()
print(contents)
I tested this code using a long csv file.,Replace the variable myfile with whatever is your file name.