I'm trying to get a message that file not found when it doesn't match the file_name*.txt pattern in specific directory.
When calling the script with file_name*.txt argument, all works fine. While entering invalid name file_*.txt throws:
File "etl.py", line 14, in main
path, file = file_path.get_source_file_path()
ValueError: too many values to unpack (expected 2)
Why is this happening?
import fnmatch
import os
class FilePrep:
def __init__(self, path_dir, file_name):
self.path = path_dir
self.file_name = file_name
def get_source_file_path(self):
source_file = []
for file_name in os.listdir(self.path):
if fnmatch.fnmatch(file_name, self.file_name):
source_file.append(file_name)
try:
source_file_name = str(source_file[0])
except IndexError:
return "file not found"
file_path = os.path.join(self.path, source_file_name)
return file_path, source_file_name
main.py
import file_prep
import xml.etree.ElementTree as element_tree
import pandas
import sys
def main():
dir = sys.argv[1]
file_input = sys.argv[2]
#python3 etl.py /home/user/projects/python/folder/ file_name*.xml
file_path = file_prep.FilePrep(dir, file_input)
path, file = file_path.get_source_file_path()
print(path)
Your problem is in this line in main.py:
path, file = file_path.get_source_file_path()
here you are unpacking the return value of get_source_file_path() into two variables. This works fine if the file exists (because you actually return two values) but does not when the file does not exist since you only return one value return "file not found".
To fix this I would raise an error instead of returning a string message in case of failure. Your code can become:
import fnmatch
import os
class FileNotFoundError(Exception):
pass
class FilePrep:
def __init__(self, path_dir, file_name):
self.path = path_dir
self.file_name = file_name
def get_source_file_path(self):
source_file = []
for file_name in os.listdir(self.path):
if fnmatch.fnmatch(file_name, self.file_name):
source_file.append(file_name)
try:
source_file_name = str(source_file[0])
except IndexError:
raise FileNotFoundError(f"file {self.file_name} not found")
file_path = os.path.join(self.path, source_file_name)
return file_path, source_file_name
import file_prep
import xml.etree.ElementTree as element_tree
import pandas
import sys
def main():
dir = sys.argv[1]
file_input = sys.argv[2]
#python3 etl.py /home/user/projects/python/folder/ file_name*.xml
file_path = file_prep.FilePrep(dir, file_input)
try:
path, file = file_path.get_source_file_path()
print(path)
except file_prep.FileNotFoundError as e:
print(e)
The previous answers somewhat got it the wrong way, saying in case of failure, "only one" value is returned. While that's true, it doesn't explain why the error message says there are "too many values to unpack", since 1 is not more than 2. The reason is that that one value is a string, which, due to to the multi-assignment, will be treated as an iterable. Since it has 14 characters, that's 14 values, and that's too many.
The issue is this following line:
except IndexError:
return "file not found"
When you hit that except condition your return from your function call returns only one variable, that string. You could modify your condition to do the following:
except IndexError:
source_file_name = "file not found"
However this won't work as you try to use this file name later on, a better approach would be to raise an error properly as shown in this answer by Matteo here.
When you call the function here, it's expected that two values will be returned
path, file = file_path.get_source_file_path()
When an exception occurs here, only one value is being returned which is causing the error
try:
source_file_name = str(source_file[0])
except IndexError:
return "file not found"
There are two ways to fix this.
Make it so only one value is expected to be returned and you modify the return at the bottom to only pass one value
You modify the exception so it returns two values
Related
I'm trying read xlsx file but gets me ValueError
zero-size array to reduction operation fmin which has no identity
views.py
def t(request):
context = {}
if request.method == "POST":
uploaded_file = request.FILES['document']
print(uploaded_file)
if uploaded_file.name.endswith('.xlsx'):
savefile = FileSystemStorage()
name = savefile.save(uploaded_file.name, uploaded_file)
d = os.getcwd()
file_directory = d + '\media\\' + name
readfile(file_directory)
return redirect(r)
else:
messages.warning(request, 'File was not uploaded. Please use xlsx file extension!')
return render(request, 't.html', {})
def read_file(filename):
my_file = pd.read_excel(filename)
data = pd.DataFrame(data=my_file)
Note: It reads some files correctly
To avoid the error pointed out in the comments wrap the line of code with try/except. Check your logs which line throws the error, I think it is this one:
try:
my_file = pd.read_excel(filename)
except ValueError:
messages.error(request, "The file has the wrong format.")
# or whatever error message you want to raise
Ideally, you want to raise the error before saving the file. I would try to read the file with pd before saving it:
# ...
uploaded_file = request.FILES['document']
if uploaded_file.name.endswith('.xlsx'):
try:
_read_test = pd.read_excel(uploaded_file)
except ValueError:
messages.error(request, "The file has the wrong format.")
# or whatever error message you want to raise
return redirect('your-error-or-form-page')
savefile = FileSystemStorage()
# ...
I'm not sure if pd can read the InMemoryFile but that's the line where you want to catch wrong files. Maybe there's already a pypi packages for cases like this
I'm trying to rewrite some code for learning purposes and got stuck with implementing try/except part into the code.
Class FilePrep takes two arguments (file_name and path_dir`), the loop checks whether the file exists and returns entire path.
How to properly implement the handler part so error message will be clear rather then throwing list index out of range?
import xml.etree.ElementTree as element_tree
import fnmatch
import os
import errno
class FilePrep:
def __init__(self, path_dir, file_name):
self.path = path_dir
self.file_name = file_name
def get_source_file_path(self):
source_file = []
for file_name in os.listdir(self.path):
try:
if fnmatch.fnmatch(file_name, self.file_name):
source_file.append(file_name)
except IndexError:
print("file not found")
source_file_old_name = str(source_file[0])
file_path = os.path.join(self.path, source_file_old_name)
return file_path
Function.py
import file_prep
file_path = file_prep.FilePrep('path', 'file_name*.xml')
print(file_path.get_source_file_path())
mainly problem into the below line:
source_file_old_name = str(source_file[0])
you can use below solution:
try:
source_file_old_name = str(source_file[0])
except IndexError:
return ""
file_path = os.path.join(self.path, source_file_old_name)
return file_path
Your try/except-block is placed at the wrong place.
Actually, the error occurs, when you're trying to access source_file[0], which is an empty list ([]) in case, that no file exists which matches the specified filename.
Therefore, I suggest to change it to the following implementation, where the lenght of the list, which should contain the matched files, is checked. If it is empty, a FileNotFoundError will be raised, like so:
if not source_file:
raise FileNotFoundError(f"No files matching '{self.file_name}'")
This results in the following class:
import xml.etree.ElementTree as element_tree
import fnmatch
import os
import errno
class FilePrep:
def __init__(self, path_dir, file_name):
self.path = path_dir
self.file_name = file_name
def get_source_file_path(self):
source_file = []
for file_name in os.listdir(self.path):
if fnmatch.fnmatch(file_name, self.file_name):
source_file.append(file_name)
if not source_file:
raise FileNotFoundError(f"No files matching '{self.file_name}'")
source_file_old_name = str(source_file[0])
file_path = os.path.join(self.path, source_file_old_name)
return file_path
I am currently working on a file sorting program and now that I have the selection of the Folders finished I need to assign the right paths to the variables to use them later on to move the files. My code right now looks like this
import os
import promptlib
sourcepath = str()
destpath = str()
pathimg = str()
pathtxt = str()
pathaudio = str()
pathvideo = str()
pathexe = str()
pathzips = str()
def get_paths():
global sourcepath
global destpath
sourcepath = promptlib.Files().dir()+"\\"
destpath = promptlib.Files().dir()+"\\"
def check_or_create_folder():
get_paths()
listToCreate = ["images\\","text documents\\","audio files\\","video files\\","executables\\","zips\\"]
pathToCreate = destpath+"sortedDownloads\\"
global paths
try: os.mkdir(pathToCreate)
except OSError: return
for elememt in listToCreate:
try: os.mkdir(pathToCreate+elememt)
except OSError: break
I thought about storing them in an array and then declaring them element by element maybe something like
def check_or_create_folder():
paths = [global pathimg, global pathtxt]
listToCreate = ["images\\","text documents\\","audio files\\","video files\\","executables\\","zips\\"]
pathToCreate = destpath+"sortedDownloads\\"
for i in range(len(listToCreate)):
try: os.mkdir(pathToCreate+listToCreate[i])
paths[i] = pathToCreate+listToCreate[i]
except OSError: break
but that doesn't work, I tried searching it up but I couldn't find anything so what would be your approach to this problem?
thank you before for answering.
(i am sorry for my English I'm still 15 years old and it's not that good)
Don't use global variables. Instead of using string operations to combine pathes, use pathlib.Path.
To store your created folders, just use a list:
import promptlib
from pathlib import Path
SUB_FOLDERS = ["images", "text documents", "audio files", "video files", "executables", "zips"]
def get_paths():
sourcepath = Path(promptlib.Files().dir())
destpath = Path(promptlib.Files().dir())
return sourcepath, destpath
def check_or_create_folder():
sourcepath, destpath = get_paths()
folders = []
for folder in SUB_FOLDERS:
folder = destpath / "sortedDownloads" / folder
folder.mkdir(parents=True, exist_ok=True)
folders.append(folder)
return sourcepath, folders
Currently am creating files using the below code,I want to create a directory based on the timestamp at that point in the cwd,save the directory location to a variable and then create the file in the newly created directory,does anyone have ideas on how can this be done?
def filecreation(list, filename):
#print "list"
with open(filename, 'w') as d:
d.writelines(list)
def main():
list=['1','2']
filecreation(list,"list.txt")
if __name__ == '__main__':
main()
You mean, something like this?
import os, datetime
mydir = os.path.join(os.getcwd(), datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
os.makedirs(mydir)
with open(os.path.join(mydir, 'filename.txt'), 'w') as d:
pass # ... etc ...
Complete function
import errno
import os
from datetime import datetime
def filecreation(list, filename):
mydir = os.path.join(
os.getcwd(),
datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
try:
os.makedirs(mydir)
except OSError as e:
if e.errno != errno.EEXIST:
raise # This was not a "directory exist" error..
with open(os.path.join(mydir, filename), 'w') as d:
d.writelines(list)
Update: check errno.EEXIST constant instead of hard-coding the error number
Can someone tell me if the following function declaration is the correct way to pass a relative path to a function? The call is only taking one variable. When I include a second variable (absolute path), my function does not work.
def extract(tar_url, extract_path='.'):
The call that does not work:
extract(chosen, path)
This works, but does not extract:
extract(chosen)
Full Code:
def do_fileExtract(self, line):
defaultFolder = "Extracted"
if not defaultFolder.endswith(':') and not os.path.exists('c:\\Extracted'):
os.mkdir('c:\\Extracted')
raw_input("PLACE .tgz FILES in c:\Extracted AT THIS TIME!!! PRESS ENTER WHEN FINISHED!")
else:
pass
def extract(tar_url, extract_path='.'):
print tar_url
tar = tarfile.open(tar_url, 'r')
for item in tar:
tar.extract(item, extract_path)
if item.name.find(".tgz") != -1 or item.name.find(".tar") != -1:
extract(item.name, "./" + item.name[:item.name.rfind('/')])
userpath = "Extracted"
directory = os.path.join("c:\\", userpath)
os.chdir(directory)
path=os.getcwd() #Set log path here
dirlist=os.listdir(path)
files = [fname for fname in os.listdir(path)
if fname.endswith(('.tgz','.tar'))]
for item in enumerate(files):
print "%d- %s" % item
try:
idx = int(raw_input("\nEnter the file's number:\n"))
except ValueError:
print "You fail at typing numbers."
try:
chosen = files[idx]
except IndexError:
print "Try a number in range next time."
newDir = raw_input('\nEnter a name to create a folder a the c: root directory:\n')
selectDir = os.path.join("c:\\", newDir)
path=os.path.abspath(selectDir)
if not newDir.endswith(':') and not os.path.exists(selectDir):
os.mkdir(selectDir)
try:
extract(chosen, path)
print 'Done'
except:
name = os.path.basename(sys.argv[0])
print chosen
It looks like you missed an escape character in "PLACE .tgz FILES in c:\Extracted AT THIS TIME!!! PRESS ENTER WHEN FINISHED!"
I don't think raw_input sees the prompt string as a raw string, just the user input.
But this shouldn't affect the functionality of your program.
Are you on Unix or windows? I was under the impression that the on Unix you use / forward slash instead of \\ backslash as a separator.
I tested some code on this file:
http://simkin.asu.edu/geowall/mars/merpano0.tar.gz
The following code:
>>> from os import chdir
>>> import tarfile
>>> chdir(r'C:\Users\Acer\Downloads')
>>> tar_url = 'merpano0.tar.gz'
>>> print tar_url
merpano0.tar.gz
>>> tar = tarfile.open(tar_url, 'r')
>>> extract_path = 'C:\\Users\\Acer\\Downloads\\test\\'
>>> for item in tar:
tar.extract(item, extract_path)
executed cleanly with no problems on my end. In the test directory I got a single folder with some files, exactly as in the original tar file. Can you explain what you're doing differently in your code that might be bugging up?