ValueError: substring not found in midi - python

Let me first start this off by stating that I am a complete beginner to coding and my attempts to fix this have been limited. I am trying to follow this Arduino controlled piano robot. It takes a textified midi file and uses python to translate it into 8-bit. The code is attached near the bottom of the link, I had some formatting issues when placing it here.
This link to the textified midi file used. before running the code I changed the input_file = open to text file path like so,
input_file = open("C:\\Users\\nby20\\Downloads\\megalovania.txt")
After running the code I get a text output file as expected however it is blank and I get a few errors:
Traceback (most recent call last):
File "C:\Users\nby20\Downloads\python_code_for_translation.py", line 184, in <module>
main()
File "C:\Users\nby20\Downloads\python_code_for_translation.py", line 23, in main
result[-1] = str(temp_time) + "," + set_bit_prev(on_off_finder(a), note_finder(a), -1)
File "C:\Users\nby20\Downloads\python_code_for_translation.py", line 178, in on_off_finder
end = in_string.index("ch=") - 1
ValueError: substring not found
Any suggestions on how to fix this would be greatly appreciated.

The Traceback is like debugging information you can use to trace which functions were called when the error was thrown. It seems the error occurred when it was executing this bit of conditional logic, lines 22-23, of the main function:
elif time_finder_comm(result[-1]) == temp_time:
result[-1] = str(temp_time) + "," + set_bit_prev(on_off_finder(a), note_finder(a), -1)
which called the on_off_finder function which just tries to figure out if the line says 'On' or 'Off'.
It seems the file reader only expects lines like this:
55248 Off ch=10 n=40 v=64
However, in the file you uploaded, there also lines like this:
55248 Meta TrkEnd
TrkEnd
The index function throws ValueError: substring not found if the substring passed in does not exist in the string, which in this case (line 178 below) is the string "ch":
end = in_string.index("ch=") - 1
Try removing those kind of lines and re-run the script? Find all lines with "Trk" and and remove them, or make 3 separate files because there seem to be 3 blocks of lines in 'megalovania.txt' that will trip up the script:
(starting at line 2469):
55248 Meta TrkEnd
TrkEnd
MTrk
...
(starting at line 4071):
58368 Meta TrkEnd
TrkEnd
MTrk
...
(starting at line 6431):
55296 Meta TrkEnd
TrkEnd

Related

Python Read Excel file with Error - "We Found a problem with some content..."

Here is my problem. We have an Excel based report that business users enter comments into two separate fields, as well as selecting a code form a drop down. We then have a manual process that collects those files and pushes the comments and codes to a Snowflake table to be able to use in various reports.
I am trying to improve the process with a Python script that will collect the files, copy them to a staging_folder location, then read in the data from the sheet, append it all together, do some cleanup and push to Snowflake. The plan is that this would be completely automated - but this is where we run into issues.
Initial step works perfectly. I have a loop that grabs the files based on the previous business day date, copies them to a staging folder. There are typically 32 files each day.
Next step reads those files to append to a dataframe. Here is the function that is loading the Excel files in my Python script.
def load_files():
file_list = glob.glob(file_path + r'\*')
df = pd.DataFrame()
print("Importing data to Pandas DF...")
for file in file_list:
try:
wb = load_workbook(file)
ws = wb["Daily Outs"]
data = ws.values
cols = next(data)[1:]
data = list(data)
idx = [r[0] for r in data]
data = (islice(r, 1, None) for r in data)
data_1 = pd.DataFrame(data, index=idx, columns=cols)
df = df.append(data_1, sort=False)
print(file + " Imported to Df...")
except Exception as e:
print("Error: " + e + " When attempting to open file: " + file)
# error_notify(e)
print(df.head(10))
return df
The problem is when we have files that have some sort of corruption. The files when opened manually will show an error like the one below.
I thought with my try, except code above this would catch an error like this and alert me with the error_notify(e) function. However, we get a result where the Python script crashes with an error like this: zipfile.BadZipFile: File is not a zip file
During handling of the above exception, another exception occurred.
There is more to the error, but I only copied & pasted this part in some communication with some folks int he office. Impossible to replicate the error on our own - I have no idea how the files get corrupted in this way - except that there are multiple people accessing the files throughout the day.
The way to make the file readable is completely manual - we must open the file, get that error, hit yes, and save the file over the existing one. Then re-launch the script. But since the try, except isn't catching it and alerting us to the failure, we have to run the script manually to see if it works or not.
Two questions - am I doing something incorrect in my try, except command? I am admittedly weak in error catching so my first thought is there is more I can do there to make that work. Secondly, is there a Python way to get past that error in the Excel workbook files?
Here is the error text:
Traceback (most recent call last):
File "G:/Replenishment/Reporting/00 - I&A Replenishment/02 - Service
Level/Daily Outs Comment Capture/Python/daily_outs_missed_files.py", line 48, in load_files
wb = load_workbook(file)
File "C:\ProgramData\Anaconda3\lib\site-packages\openpyxl\reader\excel.py", line 314, in load_workbook
data_only, keep_links)
File "C:\ProgramData\Anaconda3\lib\site-packages\openpyxl\reader\excel.py", line 124, in init
self.archive = _validate_archive(fn)
File "C:\ProgramData\Anaconda3\lib\site-packages\openpyxl\reader\excel.py", line 96, in _validate_archive
archive = ZipFile(filename, 'r')
File "C:\ProgramData\Anaconda3\lib\zipfile.py", line 1222, in init
self._RealGetContents()
File "C:\ProgramData\Anaconda3\lib\zipfile.py", line 1289, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "G:/Replenishment/Reporting/00 - I&A Replenishment/02 - Service Level/Daily Outs Comment Capture/Python/daily_outs_missed_files.py", line 123, in <module>
main()
File "G:/Replenishment/Reporting/00 - I&A Replenishment/02 - Service Level/Daily Outs Comment Capture/Python/daily_outs_missed_files.py", line 86, in main
df_output = df_clean()
File "G:/Replenishment/Reporting/00 - I&A Replenishment/02 - Service Level/Daily Outs Comment Capture/Python/daily_outs_missed_files.py", line 68, in df_clean
df = load_files()
File "G:/Replenishment/Reporting/00 - I&A Replenishment/02 - Service Level/Daily Outs Comment Capture/Python/daily_outs_missed_files.py", line 61, in load_files
print("Error: " + e + " When attempting to open file: " + file)
TypeError: can only concatenate str (not "BadZipFile") to str
Your try/except code looks correct. All user defined exceptions in python should be classes based on Exception. See BaseException and
and Exception in python documentation :
"Exception (..) All user-defined exceptions should also be derived from this class" see also the exception class hierarchy tree at the end of the python doc sesction.
If your python script "crashes" it means one of the library procedures throws an exception which is not based on the Exception class, something that "should not" be. You could look at the Traceback and try catching the offending exception type separately, or find what part of the source code and which library is the cause, fix it and submit a PR. Here are two examples of a good and bad way of deriving own exceptions
class MyBadError(BaseException):
"""
my bad exception, do not make yours that way
"""
pass
instead of recommended
class MyGoodError(Exception):
"""
exception based on the Exception
"""
pass
Where and what exactly fails is a bit of mystery still but the problems with your exception from the Traceback is not new, see zipfile.BadZipfile issue in pandas discussion. Note that xlrd used by pandas to read Excel workbooks data is currently a "no-maintainer-ware" declaration about xlrd from the authors and in case of any issues the recommendation is to use openpyxl instead or fix any issues yourself (pandas maintainers are doing pontius pilate on that, but happily use xlrd as a dependency). I suggest you catch the BadZipfile as a special known corruption error separately from all other exceptions, see python error handling tutorial for example code (you probably already have seen it, this is for other readers). If that does not work I can trace it in the source code of your libraries / python modules to the exact offending section and find the culprit, if you reach out directly.

Problems with pickle python

I recently made a program using an external document with pickle. But when it tries to load the file with pickle, I got this error (the file is already existing but it also fails when the file in't existing):
python3.6 check-graph_amazon.py
a
b
g
URL to follow www.amazon.com
Product to follow Pool_table
h
i
[' www.amazon.com', ' Pool_table', []]
p
Traceback (most recent call last):
File "check-graph_amazon.py", line 17, in <module>
tab_simple = pickle.load(doc_simple)
io.UnsupportedOperation: read
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "check-graph_amazon.py", line 42, in <module>
pickle.dump(tab_simple, 'simple_data.dat')
TypeError: file must have a 'write' attribute
Here is the code :
import pickle5 as pickle
#import os
try:
print("a")
with open('simple_data.dat', 'rb') as doc_simple:
print("b")
tab_simple = pickle.load(doc_simple)
print("c")
print(tab_simple)
print("d")
URL = tab_simple[0]
produit_nom = tab_simple[1]
tous_jours = tab_simple[2]
print("f")
except :
print("g")
URL = str(input("URL to follow"))
produit_nom = str(input("Product to follow"))
with open('simple_data.dat', 'wb+') as doc_simple:
print("h")
#os.system('chmod +x simple_data.dat')
tab_simple = []
tab_simple.append(URL)
tab_simple.append(produit_nom)
tab_simple.append([])
print(tab_simple)
print("c'est le 2")
print("p")
pickle.dump(tab_simple, 'simple_data.dat')
print("q")
The prints are here to show which lines are executed. The os.system is here to allow writing on the file but the error is persisting.
I don't understand why it's said that the document doesn't have a write attribute because I opened it in writing mode. And I neither understand the first error where it can't load the file.
If it can help you the goal of this script is to initialise the program, with a try. It tries to open the document in reading mode in the try part and then set variables. If the document doesn't exist (because the program is lauched for the first time) it goes in the except part and create the document, before writing informations on it.
I hope you will have any clue, including changing the architecture of the code if you have a better way to make an initialisation for the 1st time the program is launched.
Thanks you in advance and sorry if the code isn't well formated, I'm a beginner with this website.
Quote from the docs for pickle.dump:
pickle.dumps(obj, protocol=None, *, fix_imports=True)
Write a pickled representation of obj to the open file object file. ...
...
The file argument must have a write() method that accepts a single bytes argument. It can thus be an on-disk file opened for binary writing, an io.BytesIO instance, or any other custom object that meets this interface.
So, you should pass to this function a file object, not a file name, like this:
with open("simple_data.dat", "wb"): as File:
pickle.dump(tab_simple, File)
Yeah, in your case the file has already been opened, so you should write to doc_simple.

P4Python run method does not work on empty folder

I want to search a Perforce depot for files.
I do this from a python script and use the p4python library command:
list = p4.run("files", "//mypath/myfolder/*")
This works fine as long as myfolder contains some files. I get a python list as a return value. But when there is no file in myfolder the program stops running and no error message is displayed. My goal is to get an empty python list, so that I can see that this folder doesn't contain any files.
Does anybody has some ideas? I could not find information in the p4 files documentation and on StackOverflow.
I'm going to guess you've got an exception handler around that command execution that's eating the exception and exiting. I wrote a very simple test script and got this:
C:\Perforce\test>C:\users\samwise\AppData\local\programs\python\Python36-32\python files.py
Traceback (most recent call last):
File "files.py", line 6, in <module>
print(p4.run("files", "//depot/no such path/*"))
File "C:\users\samwise\AppData\local\programs\python\Python36-32\lib\site-packages\P4.py", line 611, in run
raise e
File "C:\users\samwise\AppData\local\programs\python\Python36-32\lib\site-packages\P4.py", line 605, in run
result = P4API.P4Adapter.run(self, *flatArgs)
P4.P4Exception: [P4#run] Errors during command execution( "p4 files //depot/no such path/*" )
[Error]: "//depot/no such path/* - must refer to client 'Samwise-dvcs-1509687817'."
Try something like this ?
import os
if len(os.listdir('//mypath/myfolder/') ) == 0: # Do not execute p4.run if directory is empty
list = []
else:
list = p4.run("files", "//mypath/myfolder/*")

struct.error: argument for 's' must be a string

i'm working on a authentication system with raspberry pi with the Fingerprint Scanner - TTL (GT-511C3) and i have a function that create a template and returns as result the following :
I want to extract only th data part so i did like this : response 1['Data']
but when i print it i get this :
i added this as header of my file :
reload(sys)
sys.setdefaultencoding('utf8')
and i tried a couple of solutions i found in the forum but i couldn't find one that works with me.
my problem is when i try to compare the created template with the ones already in the database of the scanner with a function called IdentifyTemplate and takes as a parameter a template it gives me the following error :
Traceback (most recent call last):
File "finger.py", line 169, in
indetifyResponse = f.IdentifyTemplate(makeTemplateResponse)
File "/home/pi/Desktop/fingerpi/fingerpi/fingerpi.py", line 213, in
IdentifyTemplate
if self.sendData(template, 498):
File "/home/pi/Desktop/fingerpi/fingerpi/fingerpi.py", line 53, in sendData
packet = encode_data_packet(data, data_len, device_id = self.device_id)
File "/home/pi/Desktop/fingerpi/fingerpi/base.py", line 72, in
encode_data_packet
data # Data to be sent
struct.error: argument for 's' must be a string
i suspect that the problem is comming from that Data that i'm trying to print.
Any help is much appreciated. My Python version is 2.7.13

Error when using astWCS trying to create WCS object

I'm running python2.5 and trying to use the astLib library to analyse WCS information in astronomical images. I try and get the object instanciated with the following skeleton code:
from astLib import astWCS
w = astWCS.WCS('file.fits') # error here
where file.fits is a string pointing to a valid fits file.
I have tried using the alternate method of passing a pyfits header object and this fails also:
import pyfits
from astLib import astWCS
f = pyfits.open('file.fits')
header = f[0].header
f.close()
w = astWCS.WCS(header, mode='pyfits') # error here also
The error is this:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/astro/phrfbf/build/lib/python2.6/site-packages/astLib/astWCS.py", line 79, in __init__
self.updateFromHeader()
File "/home/astro/phrfbf/build/lib/python2.6/site-packages/astLib/astWCS.py", line 119, in updateFromHeader
self.WCSStructure=wcs.wcsinit(cardstring)
File "/home/astro/phrfbf/build/lib/python2.6/site-packages/PyWCSTools/wcs.py", line 70, in wcsinit
return _wcs.wcsinit(*args)
TypeError: in method 'wcsinit', argument 1 of type 'char *'
When I run in ipython, I get the full error here on the pastebin
I know the astWCS module is a wrapped version of WCStools but i'd prefer to use the Python module as the rest of my code is in Python
Can anyone help with this problem?
Just found out the updated version of this library has fixed the problem, thanks for everyone's help
Oh sorry, I should have seen. Looking at the pastebin in more detail, the only error I can think of is that, for some reason the header has unicode in it. It can't be converted to char *, and you get the error. I tried searching for something in the header, but everything looks okay. Can you do this and post the output in another pastebin?
import pyfits
f = pyfits.open('file.fits')
header = f[0].header
f.close()
for x, i in enumerate(header.iteritems()):
if len(str(i[1])) >= 70:
print x, str(i[1])
cardlist = header.ascardlist()
cardstring = ""
for card in cardlist:
cardstring = cardstring + str(card)
print repr(cardstring)
Or, if you can check the header of your fits file for "funny" characters, getting rid of them should solve the issue.

Categories

Resources