I am working on a project in Python in which I am parsing data from a zipped folder containing log files. The code works fine for most zips, but occasionally this exception is thrown:
[Errno 22] Invalid argument
As a result, the entire file is skipped, thus excluding the data in the desired log files from the results. When I try to extract the zipped file using the default Windows utility, I am met with this error:
Zip error
However, when I try to extract the file with 7zip, it does so successfully, save 2 errors:
1 <path> Unexpected End of Data
2 Data error: x.csv
x.csv is totally unrelated to the log I am trying to parse, and as such, I need to write code that is resilient to the point where if an unrelated file is corrupted, it will still be able to parse the other logs that are not.
At the moment, I am using the zipfile module to extract the files into memory. Is there a robust way to do this without the entire file being skipped?
Update 1: I believe the error I am running into is that the zipfile is missing a footer. I realized this when looking at it in a hex editor. I do not really have any idea how to safely edit the actual file using Python.
Here is the code that I am using to extract zips into memory:
for zip in os.listdir(directory):
try:
if zip.lower().endswith('.zip'):
if os.path.isfile(directory + "\\" + zip):
logs = zipfile.ZipFile(directory + "\\" + zip)
for log in logs.namelist():
if log.endswith('log.txt'):
data = logs.read(log)
Edit 2: Traceback for the error:
Traceback (most recent call last):
File "c:/Users/xxx/Desktop/Python Porjects/PE/logParse.py", line 28, in parse
logs = zipfile.ZipFile(directory + "\\" + zip)
File "C:\Users\xxx\AppData\Local\Programs\Python\Python37\lib\zipfile.py", line 1222, in __init__
self._RealGetContents()
File "C:\Users\xxx\AppData\Local\Programs\Python\Python37\lib\zipfile.py", line 1289, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
The stacktrace seems to show that it's not your code which badly manage to read the file but the Python module managing zip that is raising an error.
It looks like that python zip manager is more strict than other program (see this bug where a user report a difference between python behaviour and other program as GNOME Archive Manager).
Maybe, there is a bug report to do.
Related
I am very new on this platform and I need help reading a file in python. First, it was a docx file because I am using Mac, I converted it into a txt file ,but still have file not found error.
Here is some code:
opdoc = open('PRACT.txt')
print(opdoc.readline())
for each in opdoc :
print(each)
and this the error output: FileNotFoundError: [Errno 2] No such file or directory: 'PRACT.txt'
Unless your file is in the exact same directory as your python file, you'll get this error. Even then, it's best practice to include some sort of relative filepath, like "./PRACT.txt". In general, your issue stems from the fact that Python doesn't know where to look for your file, and where it does know to look, the file isn't there. You need to provide the full path to the file, like:
with open("path/to/file/PRACT.txt", "rb") as f:
# read file etc
Make a new folder and put the txt file in it and also the programme itself. Or just put them in the same directory.
As you have this error ( No such file or directory: 'PRACT.txt') your code couldn't find the file because they were not in the same directory. When you use open("README.txt") the programme is assuming that the text is in the same directory as your code. If you don't want them in the same directory you could also try open(f"[file_path]") and that should work.
Abstract:
I am analysing a pcap file, with live malware (for educational purposes), and using Wireshark - I managed to extract few objects from the HTTP stream and some executables.
During my Analysis, I found instances hinting Fiestka Exploit Kit used.
Having Googled a ton, I came across a GitHub Rep: https://github.com/0x3a/tools/blob/master/fiesta-payload-decrypter.py
What am I trying to achieve?
I am trying to run the python fiesta-payload-decrypter.py against the malicious executable (extracted from the pcap).
What have I done so far?
I've copied the code onto a plain text and saved it as malwaredecoder.py. - This script is saved in the same Folder (/Download/Investigation/) as the malware.exe that I want to run it against.
What's the Problem?
Traceback (most recent call last):
File "malwaredecoder.py", line 51, in <module>
sys.exit(DecryptFiestaPyload(sys.argv[1], sys.argv[2]))
File "malwaredecoder.py", line 27, in DecryptFiestaPyload
fdata = open(inputfile, "rb").read()
IOError: [Errno 2] No such file or directory: '-'
I am running this python script in Kali Linux, and any help would be much appreciated. Thank you.
The script expects two args... What are you passing it?
Looks like it expects the args to be files and it sees a -, (dash), as the input file.
https://github.com/0x3a/tools/blob/master/fiesta-payload-decrypter.py#L44 Here it looks like the first arg is the input file and second is the output file.
Try running it like this:
python malewaredecoder.py /Download/Investigation/fileImInvestigating.pcap /Download/Investigation/out.pcap
All that said, good luck, that script looks pretty old and was last modified in 2015.
I'm using Xbee3 and I want to append data to file.
I try this script for test but I received EEXIST error if TEST.txt file exists. If this file doesn't exist the file is created for the first run but I get same error when I run this script again.
f = open("TEST.txt", 'a')
for a in range(3):
f.write("#EMPTY LINE#\n")
f.close()
Traceback (most recent call last):
File "main", line 1, in
OSError: [Errno 7017] EEXIST
I formatted xbee by the way.
It's sounds like you're using an 802.15.4, DigiMesh or Zigbee module. The file system in those modules is extremely limited, and doesn't allow for modifying existing files. There should be documentation on the product that lists those limitations (no rename, no modify/append, only one open file at a time, etc.)
XBee/XBee3 Cellular modules have a fuller file system implementation that allows for renaming files and modifying file contents.
I'm using xlrd 0.9.4 and I would like verify if the file that I must open is valid.
To do this, I wrote this code in according with this question:
try:
book = xlrd.open_workbook(file_path)
print "Done"
except XLRDError:
print "Wrong type of file."
where file_path is the path of my file.
This works fine, the problem is the following. First of all I have a valid .xls file, so script prints Done. Now, assume that the valid .xls file is renamed (also extension), for example from test.xls to test.txt.
If I run the script, i have the same result (Done).
Instead, if I use a "real" .txt file (empty or with some text), the script prints Wrong type of file.
This behavior happens because the "structure" of the file is not changed? Am I doing something wrong? There is another type of Exception that I can add to except branch?
Thanks in advance
You can see how to xlrd check the file before reading. In xldr source at lines 18-19 defined a «magic» bytes. First bytes of file compared with this byte sequence at line 85. If its not equal exception will be rise. File extention not involved.
Signatures for different file types can be found there.
I'm having trouble with os.walk() in python 2.7.8 on Windows.
When I supply it with a 'normal' path such as "D:\Test\master" it works as expected. However when I supply it with a UNC path such as "\\?\D:\Test\master" it will report the root directory as expected but it will not drill down into the sub directories, nor will it raise an exception.
My research: I read on the help page that os.walk() accepts a function argument to handle errors. By default this argument is None so no error is reported.
I passed a simple function to print the error and received the following for every directory.
def WalkError(Error):
raise Exception(Error)
Stack trace:
Traceback (most recent call last):
File "Compare.py", line 988, in StartServer
for root, dirs, files in os.walk(ROOT_DIR,True,WalkError):
File "C:\Program Files (x86)\Python2.7.8\lib\os.py", line 296, in walk
for x in walk(new_path, topdown, onerror, followlinks):
File "C:\Program Files (x86)\Python2.7.8\lib\os.py", line 281, in walk
onerror(err)
File "Compare.py", line 62, in WalkError
raise Exception(Error)
Exception: [Error 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\D:\\Test\\master\\localization/*.*'
Answer from the original author (originally posted as an edit to the question):
Instant update: In the process of inspecting \lib\os.py, I discovered the error stems from os.listdir(). I searched for the above error message in relation to os.listdir() and found this solution which worked for me.
It looks like if you're going to use UNC style paths with os. modules they need to Unixised (have their \ converted to /). \\\\?\\D:\\Test\\master\\ becomes //?/D:/Test/master/ (note: you no longer need to escape the \ which is handy).
This runs counter to the UNC 'spec' so be aware if you're working with other modules which respect Microsoft's UNC implementation.
(Sorry for the self-solution, I was going to close the tab but felt there was knowledge here which couldn't be found elsewhere.)