I am trying to acquire all files from a windows VHD.
When calling .info or .info.meta.type on a file entry it doesn't always work and throws an error. Is there any way to retrieve these files?
filesystemObject = pytsk3.FS_Info(imagehandle, offset=(partition.start * metric.block_size))
except:
print "Partition has no supported file system"
continue
print "File System Type Dectected ", filesystemObject.info.ftype
directoryObject = filesystemObject.open_dir(path=dirPath)
for entryObject in directoryObject:
if (not hasattr(entryObject, "info") or
not hasattr(entryObject.info, "name") or
not hasattr(entryObject.info.name, "name") or
entryObject.info.name.name in [".", ".."]):
continue
try:
f_type = entryObject.info.meta.type
except:
print "Cannot retrieve type of", entryObject.info.name.name
metric.total_files += 1
metric.failed_files += 1
continue
Traceback:
> Traceback (most recent call last): File "/Users/paulledwith/PycharmProjects/FYP/Server.py", line 352, in <module>
directoryRecurse(directoryObject, []) File "/Users/paulledwith/PycharmProjects/FYP/Server.py", line 185, in directoryRecurse
directoryRecurse(sub_directory, parentPath) File "/Users/paulledwith/PycharmProjects/FYP/Server.py", line 185, in directoryRecurse
directoryRecurse(sub_directory, parentPath) File "/Users/paulledwith/PycharmProjects/FYP/Server.py", line 185, in directoryRecurse
directoryRecurse(sub_directory, parentPath) File "/Users/paulledwith/PycharmProjects/FYP/Server.py", line 172, in directoryRecurse
print entryObject.info.meta.size AttributeError: 'NoneType' object has no attribute 'size'
This does retrieve about 90% of the files on the 5gb VHD I am working on but 90% is as good as 0.
Related
I`m trying to download and then open excel file (report) generated by marketplace with openpyxl.
import requests
import config
import openpyxl
link = 'https://api.telegram.org/file/bot' + config.TOKEN + '/documents/file_66.xlsx'
def save_open(link):
filename = link.split('/')[-1]
r = requests.get(link)
with open(filename, 'wb') as new_file:
new_file.write(r.content)
wb = openpyxl.open ('file_66.xlsx')
ws = wb.active
cell = ws['B2'].value
print (cell)
save_open(link)
After running this code I got the above:
Traceback (most recent call last):
File "C:\Python 3.9\lib\site-packages\openpyxl\descriptors\base.py", line 55, in _convert
value = expected_type(value)
TypeError: Fill() takes no arguments
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Home\Documents\myPython\bot_WB\main.py", line 20, in <module>
save_open(link)
File "C:\Users\Home\Documents\myPython\bot_WB\main.py", line 14, in save_open
wb = openpyxl.open ('file_66.xlsx')
File "C:\Python 3.9\lib\site-packages\openpyxl\reader\excel.py", line 317, in load_workbook
reader.read()
File "C:\Python 3.9\lib\site-packages\openpyxl\reader\excel.py", line 281, in read
apply_stylesheet(self.archive, self.wb)
File "C:\Python 3.9\lib\site-packages\openpyxl\styles\stylesheet.py", line 198, in apply_stylesheet
stylesheet = Stylesheet.from_tree(node)
File "C:\Python 3.9\lib\site-packages\openpyxl\styles\stylesheet.py", line 103, in from_tree
return super(Stylesheet, cls).from_tree(node)
File "C:\Python 3.9\lib\site-packages\openpyxl\descriptors\serialisable.py", line 103, in from_tree
return cls(**attrib)
File "C:\Python 3.9\lib\site-packages\openpyxl\styles\stylesheet.py", line 74, in __init__
self.fills = fills
File "C:\Python 3.9\lib\site-packages\openpyxl\descriptors\sequence.py", line 26, in __set__
seq = [_convert(self.expected_type, value) for value in seq]
File "C:\Python 3.9\lib\site-packages\openpyxl\descriptors\sequence.py", line 26, in <listcomp>
seq = [_convert(self.expected_type, value) for value in seq]
File "C:\Python 3.9\lib\site-packages\openpyxl\descriptors\base.py", line 57, in _convert
raise TypeError('expected ' + str(expected_type))
TypeError: expected <class 'openpyxl.styles.fills.Fill'>
[Finished in 1.6s]
If you run file properties/details you can see that this file was generated by "Go Exelize" (author: xuri). To run this file you need to separate code in two parts. First: download file. Then you need to manually open it with MS Excel, save file and close it (after this "Go Excelize" switch to "Microsoft Excel"). And only after that you can run the second part of the code correctly with no errors. Can anyone help me to handle this problem?
I had the same problem, "TypeError('expected ' + str(expected_type))", using pandas.read_excel, which uses openpyxl. If I open the file, save and close it, it will work with both, pandas and openpyxl.
Upon further attempts I could open the file using the "read_only=True" in openpyxl, but while iterating over the rows I would still get the error, but only when all the rows ended, in the end of the file.
I belive it could be something in the EOF (end of file) and openpyxl don't have ways of treating it.
Here is the code that I used to test and worked for me:
import openpyxl
wb = openpyxl.load_workbook(my_file_name, read_only=True)
ws = wb.worksheets[0]
lis = []
try:
for row in ws.iter_rows():
lis.append([cell.value for cell in row])
except TypeError:
print('Skip error in EOF')
Used openpyxl==3.0.10
I have to read from a OrientDB. To test that everything works I tried to read from the Database with the SELECT Statement.
like this:
import pyorient
client = pyorient.OrientDB("adress", 2424)
session_id = client.connect("root", "password")
client.db_open("table","root","password")
print str(client.db_size())
client.query("SELECT * FROM L1_Req",1)
The Connection works fine and also the print str(client.db_size()) line.
But at client.query("SELECT * FROM L1_Req",1) it returns the following Error Message:
Traceback (most recent call last):
File "testpy.py", line 9, in <module>
client.query("SELECT * FROM L1_Req",1)
File "C:\app\tools\python27\lib\site-packages\pyorient\orient.py", line 470, i
n query
.prepare(( QUERY_SYNC, ) + args).send().fetch_response()
File "C:\app\tools\python27\lib\site-packages\pyorient\messages\commands.py",
line 144, in fetch_response
super( CommandMessage, self ).fetch_response()
File "C:\app\tools\python27\lib\site-packages\pyorient\messages\base.py", line
265, in fetch_response
self._decode_all()
File "C:\app\tools\python27\lib\site-packages\pyorient\messages\base.py", line
249, in _decode_all
self._decode_header()
File "C:\app\tools\python27\lib\site-packages\pyorient\messages\base.py", line
176, in _decode_header
serialized_exception = self._decode_field( FIELD_STRING )
File "C:\app\tools\python27\lib\site-packages\pyorient\messages\base.py", line
366, in _decode_field
_decoded_string = self._orientSocket.read( _len )
File "C:\app\tools\python27\lib\site-packages\pyorient\orient.py", line 164, i
n read
buf = bytearray(_len_to_read)
MemoryError
I also tried some ohter SQL Statements like:
client.query("SELECT subSystem FROM L1_Req",1)
I cant't get why this happends. Can you guys help me ?
I have these "json" files that I like to insert into my mongodb database.
An example of one is:
http://s.live.ksmobile.net/cheetahlive/de/ff/15201023827214369775/15201023827214369775.json
The problem is, that it is formated like this:
{ "channelType":"TEMPGROUP", ... } # line 1
{ "channelType":"TEMPGROUP", ... } # line 2
So instead of inserting it as 1 document in the DB, it insert every single line as 1 entry. That ends up with what should be 3 documents from 3 "json" files in the database become 1189 documents in the database instead.
How can I insert the whole content of the ".json" into one document?
My code is:
replay_url = "http://live.ksmobile.net/live/getreplayvideos?"
userid = 969730808384462848
url2 = replay_url + urllib.parse.urlencode({'userid': userid}) + '&page_size=1000'
raw_replay_data = requests.get(url2).json()
for i in raw_replay_data['data']['video_info']:
url3 = i['msgfile']
raw_message_data = urllib.request.urlopen(url3)
for line in raw_message_data:
json_data = json.loads(line)
messages.insert_one(json_data)
print(json_data)
Update to give more information to answer
messages.insert(json_data) gives this error:
Traceback (most recent call last):
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 633, in _insert
blk.execute(concern, session=session)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 432, in execute
return self.execute_command(generator, write_concern, session)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 329, in execute_command
raise BulkWriteError(full_result)
pymongo.errors.BulkWriteError: batch op errors occurred
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/import_messages_dev.py", line 43, in <module>
messages.insert(json_data)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 2941, in insert
check_keys, manipulate, write_concern)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 635, in _insert
_raise_last_error(bwe.details)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/helpers.py", line 220, in _raise_last_error
_raise_last_write_error(write_errors)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/helpers.py", line 188, in _raise_last_write_error
raise DuplicateKeyError(error.get("errmsg"), 11000, error)
pymongo.errors.DuplicateKeyError: E11000 duplicate key error index: liveme.messages.$_id_ dup key: { : ObjectId('5aa2fc6f5d60126499060949') }
messages.insert_one(json_data) gives me this error:
Traceback (most recent call last):
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/import_messages_dev.py", line 43, in <module>
messages.insert_one(json_data)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 676, in insert_one
common.validate_is_document_type("document", document)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/common.py", line 434, in validate_is_document_type
"collections.MutableMapping" % (option,))
TypeError: document must be an instance of dict, bson.son.SON, bson.raw_bson.RawBSONDocument, or a type that inherits from collections.MutableMapping
messages.insert_many(json_data) gives me this error:
Traceback (most recent call last):
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/import_messages_dev.py", line 43, in <module>
messages.insert_many(json_data)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/collection.py", line 742, in insert_many
blk.execute(self.write_concern.document, session=session)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 432, in execute
return self.execute_command(generator, write_concern, session)
File "/media/anon/06bcf743-8b4d-409f-addc-520fc4e19299/PycharmProjects/LiveMe/venv1/lib/python3.6/site-packages/pymongo/bulk.py", line 329, in execute_command
raise BulkWriteError(full_result)
pymongo.errors.BulkWriteError: batch op errors occurred
messages.insert and messages.insert_many both insert 1 line and throw the error.
These files obviously do not contain properly formatted json - rather they contain a separate object on each line.
To turn them into valid json, you probably want a list of objects, i.e.:
[{ "channelType":"TEMPGROUP", ... },
{ "channelType":"TEMPGROUP", ... }]
You can achieve this by doing:
for i in raw_replay_data['data']['video_info']:
url3 = i['msgfile']
raw_message_data = urllib.request.urlopen(url3)
json_data = []
for line in raw_message_data:
json_data.append(json.loads(line))
messages.insert_one(json_data)
print(json_data)
I am trying to modify calendars using the python caldav library (version 0.5.0) on python 2.7. I am using the following code snippet to add an exdate to recurring events
for event in results:
vevent = event.instance.vevent
if vevent.status.value != "CANCELLED":
if (vevent.summary.value, vevent.dtstart.value) in cancelled:
ev = calendar.event_by_uid(vevent.uid.value)
ev.instance.vevent.add("exdate").value = vevent.dtstart.value
print ev.instance.vevent.serialize()
Although the vobject library claims support for datetime objects the last line fails with:
Traceback (most recent call last):
File "./caldavTest.py", line 31, in <module>
print ev.instance.vevent.serialize()
File "/usr/lib/python2.7/site-packages/vobject/base.py", line 254, in serialize
return behavior.serialize(self, buf, lineLength, validate)
File "/usr/lib/python2.7/site-packages/vobject/behavior.py", line 166, in serialize
out = base.defaultSerialize(transformed, buf, lineLength)
File "/usr/lib/python2.7/site-packages/vobject/base.py", line 1007, in defaultSerialize
child.serialize(outbuf, lineLength, validate=False)
File "/usr/lib/python2.7/site-packages/vobject/base.py", line 254, in serialize
return behavior.serialize(self, buf, lineLength, validate)
File "/usr/lib/python2.7/site-packages/vobject/behavior.py", line 160, in serialize
transformed = obj.transformFromNative()
File "/usr/lib/python2.7/site-packages/vobject/base.py", line 226, in transformFromNative
raise NativeError(msg, lineNumber)
vobject.base.NativeError: "In transformFromNative, unhandled exception on line None <type 'exceptions.TypeError'>: 'datetime.datetime' object has no attribute '__getitem__'"
Any ideas?
I have an excel file with a ton of rows, and a column containing HZ, NZ or SZ now is this erroring out, and I have no idea on how to fix it.
region = ws.cell(row=srow, column=3)
if region.value == "HZ":
nregion = "Houston"
elif region.value == "NZ":
nregion = "North"
elif region.value == "SZ":
nregion = "South"
It errors with the following messages:
Traceback (most recent call last):
File "PowerExp.py", line 18, in <module>
wb=load_workbook(filename = wbfile)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/reader/excel.py", line 136, in load_workbook
_load_workbook(wb, archive, filename, use_iterators, keep_vba)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/reader/excel.py", line 171, in _load_workbook
style_table = read_style_table(archive.read(ARC_STYLE))
File "/usr/local/lib/python2.7/dist-packages/openpyxl/reader/style.py", line 42, in read_style_table
font_list = parse_fonts(root, xmlns, color_index)
File "/usr/local/lib/python2.7/dist-packages/openpyxl/reader/style.py", line 160, in parse_fonts
font.size = font_node.find(QName(xmlns, 'sz').text).get('val')
AttributeError: 'NoneType' object has no attribute 'get'
I can't change the value and save the workbook as it is automatically retrieved from the email server and loaded into MySQL.