Develop Reference

Post-training quantization with tflite cause runtime error

I am trying to quantize my model (specifically pretrained faster_rcnn_inception_v2 on coco, that was downloaded from the model zoo), in hopes to speedup inference time.
I use the following code from here:
import tensorflow as tf
converter = tf.lite.TocoConverter.from_saved_model(saved_model_dir)
converter.post_training_quantize = True
tflite_quantized_model = converter.convert()
open("quantized_model.tflite", "wb").write(tflite_quantized_model)
Models directory didnt have saved_model.pb file. So i renamed frozen_inference_graph.pb to saved_model.pb.
Running the code above produce the following runtime error:
Traceback (most recent call last):
File "/home/juggernaut/pycharm-community-2018.2.4/helpers/pydev/pydevd.py", line 1664, in <module>
main()
File "/home/juggernaut/pycharm-community-2018.2.4/helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/juggernaut/pycharm-community-2018.2.4/helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/hdd/motorola/motorola_heads/tensorflow_face_detection/quantize.py", line 5, in <module>
converter = tf.lite.TocoConverter.from_saved_model(saved_model_dir)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 318, in new_func
return func(*args, **kwargs)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/lite/python/lite.py", line 587, in from_saved_model
tag_set, signature_key)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/lite/python/lite.py", line 376, in from_saved_model
output_arrays, tag_set, signature_key)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/lite/python/convert_saved_model.py", line 254, in freeze_saved_model
meta_graph = get_meta_graph_def(saved_model_dir, tag_set)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/lite/python/convert_saved_model.py", line 61, in get_meta_graph_def
return loader.load(sess, tag_set, saved_model_dir)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 318, in new_func
return func(*args, **kwargs)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/python/saved_model/loader_impl.py", line 269, in load
return loader.load(sess, tags, import_scope, **saver_kwargs)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/python/saved_model/loader_impl.py", line 420, in load
**saver_kwargs)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/python/saved_model/loader_impl.py", line 347, in load_graph
meta_graph_def = self.get_meta_graph_def_from_tags(tags)
File "/hdd/motorola/venv_py27_tf1.10/local/lib/python2.7/site-packages/tensorflow/python/saved_model/loader_impl.py", line 323, in get_meta_graph_def_from_tags
" could not be found in SavedModel. To inspect available tag-sets in"
RuntimeError: MetaGraphDef associated with tags set(['serve']) could not be found in SavedModel. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli`
What does it mean and what should i do?

Please refer to this issue. They seem to have the same issue as you.
This may be fixed in a more recent version of Tensorflow (perhaps the tag has switched from 'serve' to 'serving' in the meantime).

You should use tf.saved_model.simple_save to save the pb model.

Python layer can't read hdf5 file in caffe framework

I write python layer for caffe that can read hdf5 with some manipulation needs for me. But I have some issue when open and read hdf5 file in python setup method. Here it is.
When I used tables with the
code in setup:
def setup(self, bottom, top):
h5file = tables.open_file("/home/titan/models/hdf5/train_small.h5", driver="H5FD_CORE")
I have this error when run net:
Traceback (most recent call last):
File "/home/titan/scripts/python_layers/pydata_hdf5.py", line 37, in setup
h5file = tables.open_file("/home/titan/models/hdf5/train_small.h5", driver="H5FD_CORE")
File "/home/titan/anaconda/lib/python2.7/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, **kwargs)
File "/home/titan/anaconda/lib/python2.7/site-packages/tables/file.py", line 784, in __init__
self._g_new(filename, mode, **params)
File "tables/hdf5extension.pyx", line 465, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:4872)
tables.exceptions.HDF5ExtError: HDF5 error back trace
File "../../../src/H5FDcore.c", line 273, in H5Pset_fapl_core
not a file access property list
File "../../../src/H5Pint.c", line 3371, in H5P_object_verify
property list is not a member of the class
File "../../../src/H5Pint.c", line 3321, in H5P_isa_class
not a property list
End of HDF5 error back trace
When I used h5py
def setup(self, bottom, top):
self.data = h5py.File('/home/titan/models/hdf5_nose_mouth/train_small.h5', 'r')
I have the same error:
Traceback (most recent call last):
File "/home/titan/scripts/python_layers/pydata_hdf5.py", line 11, in <module>
import h5py
File "/home/titan/anaconda/lib/python2.7/site-packages/h5py/__init__.py", line 31, in <module>
from .highlevel import *
File "/home/titan/anaconda/lib/python2.7/site-packages/h5py/highlevel.py", line 13, in <module>
from ._hl.base import is_hdf5, HLObject
File "/home/titan/anaconda/lib/python2.7/site-packages/h5py/_hl/base.py", line 78, in <module>
dlapl = default_lapl()
File "/home/titan/anaconda/lib/python2.7/site-packages/h5py/_hl/base.py", line 65, in default_lapl
lapl = h5p.create(h5p.LINK_ACCESS)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (-------src-dir-------/h5py/_objects.c:2458)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (-------src-dir-------/h5py/_objects.c:2415)
File "h5py/h5p.pyx", line 130, in h5py.h5p.create (-------src-dir-------/h5py/h5p.c:2491)
ValueError: Not a property list class (Not a property list class)
When I used deepdish:
def setup(self, bottom, top):
self.data = dd.io.load('/home/titan/models/hdf5/train_smallest.h5')
I have error too:
Traceback (most recent call last):
File "/home/titan/scripts/python_layers/pydata_hdf5.py", line 36, in setup
self.data = dd.io.load('/home/titan/models/hdf5/train_smallest.h5')
File "/home/titan/anaconda/lib/python2.7/site-packages/deepdish/io/hdf5io.py", line 476, in load
with tables.open_file(path, mode='r') as h5file:
File "/home/titan/anaconda/lib/python2.7/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, **kwargs)
File "/home/titan/anaconda/lib/python2.7/site-packages/tables/file.py", line 784, in __init__
self._g_new(filename, mode, **params)
File "tables/hdf5extension.pyx", line 488, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:5081)
tables.exceptions.HDF5ExtError: HDF5 error back trace
File "../../../src/H5F.c", line 1582, in H5Fopen
not file access property list
File "../../../src/H5Pint.c", line 3321, in H5P_isa_class
not a property list
End of HDF5 error back trace
But when I read simply *.txt file, all is ok. Also I can read this file from console and use it in hdf5 layer in caffe. Please help me, how I can read hdf5 file from python layer?

Try to install another version of h5py. I just solved it with:
pip install h5py==prev_version
I guess it has something to do with the linking but it would be interesting to know which is the exact cause for this problem.

KeyError: 'browser' with Splinter and Behaving automated testing

I followed the instructions here: http://shon.github.io/2014/06/19/ui_testing_and_bdd.html about setting up Splinter with Behaving to run automated tests. I'm able to run a test successfully, but at the end of the test, it throws an error saying:
KeyError: 'browser'
and it won't continue testing any additional feature files. I'm pretty new to python and need some help in troubleshooting this.
Exception KeyError: 'browser'
Traceback (most recent call last):
File "/usr/local/bin/behave", line 11, in <module> sys.exit(main())
File "/Library/Python/2.7/site-packages/behave/__main__.py", line 109, in main
failed = runner.run()
File "/Library/Python/2.7/site-packages/behave/runner.py", line 672, in run
return self.run_with_paths()
File "/Library/Python/2.7/site-packages/behave/runner.py", line 693, in run_with_paths
return self.run_model()
File "/Library/Python/2.7/site-packages/behave/runner.py", line 483, in run_model
failed = feature.run(self)
File "/Library/Python/2.7/site-packages/behave/model.py", line 523, in run
failed = scenario.run(runner)
File "/Library/Python/2.7/site-packages/behave/model.py", line 867, in run
runner.run_hook('before_scenario', runner.context, self)
File "/Library/Python/2.7/site-packages/behave/runner.py", line 405, in run_hook
self.hooks[name](context, *args)
File "features/environment.py", line 48, in before_scenario
context.browser = default_browser
File "/Library/Python/2.7/site-packages/behave/runner.py", line 223, in __setattr__
record = self._record[attr]
KeyError: 'browser'

I found the issue. It is related to the Feature file structure. The Feature file was missing:
Background:
Given a browser
This also required changes to the environment.py file based on the info here: https://github.com/ggozad/behaving

Searching for a string with Web.py

I'm trying to build a python function with web.py and SQLite that will allow users to search for a given string within a description field and will return all matching results.
Right now I've gotten to the below function, which works but only if the input is an exact match.
def getItem(params, max_display):
query_string = 'SELECT * FROM items WHERE 1=1'
description = params['description']
if params['description']:
query_string = query_string + ' AND description LIKE $description'
result = query(query_string, {
'description': params['description']
I've tried to implement this feature with LIKE "%$description%"' , however I keep getting the below web.py error.
Traceback (most recent call last):
File "lib/web/wsgiserver/__init__.py", line 1245, in communicate
req.respond()
File "lib/web/wsgiserver/__init__.py", line 775, in respond
self.server.gateway(self).respond()
File "lib/web/wsgiserver/__init__.py", line 2018, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "lib/web/httpserver.py", line 306, in __call__
return self.app(environ, xstart_response)
File "lib/web/httpserver.py", line 274, in __call__
return self.app(environ, start_response)
File "lib/web/application.py", line 279, in wsgi
result = self.handle_with_processors()
File "lib/web/application.py", line 249, in handle_with_processors
return process(self.processors)
File "lib/web/application.py", line 246, in process
raise self.internalerror()
File "lib/web/application.py", line 478, in internalerror
return debugerror.debugerror()
File "lib/web/debugerror.py", line 305, in debugerror
return web._InternalError(djangoerror())
File "lib/web/debugerror.py", line 290, in djangoerror
djangoerror_r = Template(djangoerror_t, filename=__file__, filter=websafe)
File "lib/web/template.py", line 846, in __init__
code = self.compile_template(text, filename)
File "lib/web/template.py", line 926, in compile_template
ast = compiler.parse(code)
File "/Users/sokeefe/homebrew/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/transformer.py", line 51, in parse
return Transformer().parsesuite(buf)
File "/Users/sokeefe/homebrew/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/transformer.py", line 128, in parsesuite
return self.transform(parser.suite(text))
AttributeError: 'module' object has no attribute 'suite'
Any thoughts on what might be going wrong with this function?
Thanks in advance!

What do you think is going on with parser.py?
Here is the relevant portion of the error message:
File
"/Users/sokeefe/homebrew/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/compiler/transformer.py",
line 128, in parsesuite
return self.transform(parser.suite(text)) AttributeError: 'module' object has no attribute 'suite'
So, somewhere there is a file called parser.py, which defines a function called suite(), which is used by some library code that executes when your program executes. But because you named one of your files parser.py, when the library code executes, python searches for a file named parser.py, and python found your file first, and there was no function named suite() in your file.

Open a new instance of AutoCAD via Python

Does anyone have any experience using Python with AutoCAD? I'm just trying, as a test to see if I can open a new instance of AutoCAD via Python and though that PyAutocad worked well (feel free to offer other suggestions, if you have any).
Anyway based on the doc (https://pypi.python.org/pypi/pyautocad/#downloads) - it says these lines of code should do it, but nothing's happened as of yet.
from pyautocad import Autocad, APoint
acad = Autocad()
acad.prompt("Hello, Autocad from Python\n")
Just these lines of code should generate info on the commandline but instead results in about 50 lines worth of traceback (which I can post as well if anyone's interested) - any ideas?
Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
acad.prompt("Hello, Autocad")
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 153, in prompt
self.doc.Utility.Prompt(u"%s\n" % text)
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 65, in doc
return self.app.ActiveDocument
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 54, in app
self._app = comtypes.client.GetActiveObject('AutoCAD.Application')
File "C:\Python27\lib\site-packages\comtypes\client\__init__.py", line 180, in GetActiveObject
obj = comtypes.GetActiveObject(clsid, interface=interface)
File "C:\Python27\lib\site-packages\comtypes\__init__.py", line 1165, in GetActiveObject
oledll.oleaut32.GetActiveObject(byref(clsid), None, byref(p))
File "_ctypes/callproc.c", line 941, in GetResult
WindowsError: [Error -2147221021] Operation unavailable
After adding the VBA Module, the traceback now reads:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
acad.prompt("Hello")
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 153, in prompt
self.doc.Utility.Prompt(u"%s\n" % text)
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 65, in doc
return self.app.ActiveDocument
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 54, in app
self._app = comtypes.client.GetActiveObject('AutoCAD.Application')
File "C:\Python27\lib\site-packages\comtypes\client\__init__.py", line 183, in GetActiveObject
return _manage(obj, clsid, interface=interface)
File "C:\Python27\lib\site-packages\comtypes\client\__init__.py", line 188, in _manage
obj = GetBestInterface(obj)
File "C:\Python27\lib\site-packages\comtypes\client\__init__.py", line 112, in GetBestInterface
interface = getattr(mod, itf_name)
AttributeError: 'module' object has no attribute 'IAcadApplication'
Now after deleting comtypes' "gen" folder per #reclosedev, Autocad now opens but the string of text still will not appear on the command line - the traceback reads:
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
acad.prompt("Hello")
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 153, in prompt
self.doc.Utility.Prompt(u"%s\n" % text)
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 65, in doc
return self.app.ActiveDocument
File "C:\Python27\lib\site-packages\pyautocad\api.py", line 59, in app
self._app.Visible = self._visible
File "C:\Python27\lib\site-packages\comtypes\client\dynamic.py", line 116, in __setattr__
dispid = self._comobj.GetIDsOfNames(name)[0]
File "C:\Python27\lib\site-packages\comtypes\automation.py", line 643, in GetIDsOfNames
self.__com_GetIDsOfNames(riid_null, arr, len(names), lcid, ids)
COMError: (-2147418111, 'Call was rejected by callee.', (None, None, None, 0, None))

Have you considered IronPython?
http://through-the-interface.typepad.com/through_the_interface/2009/03/using-ironpython-with-autocad.html
Here's a quick example of how it could be leveraged:
import clr
import System
from System import Type, Activator
acApp = Activator.CreateInstance(Type.GetTypeFromProgID("AutoCAD.Application"))
acApp.Visible = 1
acApp.ActiveDocument.SendCommand("(Princ \"Hello World from Python!\")(Princ)\n")
Obviously the ProgId will correspond to whichever version of AutoCAD was opened last.