Develop Reference

Huggingface DETR model fails with index error when trying to train on coco2017 dataset

I am trying to build DETR model on subset of COCO dataset using huggingface. This is the tutorial that I am following https://github.com/NielsRogge/Transformers-Tutorials/blob/master/DETR/Fine_tuning_DetrForObjectDetection_on_custom_dataset_(balloon).ipynb. The tutorial loads balloon dataset, I have used COCO instead.
The code works fine up to the training part, it throws the following index error when fit is called:
IndexError: index 64 is out of bounds for dimension 0 with size 3
Stacktrace:
Cell In[26], line 4
1 from pytorch_lightning import Trainer
3 trainer = Trainer(max_steps=1, gradient_clip_val=0.1)
----> 4 trainer.fit(model)
File c:\Code\detr\venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:603, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
601 raise TypeError(f"`Trainer.fit()` requires a `LightningModule`, got: {model.__class__.__qualname__}")
602 self.strategy._lightning_module = model
--> 603 call._call_and_handle_interrupt(
604 self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
605 )
File c:\Code\detr\venv\lib\site-packages\pytorch_lightning\trainer\call.py:38, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
36 return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
37 else:
---> 38 return trainer_fn(*args, **kwargs)
40 except _TunerExitException:
41 trainer._call_teardown_hook()
File c:\Code\detr\venv\lib\site-packages\pytorch_lightning\trainer\trainer.py:645, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
638 ckpt_path = ckpt_path or self.resume_from_checkpoint
639 self._ckpt_path = self._checkpoint_connector._set_ckpt_path(
640 self.state.fn,
...
-> 2205 class_cost = -out_prob[:, target_ids]
2207 # Compute the L1 cost between boxes
2208 bbox_cost = torch.cdist(out_bbox, target_bbox, p=1)
There are 2 differences from the tutorial code:
instead of balloon, coco 2017 dataset is used
my machine does not have nvidia gpu, the gpu argument is not supplied to
trainer = Trainer(gpus=1, max_steps=300, gradient_clip_val=0.1)
I tried debugging, and found that the code fails at following line:
outputs = self.model(pixel_values=pixel_values, pixel_mask=pixel_mask, labels=labels) in common_step method of Detr class.
I tried with unmodified coco2017 dataset; it always fails with the same error.
What am I missing?

Can't find model 'en'. It looks like you're trying to load a model from a shortcut, which is obsolete as of spaCy v3.0

I was trying to run
text_field = Field(tokenize='spacy', lower=True, include_lengths=True, batch_first=True)
However, the error shows :
OSError Traceback (most recent call last)
<ipython-input-72-1ac550316aec> in <module>
2
3 label_field = Field(sequential=False, use_vocab=False, batch_first=True, dtype=torch.float)
----> 4 text_field = Field(tokenize='spacy', lower=True, include_lengths=True, batch_first=True)
5 fields = [('label', label_field), ('title', text_field), ('text', text_field), ('titletext', text_field)]
6
~/opt/anaconda3/lib/python3.8/site-packages/torchtext/data/field.py in __init__(self, sequential, use_vocab, init_token, eos_token, fix_length, dtype, preprocessing, postprocessing, lower, tokenize, tokenizer_language, include_lengths, batch_first, pad_token, unk_token, pad_first, truncate_first, stop_words, is_target)
161 # in case the tokenizer isn't picklable (e.g. spacy)
162 self.tokenizer_args = (tokenize, tokenizer_language)
--> 163 self.tokenize = get_tokenizer(tokenize, tokenizer_language)
164 self.include_lengths = include_lengths
165 self.batch_first = batch_first
~/opt/anaconda3/lib/python3.8/site-packages/torchtext/data/utils.py in get_tokenizer(tokenizer, language)
112 try:
113 import spacy
--> 114 spacy = spacy.load(language)
115 return partial(_spacy_tokenize, spacy=spacy)
116 except ImportError:
~/opt/anaconda3/lib/python3.8/site-packages/spacy/__init__.py in load(name, vocab, disable, exclude, config)
49 RETURNS (Language): The loaded nlp object.
50 """
---> 51 return util.load_model(
52 name, vocab=vocab, disable=disable, exclude=exclude, config=config
53 )
~/opt/anaconda3/lib/python3.8/site-packages/spacy/util.py in load_model(name, vocab, disable, exclude, config)
424 return load_model_from_path(name, **kwargs) # type: ignore[arg-type]
425 if name in OLD_MODEL_SHORTCUTS:
--> 426 raise IOError(Errors.E941.format(name=name, full=OLD_MODEL_SHORTCUTS[name])) # type: ignore[index]
427 raise IOError(Errors.E050.format(name=name))
428
OSError: [E941] Can't find model 'en'. It looks like you're trying to load a model from a shortcut, which is obsolete as of spaCy v3.0. To load the model, use its full name instead:
nlp = spacy.load("en_core_web_sm")
For more details on the available models, see the models directory: https://spacy.io/models. If you want to create a blank model, use spacy.blank: nlp = spacy.blank("en")
I even tried to do python -m spacy download en but still doesn't run on my jupyter notebook. Does anyone know how to fix this problem?

Tensorflow VGG16 Attribute Error: module tensorflow has no attribute get_default_graph? [duplicate]

I am doing some task related to image captioning and I have loaded the weights of inception model like this
model = InceptionV3(weights='imagenet')
But I am getting error like this:
AttributeError: module 'tensorflow' has no attribute 'get_default_graph'
What should I do? Please help.
Here is the full output of above code.
1 . --------------------------------------------------------------------------- AttributeError Traceback (most recent
call last) in ()
1 # Load the inception v3 model
----> 2 model = InceptionV3(include_top=True,weights='imagenet')
3 # InceptionV3(weights='imagenet')
~/anaconda3/lib/python3.6/site-packages/keras/applications/__init__.py
in wrapper(*args, **kwargs)
26 kwargs['models'] = models
27 kwargs['utils'] = utils
---> 28 return base_fun(*args, **kwargs)
29
30 return wrapper
~/anaconda3/lib/python3.6/site-packages/keras/applications/inception_v3.py
in InceptionV3(*args, **kwargs)
9 #keras_modules_injection
10 def InceptionV3(*args, **kwargs):
---> 11 return inception_v3.InceptionV3(*args, **kwargs)
12
13
~/anaconda3/lib/python3.6/site-packages/keras_applications/inception_v3.py
in InceptionV3(include_top, weights, input_tensor, input_shape,
pooling, classes, **kwargs)
155
156 if input_tensor is None:
--> 157 img_input = layers.Input(shape=input_shape)
158 else:
159 if not backend.is_keras_tensor(input_tensor):
~/anaconda3/lib/python3.6/site-packages/keras/engine/input_layer.py
in Input(shape, batch_shape, name, dtype, sparse, tensor)
176 name=name, dtype=dtype,
177 sparse=sparse,
--> 178 input_tensor=tensor)
179 # Return tensor including _keras_shape and _keras_history.
180 # Note that in this case train_output and test_output are the same pointer.
~/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py
in wrapper(*args, **kwargs)
89 warnings.warn('Update your `' + object_name + '` call to the ' +
90 'Keras 2 API: ' + signature, stacklevel=2)
---> 91 return func(*args, **kwargs)
92 wrapper._original_function = func
93 return wrapper
~/anaconda3/lib/python3.6/site-packages/keras/engine/input_layer.py
in __init__(self, input_shape, batch_size, batch_input_shape, dtype,
input_tensor, sparse, name)
37 if not name:
38 prefix = 'input'
---> 39 name = prefix + '_' + str(K.get_uid(prefix))
40 super(InputLayer, self).__init__(dtype=dtype, name=name)
41
~/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py
in get_uid(prefix)
72 """
73 global _GRAPH_UID_DICTS
---> 74 graph = tf.get_default_graph()
75 if graph not in _GRAPH_UID_DICTS:
76 _GRAPH_UID_DICTS[graph] = defaultdict(int)
AttributeError: module 'tensorflow' has no attribute
'get_default_graph'

Change
Import keras.<something>.<something>
to
Import tensorflow.keras.<something>.<something>
where "something" refers to the module you want to import. It worked for me.

Another cause due to which this is happening is that in tensorflow_backend.py
located in : lib/python3.6/site-packages/keras/backend/
uses tf.compat.v1.get_default_graph for obtaining graph
instead of tf.get_default_graph.
By replacing this in the directory this problem can be solved successfully.

Keras integrated into TensorFlow 2.0
The article below cleared this up for me. Key points are:
Keras is included in the TensorFlow 2.0 package
So no need to install the stand-alone Keras package in your environment
And now the fore-mentioned solutions of using "from tensorflow.keras…" make sense.
After recognizing that and making the change, my code samples work with some minor changes here and there.
https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/

I fixed this problem by replacing tensorflow.keras.* to tensorflow.python.keras.*
Working example:
from tensorflow.python.keras.models import Sequential

Two Steps solved this issue for me in Google Colabs
First Step
Change
from keras.something import something
To
from tensorflow.keras.something import something
Second Step
Use
tf.compat.v1.get_default_graph
instead of tf.get_default_graph
Reason
https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/

In my case, replacing
from keras.models import models
with:
from tensorflow.keras.models import models
in my script, fixed this problem.

Can't convert Core ML model to Onnx (then to Tensorflow Lite)

I'm trying to convert a trained Core ML model to TensorFlow Lite. I find I need convert it to Onnx first.
The problems is that I get errors. I've tried with different versions of python, onnxmltools, winmltools and it doesn't seems to work. I also tried docker image of onnx ecosystem with same result. Can any one help me with it? Thanks in advance.
Script I used:
import coremltools
import onnxmltools
input_coreml_model = '../model.mlmodel'
output_onnx_model = '../model.onnx'
coreml_model = coremltools.utils.load_spec(input_coreml_model)
onnx_model = onnxmltools.convert_coreml(coreml_model)
onnxmltools.utils.save_model(onnx_model, output_onnx_model)
IndexError Traceback (most recent call last)
<ipython-input-11-94a6dc527869> in <module>
3
4 # Convert the CoreML model into ONNX
----> 5 onnx_model = onnxmltools.convert_coreml(coreml_model)
6
7 # Save as protobuf
/usr/local/lib/python3.6/dist-packages/onnxmltools/convert/main.py in convert_coreml(model, name, initial_types, doc_string, target_opset, targeted_onnx, custom_conversion_functions, custom_shape_calculators)
16 from .coreml.convert import convert
17 return convert(model, name, initial_types, doc_string, target_opset, targeted_onnx,
---> 18 custom_conversion_functions, custom_shape_calculators)
19
20
/usr/local/lib/python3.6/dist-packages/onnxmltools/convert/coreml/convert.py in convert(model, name, initial_types, doc_string, target_opset, targeted_onnx, custom_conversion_functions, custom_shape_calculators)
58 target_opset = target_opset if target_opset else get_opset_number_from_onnx()
59 # Parse CoreML model as our internal data structure (i.e., Topology)
---> 60 topology = parse_coreml(spec, initial_types, target_opset, custom_conversion_functions, custom_shape_calculators)
61
62 # Parse CoreML description, author, and license. Those information will be attached to the final ONNX model.
/usr/local/lib/python3.6/dist-packages/onnxmltools/convert/coreml/_parse.py in parse_coreml(model, initial_types, target_opset, custom_conversion_functions, custom_shape_calculators)
465 # Instead of using CoremlModelContainer, we directly pass the model in because _parse_model is CoreML-specific.
466 _parse_model(topology, scope, model)
--> 467 topology.compile()
468
469 for variable in topology.find_root_and_sink_variables():
/usr/local/lib/python3.6/dist-packages/onnxconverter_common/topology.py in compile(self)
630 self._resolve_duplicates()
631 self._fix_shapes()
--> 632 self._infer_all_types()
633 self._check_structure()
634
/usr/local/lib/python3.6/dist-packages/onnxconverter_common/topology.py in _infer_all_types(self)
506 pass # in Keras converter, the shape calculator can be optional.
507 else:
--> 508 operator.infer_types()
509
510 def _resolve_duplicates(self):
/usr/local/lib/python3.6/dist-packages/onnxconverter_common/topology.py in infer_types(self)
108 def infer_types(self):
109 # Invoke a core inference function
--> 110 registration.get_shape_calculator(self.type)(self)
111
112
/usr/local/lib/python3.6/dist-packages/onnxmltools/convert/coreml/shape_calculators/neural_network/Concat.py in calculate_concat_output_shapes(operator)
22 if variable.type.shape[0] != 'None' and variable.type.shape[0] != output_shape[0]:
23 raise RuntimeError('Only dimensions along C-axis can be different')
---> 24 if variable.type.shape[2] != 'None' and variable.type.shape[2] != output_shape[2]:
25 raise RuntimeError('Only dimensions along C-axis can be different')
26 if variable.type.shape[3] != 'None' and variable.type.shape[3] != output_shape[3]:
IndexError: list index out of range

AttributeError: module 'tensorflow' has no attribute 'io'

My problem is when i try to run this code
if log_to_tensorboard: from torch.utils.tensorboard import SummaryWriter
if log_to_tensorboard: writer = SummaryWriter()
I get this error:
(import SummaryWriter works without any problems, but then I try run "writer = SummaryWriter()" and it doesnt work)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-23-d77d9d09e62b> in <module>
----> 1 writer = SummaryWriter()
/anaconda3/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py in __init__(self, log_dir, comment, purge_step, max_queue, flush_secs, filename_suffix)
223 # and recreated later as needed.
224 self.file_writer = self.all_writers = None
--> 225 self._get_file_writer()
226
227 # Create default bins for histograms, see generate_testdata.py in tensorflow/tensorboard
/anaconda3/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py in _get_file_writer(self)
254 if self.all_writers is None or self.file_writer is None:
255 self.file_writer = FileWriter(self.log_dir, self.max_queue,
--> 256 self.flush_secs, self.filename_suffix)
257 self.all_writers = {self.file_writer.get_logdir(): self.file_writer}
258 if self.purge_step is not None:
/anaconda3/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py in __init__(self, log_dir, max_queue, flush_secs, filename_suffix)
64 log_dir = str(log_dir)
65 self.event_writer = EventFileWriter(
---> 66 log_dir, max_queue, flush_secs, filename_suffix)
67
68 def get_logdir(self):
/anaconda3/lib/python3.6/site-packages/tensorboard/summary/writer/event_file_writer.py in __init__(self, logdir, max_queue_size, flush_secs, filename_suffix)
71 """
72 self._logdir = logdir
---> 73 if not tf.io.gfile.exists(logdir):
74 tf.io.gfile.makedirs(logdir)
75 self._file_name = os.path.join(logdir, "events.out.tfevents.%010d.%s.%s.%s" %
/anaconda3/lib/python3.6/site-packages/tensorboard/lazy.py in __getattr__(self, attr_name)
63 class LazyModule(types.ModuleType):
64 def __getattr__(self, attr_name):
---> 65 return getattr(load_once(self), attr_name)
66
67 def __dir__(self):
AttributeError: module 'tensorflow' has no attribute 'io'
How to fix it?
I uninstall and install tensorflow, upgraded tensorboard and torch - that didnt help me

I met the same problem. And my solution is as follow:
Check to see if tensorflow-tensorboard is exists.
pip uninstall tensorflow-tensorboard
install tensorboard==1.14
because an error occurred: TensorBoard logging requires TensorBoard with Python summary writer installed. This should be available in 1.14 or above.
ref: https://github.com/pytorch/pytorch/issues/20140
pip install tensorboard==1.14.0

I was able to solve this problem by using the solution recommended here which suggests to:
pip install tensorflow-io

You can upgrade the version of tensorboard