I have a saved transformers model using BertModel.from_pretrained('test_model')
I have trained this model using google colab's GPUs
Then, I want to open it, with BertModel.from_pretrained('test_model/')
but I do not have a GPU in my local PC. I get this:
/home/seiji/.local/lib/python3.8/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
What shoud I do? I have no idea of how can I open it using a CPU. And is it possible?
The best thing you can do is save the CPU version of the model, i.e:
model.cpu().save_pretrained("model_directory")
All the pre-trained Huggingface models are saved as CPU models anyway and you always need to move them to GPU explicitly.
PyTorch allows loading GPU models on CPU (see https://discuss.pytorch.org/t/on-a-cpu-device-how-to-load-checkpoint-saved-on-gpu-device/349), but the arguments of torch.load you would need to set are not exposed via the API, so you would need write your own from_pretrained method.
Related
Description
I had a setup for training using the object detection API that worked really well, however I have had to upgrade from TF1.15 to TF2 and so instead of using model_main.py I am now using model_main_tf2.py and using mobilenet ssd 320x320 pipeline to transfer train a new model.
When training my model in TF1.15 it would display a whole heap of scalars as well as detection box image samples. It was fantastic.
In TF2 training I get no such data, just loss scalars and 3 input images!! and yet the event files are huge gigabytes! where as they were in hundreds of megs using TF1.15
The thing is there is nowhere to specify what data is presented. I have not changed anything other than which model_main py file I use to run the training. I added num_visualizations: to the pipeline config file but no visualizations of detection boxes appear.
Can someone please explain to me what is going on? I need to be able to see whats happening throughout training!
Thank You
I am training on PC in virtual environment before performing TRT optimization in Linux but I think that is irrelevant here really.
Environment
GPU Type: P220
Operating System + Version: Win10 Pro
Python Version (if applicable): 3.6
TensorFlow Version (if applicable): 2
Relevant Files
TF1.15 vs TF2 screenshots:
TF1 (model_main.py) Tensorboard Results
TF2 (model_main_tf2.py) Tensorboard Results
Steps To Reproduce
The repo I am working with GitHub Object Detection API
Model
Pipeline Config File
UPDATE: I have investigated further and discovered that the tensorboard settings are being set in Object Detection Trainer 1 for TF1.15 and Object Detection Trainer 2 for TF2
So if someone who knows more than I do about this could work out what the difference is and what I need to do to get same result in tensorboard with v2 as I do with the first one that would be amazing and save me enormous headache. It would seem that this, even though it is documented as being for TF2, is not actually following TF2 syntax but I could be wrong.
I am running Python program, but I do not have a GPU, what can I do to make Python use CPU instead of GPU?
$ python extract_feature.py --data mnist --net checkpoint_4.pth.tar --features pretrained
It gives me the following warning:
=> RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
The photo is the Structure of my Python project:
I got into a similar error. Then by trying the following workaround issue is solved. (If your model is .pth or .h5 format.)
MODEL_PATH = 'Somemodelname.pth'
model.load_state_dict(torch.load(MODEL_PATH,
map_location=torch.device('cpu')))
If you want certain GPU to be used in your machine. Then,
map_location = torch.device('cuda:device_id')
I am training Keras CNN models for two different applications on the Jupyter Notebook. Given that I want to utilize the full resources of my PC, can I use Keras-GPU in one notebook and another notebook using CPU.
I learned that Keras uses GPU by default - if available- and I can force Keras to use CPU as
in Can Keras with Tensorflow backend be forced to use CPU or GPU at will?. My question is that by running this line of code,
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
will the default settings change in all the running notebooks or in that particular notebook only?
by running this line of code,
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
the default settings change in that particular notebook only
You can use
os.environ['CUDA_VISIBLE_DEVICES'] = ''
to train on CPU
I have converted a tensorflow inference graph to tflite model file (*.tflite), according to instructions from https://www.tensorflow.org/lite/convert.
I tested the tflite model on my GPU server, which has 4 Nvidia TITAN GPUs. I used the tf.lite.Interpreter to load and run tflite model file.
It works as the former tensorflow graph, however, the problem is that the inference became too slow. When I checked out the reason, I found that the GPU utilization is simply 0% when tf.lite.Interpreter is running.
Is there any method that I can run tf.lite.Interpreter with GPU support?
https://github.com/tensorflow/tensorflow/issues/34536
CPU is kind of good enough for tflite, especially multicore.
nvidia GPU likely not updated for tflite, which is for mobile GPU platform.
Conspiracy: they (TF-NVIDIA) hand-shake to not let TFlite working on GPU ? oo easy to make one.
Steve
I trained my tf model in python:
with sv.managed_session(master='') as sess:
with tf.device("/gpu:1"):#my systerm has 4 nvidia cards
and use the command line to abstract the model:
freeze_graph.py --clear_devices False
and during test phase, I set the device as follow:
tensorflow::graph::SetDefaultDevice("/gpu:1", &tensorflow_graph);
but someting is wrong:
ould not create Tensorflow Graph:
Invalid argument: Cannot assign a device to node '.../RNN_backword/while/Enter':
Could not satisfy explicit device specification '/gpu:1'
because no devices matching that specification are registered in this process;
available devices: /job:localhost/replica:0/task:0/cpu:0
so,how can I use gpu i correctly??
anyone could help??
Is it possible you're using a version of TensorFlow without GPU support enabled? If you're building a binary you may need to add additional BUILD rules from //tensorflow that enable GPU support. Also ensure you enabled GPU support when running configure.
EDIT: Can you file a bug on TF's github issues with:
1) your BUILD rule
2) much more of your code so we can see how you're building your model and creating your session
3) how you ran configure
While this API is not yet marked "public"; we want to see if there's indeed a bug you are running into so we can fix it.