So I have been trying to make this file compatible to Google Colab but I'm not able to find any way to do it.
[]
EfficientDet-DeepSORT-Tracker is the main folder of this entire package
This picture is from one of the files placed alongside backbone.py
How to fix the fact that the file isn't able to detect backbone.py?
EDIT for more context: I shared the errors I found when trying to run waymo_open_dataset.py which isn't able to detect the other .py files alongside it.
According to this past question you could import 'filename' of the filename.py. So in the main.py file you are trying to run in colab, then import the required files in the main.py file.
Related
First off, apologies if my question is worded naively, as I am new to Python and Colab development. I have mounted my Drive onto my Colab notebook and inserted a path to a directory labeled "backend". It is structured as follows:
backend->
solutions->
__init__.py
**(other files)
**(other files)
I am trying to import my solutions directory as a package into my main.py script, which I have as a code block at the end of my notebook. I am attempting to "import solutions" into main.py, but it is telling me that "import solutions cannot be resolved" even though it recognizes its path. Does anyone have any ideas of what is happening here? The code and import works as expected on my local machine. I even attempted to make solutions into a package by including a README and a simple setup.py file, and while I was able to install it, it gave me a SystemExit error which I suspect originated from my setup.py.
---EDIT---
new setup that copies the "solutions" directory to content. Still getting same error
What have you tried yet?
Have you tried the following?
# Mount your google drive in google colab
from google.colab import drive
drive.mount('/content/gdrive')
# Insert the directory
import sys
sys.path.insert(0,'/content/gdrive/My Drive/Colab Notebooks')
From here, be sure to have given access to your "My Drive" (security pop-up)
Then check:
!ls gdrive/MyDrive
On the left panel you should see something like this now:
"test-colab" is a folder I've created just to show you
Then you copy/import the folder you want into your content:
Adapt the code below with your own paths
!cp -av '/content/gdrive/MyDrive/test-colab' '/content/'
Check:
Check working directory before trying to import a .yp:
I have cloned a GitHub repository and it has successfully imported all of the .py files I need on the left. I have set up a for loop that takes all of the .py files and saves them into an array. Now I want to loop through and run them all so that I can use their classes and functions without actually having to repaste their code into colab.
Example of list of .py files
I thought the below method would work:
Error message
But as shown, it says there is no such file or directory. Anybody have any ideas?
I got a question/problem related to how to use init.py file for a project im working on. Im trying to run train_pipeline.py, which is located at
packages/cnn_modality_clf/cnn_modality_clf/train_pipeline.py
This file uses scripts that are located in the same folder, and also files that are within folders.
My imports (within train_pipeline.py
from cnn_modality_clf.pipeline import pipe
from cnn_modality_clf.config import config
from cnn_modality_clf.preprocessing import data_management as dm
from cnn_modality_clf.preprocessing import preprocessors as pp
However, when I run this train_pipeline file, I get a ModuleNotFoundError: No module named cnn_modality_clf and the error is on from cnn_modality_clf.pipeline import pipe
I have worked on previous projects (not that many) where using the init.py file will solved this issue. However, I have added this file into every folder. But that doesnt seem to work out.
My project structure looks like this:
I want to be able to use the python files that are within the same path, or that are within a folder (cnn.modality_clf.pipeline import pipe)
*Im using anaconda environment, and im using python 3.7.0
Any suggestions on how to overcome this problem?
Thanks in advance!
I'm trying to run a custom project that uses large parts of the SiamMask project. When the code is ran, one of the many imports is this one:
from . import region
located in an __init__.py file. It is trying to import a .so file called region.cpython-36m-x86_64-linux-gnu.so which is located in the same directory as the __init__.py file.
However, when I run the code, I get the following error:
ImportError: /scratch/[hidden]/project/libs/siamMask/utils/pyvotkit/region.cpython-36m-x86_64-linux-gnu.so: failed to map segment from shared object: Operation not permitted
I'm pretty confident that this error is caused due to the following fact. The scratch disk I'm working on, which is mounted to the GPU server, doesn't have direct 'execute' permissions for Python related things. (There's nothing I can change about this) This is also the reason why the Python virtual environment I'm working on is located on a different drive (where the system has direct execute permissions on everything that is Python related).
My question however is, how can I import this .so file in a different way that wouldn't interfere with the above fact? Could I transfer the file to the other drive and somehow import it from there via that __init__ file?
Thank you in advance!
I was able to fix this issue in the following way:
I moved the region.cpython-36m-x86_64-linux-gnu.so file to the other drive where Python can execute. I then used the following code in the __init__ file:
import sys
sys.path.append('/path/to/.sofile')
import region
In my case, I had a virtual environment on a separate disk, and the reason for denied access was user flag set for the disk with git repository in /etc/fstab.
After removing the flag and remounting the disk, the error was gone
I am trying to run my program on Google Colab; where my code make use of .py files written seprately.
In normal system I have all files inside one folder and it works using import xyz, but when I tried using same folder in Google drive it gives import error.
Now in googlecolab(Nov 18) you can upload your python files easily
Navigate to Files (Tab on your left panel)
Click on UPLOAD Upload your python folder or .py files
Use googlecolab book to access the file.
Please check my screenshot below!
If you have just 2-3 files, you can try the solution I gave in another question here.
Importing .py files in Google Colab
But if you have something like 5-10 files, I would suggest you put your library on github, then !git clone it to Google Colab. Another solution is to zip all you library files, then modify the first solution by unzipping with !unzip mylib.zip
If those library files are not in a folder structure, just a few files in the same folder. You can upload and save them then import them. Upload them with:
def upload_files():
from google.colab import files
uploaded = files.upload()
for k, v in uploaded.items():
open(k, 'wb').write(v)
return list(uploaded.keys())
For example you have a module like this
simple.py
def helloworld():
print("hello")
Click arrow on left panel => Choose File tab => Upload simple.py
In notebook code like this
import simple
simple.helloworld()
=> hello
Something I've used when I have multiple python scripts and want to automatically import through code is to set it up as a package and clone from the repo.
First set up the scripts in a repo with a setup.py and __init__.py files (obviously).
Then add this to the top of your notebook:
!rm -rf <repo-name> # in case you need to refresh after pushing changes
!git clone https://github.com/<user>/<repo-name>.git
Then install the package:
!pip install ./<repo-name>
Now conveniently import functions or whatever:
from <app-name>.<module> import <function>
I found this easiest way
from google.colab import drive
drive.mount('/content/drive')
%cd /content/drive/MyDrive/directory-location