How to upload files to Colab from Github without using Google Drive? - python

I have started to use Colab. Now I want to from the notebook automatically upload a couple of smaller files from my Github repository.
My idea is that I should try to upload these files directly to the workspace of the Colab virtual machine and use of Google Drive not necessary. This strategy should also facilitate sharing the notebook with others.
My Colab notebook code is as follows:
%%bash
git clone https://github.com/my_repository/folder1
%load folder1/file1.py
run -i file1.py
%load folder1/file2.zip
The first two command works fine but the two last gives error messages.
The error message when I try run file1.py is:
ERROR: root:File 'file1.py' not found.
And the error message when I try to load file2.zip
File "<string>", line unknown
SyntaxError: invalid or missing encoding declaration for 'folder1/file2.zip'
(The file2.zip contains both some text file and an executable file for linux environment)
How to solve this?
Note1. If I check the directory after the second command with !ls I see I have folder1
and when I do !ls folder1 then I see the content of that folder1. So looks ok so far.
Note2. If I mount my Google Drive and upload the folder here then I can get it all to work. But I want to avoid using Google Drive since that complicates sharing of the notebook, in my eyes.
Note3. What I can see the zip-file contains a binary that is described as ELF 64-bit LSB shared object, x86-64, version 1 (SYSV)

I found a solution I think and the code should be:
%%bash
git clone https://github.com/my_repository/folder1
%cd folder1
run -i file1.py

Related

Importing a package in Google Colab

First off, apologies if my question is worded naively, as I am new to Python and Colab development. I have mounted my Drive onto my Colab notebook and inserted a path to a directory labeled "backend". It is structured as follows:
backend->
solutions->
__init__.py
**(other files)
**(other files)
I am trying to import my solutions directory as a package into my main.py script, which I have as a code block at the end of my notebook. I am attempting to "import solutions" into main.py, but it is telling me that "import solutions cannot be resolved" even though it recognizes its path. Does anyone have any ideas of what is happening here? The code and import works as expected on my local machine. I even attempted to make solutions into a package by including a README and a simple setup.py file, and while I was able to install it, it gave me a SystemExit error which I suspect originated from my setup.py.
---EDIT---
new setup that copies the "solutions" directory to content. Still getting same error
What have you tried yet?
Have you tried the following?
# Mount your google drive in google colab
from google.colab import drive
drive.mount('/content/gdrive')
# Insert the directory
import sys
sys.path.insert(0,'/content/gdrive/My Drive/Colab Notebooks')
From here, be sure to have given access to your "My Drive" (security pop-up)
Then check:
!ls gdrive/MyDrive
On the left panel you should see something like this now:
"test-colab" is a folder I've created just to show you
Then you copy/import the folder you want into your content:
Adapt the code below with your own paths
!cp -av '/content/gdrive/MyDrive/test-colab' '/content/'
Check:
Check working directory before trying to import a .yp:

Google Colab - FileNotFoundError

i am trying to train YOLOv3 with custom dataset on Google Colab. I uploaded my folders, weights etc. When I run my train.py, I get path error. I run the code like this:
!python3 "drive/TrainYourOwnYolo/2_Training/Train_YOLO.py"
The error says,
content/TrainYourOwnYolo/2_Training/dataset/img20.jpg is not found.
As I understand on Colab, my all folders are under drive folder. I don't understand why yolo is trying to find my dataset under content folder. Do you have any idea?
As it seems, you have uploaded your data to /drive/TrainYourOwnYolo/, and not to /content/TrainYourOwnYolo/, where your script is looking.
The /content folder is normally used by Colab when saving, in case you don't use Google Drive. But you have mounted your Google Drive under /drive, so your notebook unsurprisingly fails to find the files.
You should change the file paths in your Train_YOLO.py" script to replace references to /content with /drive.
If this is not possible, you can find the /content folder on the file catalogue on the left of your Colab notebook:
and by right-clicling on it, you'll see an option for uploading files there.

How to use Google Colab to run .py files from Github repositories?

I would like to use Google Colab to accelerate the process of calculation of my deep learning model. However, I cannot run the model directly from the .ipynb file in the Google Colaboratory, since it has several .py functions written separately and then the main program will call them together in another .py file.
One of the proposed solutions consisted of following three steps:
Commit the code on Github
Clone on collab
run this command: !python model_Trainer.py on Colab
I have done steps 1 and 2 successfully, however, I still cannot run the third step. And I am getting the following error: python3: can't open file 'model_trainer.py': [Errno 2] No such file or directory.
Does anyone have a solution to this problem?
I look forward to hearing from you.
When you clone a GitHub repo in Colab, it will create a directory for that repo.
To run your python script, you must use %cd before using !python.
Example:
I have the same problem.
you can use !pwd to see the current dictionary and cd to change the dictionary which your python files locate.
If you wanna run the .py file, you can just use !python example.py to run.

Can I run python scripts in colab?

I am trying to get some open source software working. It uses things that I don't have on my system (pytorch for example) and so I thought that I could try to run it on Google Colab.
When I tried to do it though, there are some python scripts that I have to run after cloning a directory from a github repository. I guess I can't run another python script from inside a Jupyter Notebook, and so I suppose that I'm trying to do something with Colab that it isn't designed to do?
Is there something available that is more like a terminal, but using the software, GPUs etc. that are available on Colab?
You can run any shell command from jupyter-like environment (which includes colab) using ! in code cell, for example
!ls
Would list all files in colab's cwd.
To run python script you could do:
!python script.py
It works just like terminal (it might be python3, not sure how it's setup un colab)
You can call your script too.
!python script.py
But you need to put the script there, probably by git clone or direct uploading.
As Wayne mentions in the comment korakot's answer, you can use the magic command
%run 'script.py'
This also allows you to do e.g. run in the notebook's namespace by using the -i parameter
%run -i 'script.py'

Google CoLab - How to run a jupyter notebook file that is in the 'Files' tab (i.e. /content/) of my CoLab environment

In Google CoLab on the left is a pane that can be opened that shows Table of Contents, Code snippets, and Files.
In the Files pane there is an upload button, I can upload a notebook file to this Files area. But once the notebook file is uploaded, there is no option to run it as a notebook. The menu option File->OpenNotebook doesn't show the CoLab /content/ files as an option to start a notebook.
Is there a way to do this? Or can it be added in future releases?
The reason for this request is I'd like to git-clone a repo with multiple notebook files into the /content (or Files) area of CoLab. And then be able to easily switch between the notebooks, much like the native Jupyter notebook interface that shows a directory with potentially multiple notebooks that can be started.
I've tried right-clicking on the notebook file in Files but there is no option to start the notebook. I've tried using File->Open_notebook... the Files files aren't shown as an option in any of the tabs.
The desired results is that I can start .ipynb files (i.e. Jupyter notebooks) directly from the 'Files' or /content/ section of Google CoLab.
You can run other notebooks in your current notebook like this:
# if the file was on the google drive
%run /content/gdrive/My\ Drive/Colab\ Notebooks/DenseVideoArchitecture.ipynb
# simply replace the path in your case
%run /content/DenseVideoArchitecture.ipynb
But what you are asking is to switch between different notebooks in the same environment which might not be possible in collab.
I couldn't understand what you actually need, but I hope below code help you:
from google.colab import drive
drive.mount('/content/gdrive')
!cd content/gdrive/My Drive/Colab Notebooks
You should mount your google drive and now you have access to drive as a local drive. In this code, at first two lines, I mount gdrive and then I redirect to some place in google drive for example "Colab Notebooks" and you can run everything you want.

Categories

Resources