I am trying use Google Earth Engine's Python API on my 'Windows 10 Home' computer, for which Google recommend I set up a docker container (https://developers.google.com/earth-engine/python_install).
Following the instructions here (https://developers.google.com/earth-engine/python_install-datalab-local), I have downloaded Docker Toolbox and successfully run the docker run hello-world command.
However, when I try to run the following code:
set "GCP_PROJECT_ID=YOUR_PROJECT_ID"
set "CONTAINER_IMAGE_NAME=gcr.io/earthengine-project/datalab-ee:latest"
set "HOME=%HOMEDRIVE%%HOMEPATH%"
set "WORKSPACE=%HOME%\workspace\datalab-ee"
mkdir "%WORKSPACE%"
cd %WORKSPACE%
I get the following error on the 5th line: mkdir: cannot create directory '%WORKSPACE': Permission denied.
Does anyone know what's causing this? I have only ever use Anaconda Prompt and am not used to the syntax of this terminal.
Also, just to clarify, I entered the correct project ID into the terminal for line 1, but have not shared it here.
Problem solved. I was using the Docker Quickstart Terminal. Switched to Windows Command Prompt and everything running fine.
You may need to add yourself to the docker user group.
Run the following command in your shell or terminal window:
sudo usermod -a -G docker ${USER}
where user is the google user with which I log in to the console, after executing the command it is necessary to restart the terminal.
Related
I am running a python script that downloads CSV file using some API with a docker run command.
I am using one Dockerfile to install all the OS level dependencies and requirements.
Once the build is created, I am using the following command :
docker run -v $(pwd)/Reports:/usr/src/app/Reports --rm ImgName python myScript.py -d 2015-11-25
As mentioned in the above command I have one directory named Reports.
I have mounted that directory with Docker.
The Script Executes successfully and downloads a CSV file but The Problem is, The Downloaded CSV file is in read-only mode. I am not able to delete it.
I need to have the flexibility to delete any file downloaded via script.
Note: When I run the script without docker i.e. python myScript.py I can read, write and delete the file.*
Any feedback will be appreciated.
you can run docker with your own user with the following command:
docker run --user $(id -u):$(id -g) ...
This will make the container run as your user and all the files will be created with the right owner (see docker run docs). Don't forget to delete the files you already have there or create a new folder for this
I'm working on a project with python where I want to automate docker containers creation. I have the project folder already with includes all the files required to create the image.
One of these is create_image.sh
docker build -t my_container:latest .
Currently I do:
sudo bash create_image.sh
But now I need to automate this process from python.
I have tried:
import os
import subprocess
subprocess.check_call("bash -c '. create_image.sh'", shell=True)
But I get this error:
CalledProcessError: Command 'bash -c '. create_image.sh'' returned non-zero exit status 1.
EDIT:
The use case is to automate containers creation through an API, I have the code in flask and python until this point, where I got stuck in the images creation from the docker file. The rest is automated from templates.
You can try:
subprocess.call(['sudo', 'bash', 'create_image.sh' ])
which is equivalent of
sudo bash create_image.sh
Note: Let me say that there are better ways of automating docker container creation - please check docker-compose which can build and start the container easily. If you can elaborate more on the use case, we could help you with an elegant solution for docker. It might not be a python problem
EDIT:
Following the comments, it would be better to create a docker-compose and makefile is used to issue docker commands. Inspiration - https://medium.com/#daniel.carlier/how-to-build-a-simple-flask-restful-api-with-docker-compose-2d849d738137
In case that's because your user can't run docker without sudo, probably it's better to grant him docker API access by including him to docker group: https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo
Simply adding user to docker group:
sudo gpasswd -a $USER docker
Also if You want to automate docker operations on python, I'd recommend to use python library for docker: How to build an Image using Docker API Python Client?
I followed the graph-tool docker installation instructions here. I've set up Docker Toolbox (can't use Docker for Windows, not on Pro), and I've gotten jupyter running with the Docker image.
However, I need to access a notebook in my C: drive. For the sake of this post let's say the notebook is in C:\Users\Gab\Desktop. I've successfully moved into that location, but when I run the command docker run -p 8888:8888 -p 6006:6006 -it -u user -w /home/user tiagopeixoto/graph-tool bash, it opens a bash in /home/user, not in the directory I cd'd into previously.
From what I understand, the -w /home/user is what tells it where to open, but I'm not sure how to tell it to open in the Desktop folder.
How can I set things up properly so that I can run the command jupyter notebook --ip 0.0.0.0, and still be able to access the notebook I need?
Thanks!
Here's the deal with Docker. When you execute the docker run command, what happens is that an entirely new subsystem is created which is separate from your host Operating System ( this is known as a Docker container ). This subsystem runs the tiagopeixoto/graph-tool image in it so hence the graph-tool ( and hence jupyter-notebook ) and the entrypoint /home/user is present inside this system instead of the host OS you use to run the Docker container ( in your case its Windows ). Unfortunately for you the notebook that you wish to view using jupyter-notebook isn't present inside the container and is located elsewhere ( Windows to be exact ).
What you can do in this case is mount a folder of the host Operating System to the container such that this folder contains the notebook you wish to view :-
Open a new command prompt, and type in this command as follows :- docker run -p 8888:8888 -p 6006:6006 -v /c/Users/Gab/Desktop:/mnt/temp/ -it -u user -w /home/user tiagopeixoto/graph-tool bash
The main difference to note here is the -v switch which mounts the C:\Users\Gab\Desktop volume from the host system to the Docker container in /mnt/temp/. Once that is done try viewing the notebook present in /mnt/temp in jupyter-notebook.
According to this post there exists an issue related to mounting a volume in Windows, so please check this out as well :- docker toolbox mount file on windows
I just started a django project with docker, I used the cookiecutter-django template that is discussed in the book Two scoops of django.
I am trying to set it all up in OSX, but I am having some trouble with the following part from the documentation:
Saving changes
If you are using OS X or Windows, you need to create a
/data partition inside the virtual machine that runs the docker deamon
in order make all changes persistent. If you don’t do that your /data
directory will get wiped out on every reboot. To create a persistent
folder, log into the virtual machine by running:
$ docker-machine ssh dev1
$ sudo su
$ echo 'ln -sfn /mnt/sda1/data /data' >>
/var/lib/boot2docker/bootlocal.sh
However, if I execute these commands, and try to start my docker project, I get the following error:
ERROR: Cannot start container 182a38022fbdf65f7a64f1ca5475a3414084d11c91f1cb48bffc6f76491baf4a: mkdir /data: file exists
I'm quite stuck at this point, do you guys have an idea what I could do to get this up and running?
So in the end this was fixed by making the directory in the local machine. I did that with the following code when adding the line to the bootlocal.sh file:
$ mkdir /mnt/sda1/data
Hi there I'm trying to deploy my python app using Travis CI but I'm running into problems when I run the "travis setup heroku" command in the cmd prompt.
I'm in my project's root directory, there is an existing ".travis.yml" file in that root directory.
I've also installed ruby correctly and travis correcty because when I run:
"ruby -v" I get "ruby 2.2.2p95 (2015-04-13 revision 50295) [x64-mingw32]"
"travis -v" I get "1.7.7"
When I run "travis setup heroku" I get this message "The system cannot find the path specified" then prompts me for a "Heroku API token:"
What's the issue?
If you hadn't had Heroku Toolbelt setup to the $PATH environment variable during installation, here are some steps to check:
Check if Heroku toolbelt is set in PATH variable. If not, cd to your Heroku toolbelt installation folder, then click on the address bar and copy it.
Go to the Control Panel, then click System and Advanced System Protection.
Go to Environment Variables, then look for $PATH in the System Variables
After the last program in the variable, put a ; then paste in your Heroku CLI folder and click OK. (This requires cmd to be restarted manually)
Login to Heroku CLI
grab the token key from heroku auth:token
run travis setup heroku if the setup goes smoothly, you shouldn't get the command not found and prompt you for heroku auth key. It will ask that you want to encrypt the auth key (highly recommend) and verify the information you provided with the toolbelt and Travis CLI.
commit changes
you should be able to get your app up and running within your tests.