I wish to debug python code which is deployed remotely on an edge device. The device has aarch64 cpu architecture and does not have connectivity to internet (It serves as an access point itself).
I tried debugging using vscode Remote-SSH ,
but that doesn't work.I tried a few versions of the plugin: 0.49, 0.51 and also changed settings such as Remote.SSH: Use Local Server , but it always fail,
since host can't run wget/curl to download data.
I also tried using pydevd and debugpy, but it seems they do not support aarch64.
Any suggestions on how to use one of the above, or another tool to get the job done are most appreciated.
To answer my own question,
I was able to clone debugpy and do the following changes:
deleted:
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_linux_amd64.so
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_linux_x86.so
modified: src/debugpy/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/compile_linux.sh
src/debugpy/_vendored/pydevd/pydevd_attach_to_process/linux_and_mac/compile_mac.sh and commented out the lines for building amd64/x86
I was than able to build and deploy manually the python site package.
When running it on the device, there is a warning about amd64, but seems to be working.
Related
I have PyCharm on my machine (8GB RAM). I am required to to heavy data processing, and would like to use an institutionally provided server. This server has Python installed, but without any IDE. So all I see is a CUI, and it is difficult to program in such an environment.Also note that I cannot ask server admin to install software on the server for me. So, how can one connect one's local PyCharm to a python installed on a remote server? Is this even possible?
You can configure an interpreter using SSH:
Open the Add Python Interpreter dialogue
In the left-hand pane of the Add Python Interpreter dialogue, click SSH Interpreter.
Follow the wizard.
For more detailed instructions, check:
https://www.jetbrains.com/help/pycharm/configuring-remote-interpreters-via-ssh.html
Note: unfortunately, this option is not available in the PyCharm Community Edition.
I'm on a Windows 10 machine. I have GPU running on the Google Cloud Platform to train deep learning models.
Historically, I have been running Jupyter notebooks on the cloud server without problem, but recently began preferring to run Python notebooks in VS Code instead of the server based Jupyter notebooks. I'd like to train my VS Code notebooks on my GPUs but I don't have access to my google instances from VS Code, I can only run locally on my CPU.
Normally, to run a typical model, I spin up my instance on the cloud.google.com Compute Engine interface. I use the Ubuntu on the Windows Subsystem for Linux installation and I get in like this:
gcloud compute ssh --zone=$ZONE jupyter#$INSTANCE_NAME -- -L 8080:localhost:8080
I have tried installing the Cloud Code extension so far on VS Code, but as I go through the tutorials, I always sort of get stuck. One error I keep experiencing is that gcloud won't work on anything EXCEPT my Ubuntu terminal. I'd like it to work in the terminal inside VS Code.
Alternatively, I'd like to run the code . command on my Ubuntu command line so I can open VS Code from there, and that won't work. I've googled a few solutions, but they lead me to these same problems with neither gcloud not working, nor code . working.
Edit: I just tried the Google Cloud SDK installer from https://cloud.google.com/sdk/docs/quickstart-windows
and then I tried running gcloud compute ssh from the powershell from within VSCODE. This is the new error I got:
(base) PS C:\Users\user\Documents\dev\project\python> gcloud compute ssh --zone=$ZONE jupyter#$INSTANCE_NAME -- -L 8080:localhost:8080
WARNING: The PuTTY PPK SSH key file for gcloud does not exist.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: The private SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
ERROR: (gcloud.compute.ssh) could not parse resource []
It still runs from Ubuntu using WSL, I logged in fine. I guess I just don't know entirely enough about how they're separated, what's shared, and what is missing, and to how to get all my command lines using the same stuff.
It seems as if your ssh key paths are configured correctly for your Ubuntu terminal but not for the VS Code one. If your account is not configured to use OS Login, with which Compute Engine stores the generated key with your user account, local SSH keys are needed. SSH keys are specific to each instance you want to access and here is where you can find them. Once you have find them you can specify their path using the --ssh-key-file flag.
Another option is to use OS Login as I have mentioned before.
Here you have another thread with a similar problem than yours.
I'm actually learning to use the Django framework with PostgreSQL with Docker and docker-compose.
Regularly, when I make a mistake (for example a syntax error in the views.py file), I cannot reach my Django app anymore trough my web browser.
Firefox tells me:
Unable to connect
Firefox can't establish a connection to the server at localhost:8000
Chrome tells me:
This site can’t be reached
localhost refused to connect.
ERR_CONNECTION_REFUSED
I had this several times and I always managed to find the error in my code, to correct it and then everything went well again.
Currently, my code is working fine. But if I encounter this again (and this happens very often), I would like to be able to find the error quickly by myself.
So here is my question:
How can I see which file at which line contains the error ?
I would like to have a correct error message telling me what went wrong instead of that annoying ERR_CONNECTION_REFUSED browser page over and over.
I hope I explained my issue well because I struggled to describe it to Google.
Thanks a lot in advance. :)
FYI:
Ubuntu 18.04.3 LTS Bionic (window manager i3wm)
Docker 19.03.4
docker-compose 1.17.1
python 3.7 (docker image)
Django 2.2.6 (inside the python 3.7 image)
PostgreSQL 12.0 (docker image)
Visual Studio Code 1.39.2
I finally found a solution.
I had the bad habit to run my docker-compose in detached mode.
When attached, the syntax errors are shown directly from the terminal when de container is stopped.
I also added a script where I run my server in a loop. This way, the server will relaunch automatically over and over until I correct the error. I don't have to restart my Django server manually.
Thank you for helping me anyway.
Is there any way to use the python extension to edit files that reside on a remote server? I have tried NFS and remoteFS, but I do not see any way to get Intellisense working using the remote installation. I normally edit and test on a windows machine, while the target runs on Linux.
I realise this is not limited to this extension, but is a more general issue.
Visual Studio Code now officially supports this using an Extension: Remote SSH
Read the release notes here: https://code.visualstudio.com/blogs/2019/05/02/remote-development
Today we're excited to announce the preview of three new extensions for Visual Studio Code that enable seamless development in Containers, remotely on physical or virtual machines, and with the Windows Subsystem for Linux (WSL). You can get started right away by installing the Remote Development Extension Pack.
As a workaround, I'm using a Linux Hosted virtual machine which has a similar setup as the target. This works surprisingly well. It is a shame VMware 12 removed support for unity.
I use SSHFS (wikipedia) (github repo)
sshfs OWN_USER#SERVER:/PATH_TO_FILES/ MOUNT_POINT
This makes the remote files visible to any program on your computer, as-if they were local files, through a virtual "FUSE" filesystem.
If your own user can't access the files (you need root or some other user), you can sudo like so:
sshfs -o sftp_server="sudo -u SYSTEM_USER /usr/libexec/openssh/sftp-server" \
OWN_USER#SERVER:/PATH_TO_FILES/ MOUNT_POINT
You can install sshfs for Linux, Mac, or Windows, check out Digital Ocean's guide in my first link.
Don't forget to umount, fusermount -u, or eject that MOUNT_POINT once you're done.
I don't know if other VS Code plugins like IntelliSense would work with this. They should because the sshfs makes the files visible just like any others. But, it would require that the python tool chain you have installed locally be the same on your laptop and on the server. It'd be interesting to find out.
Or, Microsoft just announced some new plugins on the way
Yes there are some. I used this one. It allows to synchronize code between local and remote server.
You will have to keep copy on local host and it can be configured to automatically update code on remote.
https://gurumantra.themillennialpost.info/2020/05/edit-linux-files-remotely-in-vscode.html
Download and install vscode in your localPc if you don’t have it. (click here to download vscode)
Summary :
Install Vscode, remote vscode – LocalPC
Install ssh and Rmate – RemotePc
Ready to access files/data
Detailed Steps:
https://gurumantra.themillennialpost.info/2020/05/edit-linux-files-remotely-in-vscode.html
I am currently running Python 3.5 scripts on two VM instances on GCP from a local PyCharm session running on my Mac (see below for detailed environment specifications).
I have two different projects in GCP which look similar. I reviewed their setup with our cloud admin and we can't see any major difference, at least any trivial one. I created two Deep Learning images on GCP using the following cloud SDK command line, one within each project:
export PROJECT=[MY_PROJECT_NAME]
export INSTANCE_ROOT=$USER-vm
export ZONE=europe-west4-a
export IMAGE_FAMILY=tf-latest-gpu
export INSTANCE_TYPE=n1-highmem-8
export GPU_TYPE=v100
export GPU_COUNT=1
export INSTANCE_NAME=$INSTANCE_ROOT-$GPU_TYPE-$GPU_COUNT
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator=type=nvidia-tesla-$GPU_TYPE,count=$GPU_COUNT \
--machine-type=$INSTANCE_TYPE \
--boot-disk-size=200GB \
--metadata=install-nvidia-driver=True \
--scopes=storage-rw
Both images are completely similar.
I configured two remote ssh interpreters in PyCharm and deployed my Python code on both virtual machines. Everything is absolutely similar in terms of VM instance configuration (OS, Python version / libs, source code, etc.) and PyCharm remote interpreter configuration.
In both cases, the ssh ingress connection to the instance (on port 22) works pretty well.
Yet, when calling plt.show() to display images using matplotlib, the images get displayed in one setup but not in the other one.
This is not a matter of setting the proper ssh configuration (-X option on the command line, X11Forwarding, etc.). I already checked that, and anyway one of my VMs does a pretty good job of displaying my images within this configuration.
I debugged the execution and discovered that PyCharm automatically handles X display by implementing its own matplotlib FigureCanvas. When in remote ssh, the show() function actually opens a socket on the defined host (i.e. my local Mac) and sends the buffer to be displayed:
sock = socket.socket()
sock.connect((HOST, PORT))
[..]
sock.send(buffer)
This is precisely where my two configurations diverge:
The one working tries to connect on localhost:53725 and succeeds:
<socket.socket fd=28, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 42316), raddr=('127.0.0.1', 53725)>
The one failing tries to connect on localhost:53725 as well but gets an exception.
My strongest assumption is that some network configuration in the two GCP projects differs somehow and prevents the connection on localhost:53725 for the second one.
However, beyond that I have no idea what might happen and/or how to fix it.
Any idea / suggestion will be appreciated.
Thanks,
Laurent
--
Detailed environment specifications:
PyCharm 2018.2.4 (Professional Edition)
Build #PY-182.4505.26, built on September 19, 2018
Licensed to PyCharm Evaluator
Expiration date: October 27, 2018
JRE: 1.8.0_152-release-1248-b8 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14
Ok. It seems to be a bug and I found a workaround.
I share it as it might save hours of troubleshooting and debugging to anyone stumbling on the same problem.
The problem actually occurs when you remain in the same PyCharm session and switch from one interpreter to the other one.
If you quit PyCharm and start it again, the local display will work with either of the interpreters / VMs you run first. Then, if you switch to the second one it fails.
Everything looks as if there were some kind of lock set on the port or anywhere else by PyCharm which prevents you from switching seamlessly from one interpreter to another.
I'll share these insights with the PyCharm support team. BTW, other than that, this local display feature with remote interpreters is awesome and works just fine.