I want to set %xmode Verbose to get error messages in greater detail, but since I always need it, I rather want to set it at launch.
So I tried adding it on at the bottom of my jupyter_notebook_config.py, but then when I launch it again, it got the following error:
Exception while loading config file /Users/me/jupyter/jupyter_notebook_config.py
SyntaxError: invalid syntax
However, I also added import os, sys, random, asyncio, argparse at the bottom of the file, which did not lead to the error. So I feel the magic command does not work at the config file.
So is there any way to enable Verbose debug mode at default in Jupyter notebook?
Clarification
This question is about Jupyter notebook (open via jupyter notebook), not IPython (open via ipython). I found the %xmode command works in IPython config file (which I have actually used) and want to enable it on Jupyter notebook as well, which is my point.
Go to ~/.ipython/profile_default/startup.
Create a file named 00-set-traceback-mode.py.
Add the following lines:
from IPython import get_ipython
ip = get_ipython()
ip.InteractiveTB.set_mode(mode="Verbose")
You have done.
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
x = np.linspace(1,1000)
plt.plot(np.linspace(1, 1000))
print("Works")
plt.show()
I am trying to run the simple code above within PyCharm on a remote machine, but showing the plots on my local machine (mac). The plot does not appear. I do have xQuartz X11 Server running.
Pycharm runs the remote interpreter fine.
If I run it from macOS terminal, using
ssh -X pier#129.168.0.181
python test.py
plt.show() works.
I reckon that the missing piece is the -X which enables the X11 to be forwarded to my local machine.
Where do I include this with PyCharm's command to ssh? I'm spending too much time trying to figure this out...
Note: I'm also not able to use PyCharm's Python Console to do plotting. No errors are shown but the plot is not forwarded to my local machine.
Ok, I found I needed to do two things to get it working well enough for me :
(1) Set DISPLAY = localhost:10.0 in the Environment Variables under Build, Execution, Deployment -> Python Console
(2) Right after
import matplotlib
matplotlib.use('Qt5Agg')
With this, I can use the remote interpreter as if it were local.
Building of #Ippiers answer, on windows this works via:
install xming (and have it running)
run a putty session with X11 forwarding enabled
env on the putty session and check the DISPLAY variable, it will probably be localhost:10.0
Set this display variable in pycharms run configuration DISPLAY=localhost:10.0
matplotlib.use('TkAgg') as Qt5 gave me errors
My local machine and remote server are both Ubuntu. And I have succeed setting remote debugging in pycharm, all things work well excepting plotting figure.
I tried matplotlib but always failed to install pygtk or gtk package in conda py27 environment.
I also tried ssh -x and it works well in terminal. I can see figures on my local machine. But it is not convenient to work on both pycharm and terminal.. I didn't find where can set '-x' argument in pycharm, is it possible to do that? The codes always broke by errors no 'DISPLAY' Variable.
_tkinter.TclError: no display name and no $DISPLAY environment variable
I have checked "ForwardX11 yes". I tried to add 'DISPLAY' in pycharm environment, by setting "DISPLAY my_local_ip:0.0" but the program just suspend at the plotting step. it seems the figure has been plot somewhere? just not my local machine....
Any one know how to deal with such issues?
Test code:
from __future__ import print_function
import os
print("Display is: %s" % (os.environ['DISPLAY']))
import matplotlib
print("Default backend is: %s",matplotlib.get_backend())
matplotlib.use('TkAgg')
print("Backend is now: %s",matplotlib.get_backend())
from matplotlib import pyplot
pyplot.ioff()
pyplot.plot([1, 2, 3])
pyplot.show()
Lately I have been doing some interactive work in Python.
My setup is an IPython notebook running on a server that uses a grid engine to manage jobs.
Today I was trying to get an IPython cluster going following an example posted here that uses subprocess.Popen to start a the cluster.
I couldn't get the example to work so I tried opening up the IPython/Jupyter terminal emulator and typing in the ipcluster start command and the cluster started right up!
After playing around with things for a while I realized that if I typed env in the terminal emulator I got a different list of environment variables than when I looked at the os.environ variable in Python. The source of the problem seemed to be that the PATH variables were different.
Now I know that I can change the PATH variable in os.environ, but I'm wondering why it is different in the first place? I know very little about environment variables, so this may be a stupid question, but I would have assumed that a terminal emulator and notebook running on the exact same node in the exact same IPython notebook server would have had the exact same environment variables.
Any insight on why the environment variables in the terminal and notebook might be different will be greatly appreciated.
Update: In case it matters, the server I am working on uses the Univa Grid Engine. Also I have noticed that it seems to make a difference whether I use qrsh or qsub to start the notebook server.
Previously I had been using qsub, but by starting the notebook server with qrsh I eliminate many of the differences between env and os.environ. There are still differences, but much fewer. Still not sure what any of that means though:)
As per manual page for qsub, qsh, qrsh, to propagate current shell environment to the job use the -V option:
-V Available for qsub, qsh, qrsh with command and qalter.
Specifies that all environment variables active within the qsub utility be exported to the context of the job.
All environment variables specified with -v, -V or the DISPLAY variable provided with -display will be exported to the defined JSV instances only optionally when this is
requested explicitly during the job submission verification. (see -jsv option above or find more information concerning JSV in jsv(1))
I have a python code doing some calculation on a remote machine, named A. I connect on A via ssh from a machine named B.
Is there a way to display the figure on machine B?
Sure, you can enable X11 forwarding. Usually this is done by passing the -X or -Y option to ssh when you connect to the remote computer
ssh -X computerA
Note that the SSH daemon on computer A will also have to be configured to enable X11 forwarding. This is done by putting
X11Forwarding yes
in computer A's sshd_config configuration file.
If computer A's SSH daemon does not have X11 forwarding enabled, you can always have Python write the result of the calculation to a text file, download it to computer B, and use Matplotlib locally.
If you use matplotlib on Mac OS X on the remote machine (B), you must first make sure that you use one of the X11-based display back-ends, since the native Mac OS X back-end cannot export its plots to another display. Selecting a back-end can be achieved with
import matplotlib
matplotlib.use('GTK') # Or any other X11 back-end
The list of supported back-ends can be obtained by giving use() an incorrect back-end name: matplotlib then prints an error message listing the possible back-ends.
ssh X11 forwarding can then be used to display matplotlib plots.
The following worked for me using Mac OS X on the local machine (machine B) and ubuntu on the remote (machine A).
You need X11 server installed on your local machine to do this.
If you're running a recent version of Mac OSX (OS X Mountain Lion or newer), it would NOT have come with X11 pre-installed (see http://support.apple.com/kb/ht5293). Check if you have X11 by opening up Mac terminal, and run command xterm.
If an X11 window opens up, you're all set. If it says command not found, then go to http://xquartz.macosforge.org/landing/ and install X11 server. Then logout and log back in to your mac.
After you log back in, try to run xterm command again. It should open up X11 window.
At this point your $DISPLAY variable should also be set correctly. If it's not set, make sure you've logged in/out since installing X11 from XQuartz.
echo $DISPLAY
/tmp/launch-I9I3aI/org.macosforge.xquartz:0
Then from your local machine, use ssh -X to remote into remote machine A:
ssh -X user#machineA
Then on the remote machine:
python
>>> import matplotlib
>>> matplotlib.use('GTKAgg') #I had to use GTKAgg for this to work, GTK threw errors
>>> import matplotlib.pyplot as plt #... and now do whatever you need...
Make sure you call matplotlib.use BEFORE importing anything else from matplotlib (e.g. matplotlib.pyplot)
Other useful troubleshooting tips on using ssh -X : http://oroborosx.sourceforge.net/remotex.html#usessh
GTK seems impossible to get working on Ubuntu with Python3. Instead, I used tkagg (from this answer):
import matplotlib
matplotlib.use('tkagg')
import matplotlib.pyplot as plt
Test that it's working with this:
import matplotlib
matplotlib.use('tkagg')
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
plt.show()
I have used IPython to solve the related problem. The steps are as follows:
Step 1: Install IPython and Jupyter in the remote machine (A) locally (assuming no root privilege) using the following commands:
pip install --user ipython
pip install --user jupyter
Update matplotlib:
pip install --user -U matplotlib
Step 2:
Run Jupyter with no browser from the code directory in the remote machine (A):
cd PATH/TO/THE/CODE
jupyter notebook --no-browser --port=8080
After this command, a URL will be given something similar to below:
http://localhost:8080/?token=5528ab1eeb3f621b90b63420f8bbfc510edf71f21437c4e2
Step 3:
Now open another terminal in the local machine (B) and connect to the remote machine (A) using ssh:
ssh -N -L 8080:localhost:8080 user_id#remote.host
The port number has to be same in step 2 and step 3. In this example, the port number is 8080.
Step 4:
Copy and paste the URL in the step 3 to a browser in your local machine (B).
Now, the notebook in the remote machine can be used through the browser and plot can be generated using the data in the remote machine.
export MPLBACKEND="agg" this worked for me.
obviously you can set it via code as well.
if that doesn't work you could also try:
import matplotlib.pyplot as plt
plt.switch_backend('agg')
or
import matplotlib.pyplot as plt
plt.switch_backend('TkAgg')
this seemed to work for me
Yet, if you are trying to get a GUI working I suggest you look at this link: http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/
Just wanted to add - if you're on Windows as the local machine, make sure you've set up Xming (an X Windows server) and Putty so you can see the remote Linux graphical applications.
I followed the instructions from here: http://laptops.eng.uci.edu/software-installation/using-linux/how-to-configure-xming-putty to do this. It also sets your display environment and variable so you don't get an error when using tkagg as the backend.
You can use the library I created to solve this problem https://pypi.org/project/remote-plot/.
It uses the exact same API as matplotlib, but it renders the plots in a web browser which you can view from your machine B.
Install by:
pip install remote_plot
And then run in python like this:
from remote_plot import plt
plt.plot([1, 2, 3], [4, 5, 6])
By default it opens the rendering on port 8000 but you can modify this easily. If you are connecting via ssh, don't forget to forward the port by adding the following flag to your ssh command:
ssh YOUR_USER_NAME#YOUR_MACHINE_IP -L 8000:localhost:8000
If you're using a mac as a client machine, try this.
You basically need to make sure two things are working properly.
Using GTK or Cairo as matplotlib's backend.
Forwarding the display.
Using GTK or Cairo as matplotlib's backend
If you're using python3, you must install cairocffi.
pip install cairocffi
Then use the GTK3Agg as a backend of matplotlib.
import matplotlib
matplotlib.use('GTK3Agg')
import matplotlib.pyplot as plt
See following description from matplotlib document for more detail.
Both GTK2 and GTK3 have implicit dependencies on PyCairo regardless of
the specific Matplotlib backend used. Unfortunatly the latest release
of PyCairo for Python3 does not implement the Python wrappers needed
for the GTK3Agg backend. Cairocffi can be used as a replacement which
implements the correct wrapper.
Forwarding the display
Install launch the latest version of XQuartz.
Connect to the remote server using ssh -X. ex) ssh username#ipaddress -X
This is what I did with MacOS and a Linux remote machine.
In ~/.ssh/config I added the following. I add this since it's possible that your machine might not be taking the xauth location properly.
XAuthLocation /opt/X11/bin/xauth
ssh -Y machine_name
Ran the following dummy program:
import matplotlib.pyplot as plt
x = [1,2,3]
y = [2,4,1]
plt.plot(x, y)
plt.show()