I have just started learning spark and have been using R & Python on Jupyter notebook in my company.
All spark and Jupyter are installed on my computer locally and function perfectly fine individually.
Instead of creating .py script for pyspark in cmd every single time, could I possibly connect it to my Jupyter notebook live and run the scripts there? I have seen many posts on how to achieve that on Linux and Mac but sadly I will have to stick with Window 7 at this case.
Thanks!
Will
You could use the Sandbox from Hortonworks (http://hortonworks.com/downloads/#sandbox) and run your code in Apache Zeppelin.
No setup necessary. Install virtual box and run the sandbox. Then access zeppelin and ambari via your host (windows) browser and you are good to go to run your %pyspark code. Zeppelin has a look an feel like Jupyter.
Related
I am running Jupyter Notebook 6.4.8 on a machine in my network using the command
jupyter notebook --no-browser --port=[openport]
and forwarding the appropriate ports like so
ssh -L [openport]:localhost:[openport] user#[jupyterhost]
so that I can access the jupyter interface from another machine on my network at http://localhost:[openport] (where [openport] is the same port in all commands and [jupyterhost] is the ip address of the machine running jupyter).
This has been working relatively smoothly while running/modifying notebooks I have already uploaded to the machine. Today I started to try and use this method to start something from scratch and noticed that the new button wouldn't work. I continued to find that all the buttons on the top right hand side of the file navigation interface are not working.
File navigation toolbar
Specifically, the quit, logout, upload, new, refresh, and all file ordering buttons do nothing when clicked. All tabs on the left side (i.e., files, running, clusters) as well as all file navigation links all work. On the running tab, being able to shutdown running notebooks is something that once worked but has stopped (I'm assuming as a byproduct of trying to fix the problem but not 100% sure).
So far I have tried to upgrade ipython via
pip install ipython --upgrade
and have deleted the ~/.ipython/ folder based on advice in previous related posts (in that order). Restarting jupyter after each attempt and re-testing the interface showed no progress. I noticed the shutdown button not working on the running page only after trying both of the above proposed fixes.
The only warnings I am receiving while starting jupyter are related to the port configuration which I'm not 100% how to resolve but am also reasonably sure are unrelated to the problem.
[W 2022-08-31 23:28:18.853 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
I also have tested this on the local machine without the flags for remote operation, i.e.,
jupyter notebook
and the same buttons not working on the remote setup appear to be operational when viewed on a local browser.
Any suggestions on how I can debug/fix the interface issues would be greatly appreciated! Thanks!
I wrote a code for a Django server, and it works perfectly inside the shell of Pycharm.
Now, I want to run this server on a local computer constantly without being inside Pycharm's shell.
Also, because it's for a client of mine I don't want any open CMD windows or any other weird GUI- I want him to just access the website like any other website.
I've seen all kinds of solutions- running runserver with &, creating a virtual machine and running the server on it and etc.
I am familiar with Vmware and all, so if the proper solution is this It's OK. But I wonder- are there any other ways to run a server on a PC without installing any additional programs?
I need some advice. So I am a big fan of VS Code and I always use its embedded notebooks. I built a remote Jupyter Server on Oracle Cloud hoping I could connect from vscode. To create the server I based on this article, but migrating as advised by Jupyter to JupyterServer. I've also used miniconda isntead of venv.
The server seems to work correctly, I can access it from my browser and in my Windows Terminal SSH, open Jupyter Lab, create and run noteboooks in it, etc. The problem is when I try to use it with VS Code, when I try to specify de Jupyter Server for connections, it allows me to do it, it even prompts me that it is an insecure connection (I use self signed ssl certificate), and it does mark Jupyter Server: Remote BUT, when I try to select my interpreter, change my kernel, it only shows my local conda envs. if I run !hostname it shows me my local hostname, not my remotes, it isn't really connecting or using the remote Jupyter server to run the cells.
I've looked around and can`t find a way to make it work, I really want it to work with VS Code, any help?
This has no impact on the actual use of jupyter. Your confusion is actually a misunderstanding caused by the definition of names.
As stated in the official document, when you connect to a remote server, everything runs in the server ather than the local computer.
At present, there is an issue for changing the naming on GitHub, which you can read in detail.
Remotely accessed Jupyter notebook from a linux machine has working too slowly and ambari server hosts shows the components are down ?
Working jupyter notebook can connected via putty from that machine using no-browser commnd.
I am leading a team of analysts and want to introduce them to Jupyter Notebook as a window into Python programming.
We have Anaconda downloaded and installed on our Linux server. I've asked our IT to help set it up to run on Google Chrome and they have been able to only provide the following steps:
source /R_Data/anaconda3/etc/profile.d/conda.sh
this kicks off Anaconda on the server, must run in PUTTY. We stored the installation in the same location as RStudio hence the R_Data in the filepath.
/R_Data/anaconda3/bin/jupyter-notebook --ip 0.0.0.0 --port 8889
This sets up the port 8889 with a token generated each time from scratch. We then need to grab the token id and paste into Chrome with the full URL per step 3
http://localhost:8889/?token=ea97e502a7f45d....
When I paste this in Chrome it loads Jupyter.
While this gets the job done it seems less than ideal for an entire team of analysts to have to do this each time. We also have RStudio installed on the same server but that simply opens from Chrome using a URL since I assume it is always running in the background. Jupyter and Anaconda seem to only run once they are kicked off first in PUTTY and I would like a way to bypass those steps.
I am familiar with the Jupyter config file however my limited understanding as a non-developer tells me it applies only to each user and cannot be applied to all users simultaneously (i.e. as a root user on the server or something to that effect).
I am hoping someone here might point me in the right direction. I should also point out that as a Redhat user I can't follow instructions based in Ubuntu since that syntax seems different.
Many thanks for the help.
Yoni
A convenient way is to run jupyter notebook --no-browser --port=12345 on your server while connecting through the ssh tunel as ssh -N -f -L 12345:localhost:12345 myserveralias. Now jupyter is on your 12345 localhost. Things like AutoSSH or Keep Alive will help with an erratic network, however, take security into account.