I am trying to obtain the current NoteBook name when running the IPython notebook. I know I can see it at the top of the notebook. What I am after something like
currentNotebook = IPython.foo.bar.notebookname()
I need to get the name in a variable.
adding to previous answers,
to get the notebook name run the following in a cell:
%%javascript
IPython.notebook.kernel.execute('nb_name = "' + IPython.notebook.notebook_name + '"')
this gets you the file name in nb_name
then to get the full path you may use the following in a separate cell:
import os
nb_full_path = os.path.join(os.getcwd(), nb_name)
I have the following which works with IPython 2.0. I observed that the name of the notebook is stored as the value of the attribute 'data-notebook-name' in the <body> tag of the page. Thus the idea is first to ask Javascript to retrieve the attribute --javascripts can be invoked from a codecell thanks to the %%javascript magic. Then it is possible to access to the Javascript variable through a call to the Python Kernel, with a command which sets a Python variable. Since this last variable is known from the kernel, it can be accessed in other cells as well.
%%javascript
var kernel = IPython.notebook.kernel;
var body = document.body,
attribs = body.attributes;
var command = "theNotebook = " + "'"+attribs['data-notebook-name'].value+"'";
kernel.execute(command);
From a Python code cell
print(theNotebook)
Out[ ]: HowToGetTheNameOfTheNoteBook.ipynb
A defect in this solution is that when one changes the title (name) of a notebook, then this name seems to not be updated immediately (there is probably some kind of cache) and it is necessary to reload the notebook to get access to the new name.
[Edit] On reflection, a more efficient solution is to look for the input field for notebook's name instead of the <body> tag. Looking into the source, it appears that this field has id "notebook_name". It is then possible to catch this value by a document.getElementById() and then follow the same approach as above. The code becomes, still using the javascript magic
%%javascript
var kernel = IPython.notebook.kernel;
var thename = window.document.getElementById("notebook_name").innerHTML;
var command = "theNotebook = " + "'"+thename+"'";
kernel.execute(command);
Then, from a ipython cell,
In [11]: print(theNotebook)
Out [11]: HowToGetTheNameOfTheNoteBookSolBis
Contrary to the first solution, modifications of notebook's name are updated immediately and there is no need to refresh the notebook.
As already mentioned you probably aren't really supposed to be able to do this, but I did find a way. It's a flaming hack though so don't rely on this at all:
import json
import os
import urllib2
import IPython
from IPython.lib import kernel
connection_file_path = kernel.get_connection_file()
connection_file = os.path.basename(connection_file_path)
kernel_id = connection_file.split('-', 1)[1].split('.')[0]
# Updated answer with semi-solutions for both IPython 2.x and IPython < 2.x
if IPython.version_info[0] < 2:
## Not sure if it's even possible to get the port for the
## notebook app; so just using the default...
notebooks = json.load(urllib2.urlopen('http://127.0.0.1:8888/notebooks'))
for nb in notebooks:
if nb['kernel_id'] == kernel_id:
print nb['name']
break
else:
sessions = json.load(urllib2.urlopen('http://127.0.0.1:8888/api/sessions'))
for sess in sessions:
if sess['kernel']['id'] == kernel_id:
print sess['notebook']['name']
break
I updated my answer to include a solution that "works" in IPython 2.0 at least with a simple test. It probably isn't guaranteed to give the correct answer if there are multiple notebooks connected to the same kernel, etc.
It seems I cannot comment, so I have to post this as an answer.
The accepted solution by #iguananaut and the update by #mbdevpl appear not to be working with recent versions of the Notebook.
I fixed it as shown below. I checked it on Python v3.6.1 + Notebook v5.0.0 and on Python v3.6.5 and Notebook v5.5.0.
import jupyterlab
if jupyterlab.__version__.split(".")[0] == "3":
from jupyter_server import serverapp as app
key_srv_directory = 'root_dir'
else :
from notebook import notebookapp as app
key_srv_directory = 'notebook_dir'
import urllib
import json
import os
import ipykernel
def notebook_path(key_srv_directory, ):
"""Returns the absolute path of the Notebook or None if it cannot be determined
NOTE: works only when the security is token-based or there is also no password
"""
connection_file = os.path.basename(ipykernel.get_connection_file())
kernel_id = connection_file.split('-', 1)[1].split('.')[0]
for srv in app.list_running_servers():
try:
if srv['token']=='' and not srv['password']: # No token and no password, ahem...
req = urllib.request.urlopen(srv['url']+'api/sessions')
else:
req = urllib.request.urlopen(srv['url']+'api/sessions?token='+srv['token'])
sessions = json.load(req)
for sess in sessions:
if sess['kernel']['id'] == kernel_id:
return os.path.join(srv[key_srv_directory],sess['notebook']['path'])
except:
pass # There may be stale entries in the runtime directory
return None
As stated in the docstring, this works only when either there is no authentication or the authentication is token-based.
Note that, as also reported by others, the Javascript-based method does not seem to work when executing a "Run all cells" (but works when executing cells "manually"), which was a deal-breaker for me.
The ipyparams package can do this pretty easily.
import ipyparams
currentNotebook = ipyparams.notebook_name
On Jupyter 3.0 the following works. Here I'm showing the entire path on the Jupyter server, not just the notebook name:
To store the NOTEBOOK_FULL_PATH on the current notebook front end:
%%javascript
var nb = IPython.notebook;
var kernel = IPython.notebook.kernel;
var command = "NOTEBOOK_FULL_PATH = '" + nb.base_url + nb.notebook_path + "'";
kernel.execute(command);
To then display it:
print("NOTEBOOK_FULL_PATH:\n", NOTEBOOK_FULL_PATH)
Running the first Javascript cell produces no output.
Running the second Python cell produces something like:
NOTEBOOK_FULL_PATH:
/user/zeph/GetNotebookName.ipynb
Yet another hacky solution since my notebook server can change. Basically you print a random string, save it and then search for a file containing that string in the working directory. The while is needed because save_checkpoint is asynchronous.
from time import sleep
from IPython.display import display, Javascript
import subprocess
import os
import uuid
def get_notebook_path_and_save():
magic = str(uuid.uuid1()).replace('-', '')
print(magic)
# saves it (ctrl+S)
display(Javascript('IPython.notebook.save_checkpoint();'))
nb_name = None
while nb_name is None:
try:
sleep(0.1)
nb_name = subprocess.check_output(f'grep -l {magic} *.ipynb', shell=True).decode().strip()
except:
pass
return os.path.join(os.getcwd(), nb_name)
There is no real way yet to do this in Jupyterlab. But there is an official way that's now under active discussion/development as of August 2021:
https://github.com/jupyter/jupyter_client/pull/656
In the meantime, hitting the api/sessions REST endpoint of jupyter_server seems like the best bet. Here's a cleaned-up version of that approach:
from jupyter_server import serverapp
from jupyter_server.utils import url_path_join
from pathlib import Path
import re
import requests
kernelIdRegex = re.compile(r"(?<=kernel-)[\w\d\-]+(?=\.json)")
def getNotebookPath():
kernelId = kernelIdRegex.search(get_ipython().config["IPKernelApp"]["connection_file"])[0]
for jupServ in serverapp.list_running_servers():
for session in requests.get(url_path_join(jupServ["url"], "api/sessions"), params={"token": jupServ["token"]}).json():
if kernelId == session["kernel"]["id"]:
return Path(jupServ["root_dir"]) / session["notebook"]['path']
Tested working with
python==3.9
jupyter_server==1.8.0
jupyterlab==4.0.0a7
Modifying #jfb method, gives the function below which worked fine on ipykernel-5.3.4.
def getNotebookName():
display(Javascript('IPython.notebook.kernel.execute("NotebookName = " + "\'"+window.document.getElementById("notebook_name").innerHTML+"\'");'))
try:
_ = type(NotebookName)
return NotebookName
except:
return None
Note that the display javascript will take some time to reach the browser, and it will take some time to execute the JS and get back to the kernel. I know it may sound stupid, but it's better to run the function in two cells, like this:
nb_name = getNotebookName()
and in the following cell:
for i in range(10):
nb_name = getNotebookName()
if nb_name is not None:
break
However, if you don't need to define a function, the wise method is to run display(Javascript(..)) in one cell, and check the notebook name in another cell. In this way, the browser has enough time to execute the code and return the notebook name.
If you don't mind to use a library, the most robust way is:
import ipynbname
nb_name = ipynbname.name()
If you are using Visual Studio Code:
import IPython ; IPython.extract_module_locals()[1]['__vsc_ipynb_file__']
Assuming you have the Jupyter Notebook server's host, port, and authentication token, this should work for you. It's based off of this answer.
import os
import json
import posixpath
import subprocess
import urllib.request
import psutil
def get_notebook_path(host, port, token):
process_id = os.getpid();
notebooks = get_running_notebooks(host, port, token)
for notebook in notebooks:
if process_id in notebook['process_ids']:
return notebook['path']
def get_running_notebooks(host, port, token):
sessions_url = posixpath.join('http://%s:%d' % (host, port), 'api', 'sessions')
sessions_url += f'?token={token}'
response = urllib.request.urlopen(sessions_url).read()
res = json.loads(response)
notebooks = [{'kernel_id': notebook['kernel']['id'],
'path': notebook['notebook']['path'],
'process_ids': get_process_ids(notebook['kernel']['id'])} for notebook in res]
return notebooks
def get_process_ids(name):
child = subprocess.Popen(['pgrep', '-f', name], stdout=subprocess.PIPE, shell=False)
response = child.communicate()[0]
return [int(pid) for pid in response.split()]
Example usage:
get_notebook_path('127.0.0.1', 17004, '344eb91bee5742a8501cc8ee84043d0af07d42e7135bed90')
To realize why you can't get notebook name using these JS-based solutions, run this code and notice the delay it takes for the message box to appear after python has finished execution of the cell / entire notebook:
%%javascript
function sayHello() {
alert('Hello world!');
}
setTimeout(sayHello, 1000);
More info
Javascript calls are async and hence not guaranteed to complete before python starts running another cell containing the code expecting this notebook name variable to be already created... resulting in NameError when trying to access non-existing variables that should contain notebook name.
I suspect some upvotes on this page became locked before voters could discover that all %%javascript-based solutions ultimately don't work... when the producer and consumer notebook cells are executed together (or in a quick succession).
All Json based solutions fail if we execute more than one cell at a time
because the result will not be ready until after the end of the execution
(its not a matter of using sleep or waiting any time, check it yourself but remember to restart kernel and run all every test)
Based on previous solutions, this avoids using the %% magic in case you need to put it in the middle of some other code:
from IPython.display import display, Javascript
# can have comments here :)
js_cmd = 'IPython.notebook.kernel.execute(\'nb_name = "\' + IPython.notebook.notebook_name + \'"\')'
display(Javascript(js_cmd))
For python 3, the following based on the answer by #Iguananaut and updated for latest python and possibly multiple servers will work:
import os
import json
try:
from urllib2 import urlopen
except:
from urllib.request import urlopen
import ipykernel
connection_file_path = ipykernel.get_connection_file()
connection_file = os.path.basename(connection_file_path)
kernel_id = connection_file.split('-', 1)[1].split('.')[0]
running_servers = !jupyter notebook list
running_servers = [s.split('::')[0].strip() for s in running_servers[1:]]
nb_name = '???'
for serv in running_servers:
uri_parts = serv.split('?')
uri_parts[0] += 'api/sessions'
sessions = json.load(urlopen('?'.join(uri_parts)))
for sess in sessions:
if sess['kernel']['id'] == kernel_id:
nb_name = os.path.basename(sess['notebook']['path'])
break
if nb_name != '???':
break
print (f'[{nb_name}]')
just use ipynbname , which is practical
import ipynbname
nb_fname = ipynbname.name()
nb_path = ipynbname.path()
print(f"{nb_fname=}")
print(f"{nb_path=}")
I found this in https://stackoverflow.com/a/65907473/15497427
Related
I'm trying to create a process that dynamically watches jupyter notebooks, compiles them on modification and imports them into my current file, however I can't seem to execute the updated code. It only executes the first version that was loaded.
There's a file called producer.py that calls this function repeatedly:
import fs.fs_util as fs_util
while(True):
fs_util.update_feature_list()
In fs_util.py I do the following:
from fs.feature import Feature
import inspect
from importlib import reload
import os
def is_subclass_of_feature(o):
return inspect.isclass(o) and issubclass(o, Feature) and o is not Feature
def get_instances_of_features(name):
module = __import__(COMPILED_MODULE, fromlist=[name])
module = reload(module)
feature_members = getattr(module, name)
all_features = inspect.getmembers(feature_members, predicate=is_subclass_of_feature)
return [f[1]() for f in all_features]
This function is called by:
def update_feature_list(name):
os.system("jupyter nbconvert --to script {}{} --output {}{}"
.format(PATH + "/" + s3.OUTPUT_PATH, name + JUPYTER_EXTENSION, PATH + "/" + COMPILED_PATH, name))
features = get_instances_of_features(name)
for f in features:
try:
feature = f.create_feature()
except Exception as e:
print(e)
There is other irrelevant code that checks for whether a file has been modified etc.
I can tell the file is being reloaded correctly because when I use inspect.getsource(f.create_feature) on the class it displays the updated source code, however during execution it returns older values. I've verified this by changing print statements as well as comparing the return values.
Also for some more context the file I'm trying to import:
from fs.feature import Feature
class SubFeature(Feature):
def __init__(self):
Feature.__init__(self)
def create_feature(self):
return "hello"
I was wondering what I was doing incorrectly?
So I found out what I was doing wrong.
When called reload I was reloading the module I had newly imported, which was fairly idiotic I suppose. The correct solution (in my case) was to reload the module from sys.modules, so it would be something like reload(sys.modules[COMPILED_MODULE + "." + name])
I have recently installed xlwings on my Mac and am currently trying to write a small programme to update some data(via requests). As a test, I tried to update the cryptocurrency prices via an API and write them into excel.
Without using runpython, the code works. However as soon as I run my VBA code,
I get this error:
File "<string>", line 1
import sys, os;sys.path.extend(os.path.normcase(os.path.expandvars('/Users/Dennis/Documents/crypto;
^
SyntaxError: EOL while scanning string liberal
I have searched numerous threads and forums, but can't seem to find an answer to my problem.
For a better understanding,
my python code:
import requests, json
from datetime import datetime
import xlwings as xw
def do():
parameter = {'convert' : 'EUR'}
#anfrage über API
query_ticker = requests.get('https://api.coinmarketcap.com/v1/ticker', params = parameter)
#anfragedaten in JSON-format
data_ticker = query_ticker.json()
wb = xw.Book.caller()
ws0 = wb.sheets['holdings']
for entry in data_ticker:
# update eth price
if entry['symbol'] == 'ETH':
ws0.range('B14').value = float(entry['price_eur'])
#update btc price
if entry['symbol'] == 'BTC':
ws0.range('B15').value = float(entry['price_eur'])
if entry['symbol'] == 'NEO':
ws0.range('B16').value = float(entry['price_eur'])
if entry['symbol'] == 'XRP':
ws0.range('B17').value = float(entry['price_eur'])
now = datetime.now()
write_date = '%s.%s.%s' %(now.day, now.month, now.year)
write_time = '%s:%s:%s' %(now.hour, now.minute,now.second)
ws0.range('B2').value = write_date
ws0.range('B3').value = write_time
wb.save('holdings.xlsm')
#wb.close()
this is my vba code:
Sub update_holdings()
RunPython ("import update_holdings; update_holdings.do()")
End Sub
Solved this. I just wanted to post the solution for anyone who might be confronted with the same issue.
I went to check my xlwings.conf file, in order to see the setup for "INTERPRETER" and "PYTHONPATH". I never did editing on this, however, it was formatted incorrectly.
The correct format is:
"INTERPRETER","pythonw"
"PYTHONPATH",""
My config file was setup this way:
"PYTHONPATH","
"
"INTERPRETER","Python"
Also, the path to my python was set incorrectly by default. Even though my command line works with Anaconda python 3.6, "pythonw" used the interpreter set in .bash_profile referenced to python 2.7 which came pre-installed with macOS.
Editing the config file "INTERPRETER" solved this issue.
Thanks everyone.
I would like to leave an IPython notebook running to do some computation + show some visuals.
Once the IPython notebook has finished, I want the last cell in the IPython notebook to programmatically save the IPython notebook. Then I want to copy the notebook (with all output) to another directory to keep a record of results.
The copying bit I can code up easily, but I am not sure how to get an IPython notebook to programatically save itself? Is this possible? Thanks in advance!
I am taking #Taar's comment and making it an actual answer since it worked for the original person who asked the question and for myself.
from IPython.display import display, Javascript
display(Javascript('IPython.notebook.save_checkpoint();'))
This will create checkpoints - same thing as CTRL-s.
Note: in Jupyter, CTRL-s triggers an async process and the file save is actually completed only a few seconds later. If you want a blocking save operation in a notebook, use this little function (file_path is the path to the notebook file):
import time
from IPython.display import display, Javascript
import hashlib
def save_notebook(file_path):
start_md5 = hashlib.md5(open(file_path,'rb').read()).hexdigest()
display(Javascript('IPython.notebook.save_checkpoint();'))
current_md5 = start_md5
while start_md5 == current_md5:
time.sleep(1)
current_md5 = hashlib.md5(open(file_path,'rb').read()).hexdigest()
The ipython magic command %notebook will help you here. It is shown on this page (search for %notebook).
To save your current notebook history to the file "foo.ipynb" just enter:
%notebook -e foo.ipynb
At the point you want it to happen
In [1]:
import os
import time
import pathlib
from IPython.display import display, HTML, Javascript
html = '''
<script>
IPython.notebook.kernel.execute(
'filename = "' + IPython.notebook.notebook_name + '"'
)
</script>
'''
def save_notebook():
file_path = str(
pathlib.Path(
os.getcwd(),
filename
)
)
start_mtime = os.path.getmtime(file_path)
display(
Javascript(
'IPython.notebook.save_checkpoint();'
)
)
current_mtime = start_mtime
while start_mtime == current_mtime:
time.sleep(1)
current_mtime = os.path.getmtime(file_path)
display(HTML(html))
In [2]:
save_notebook()
This works when you have not made any new changes to the notebook.
So, I am passing a environment variable from bash to python;
#!/usr/bin/env python2
import os
#connect("weblogic", "weblogic", url=xxx.xxx.xxx.xxx:xxxx)
os.environ['bash_variable']
via wlst.sh I can print exported bash_variable, but how do I execute stored variable? Basically, I am trying to remove the original connect statement and pass a variable that has said information. Thanks
Question though, why wouldn't you called the script with the variable as an argument and use sys.argv[] ?
By example something like this.
import os
import sys
import traceback
from java.io import *
from java.lang import *
wlDomain = sys.argv[1]
wlDomPath = sys.argv[2]
wlNMHost = sys.argv[3]
wlNMPort = sys.argv[4]
wlDPath="%s/%s" %(wlDomPath,wlDomain)
wlNMprop="/apps/bea/wls/scripts/.shadow/NM.prop"
try:
print "Connection to Node Manager"
print ""
loadProperties(wlNMprop)
nmConnect(username=NMuser,password=NMpass,host=wlNMHost,port=wlNMPort,domainName=wlDomain,domainDir=wlDPath,mType='ssl',verbose='true')
except:
print "Fatal Error : No Connection to Node Manager"
exit()
print "Connected to Node Manager"
The NM.prop file is a 600 file with the username/password for the NM.
EDIT :
So from what I understand you want to do something like this :
URLS = ['t3s://Host1:Port1','t3s://Host2:Port2','t3s://Host3:Port3']
for urls in URLS:
connect('somebody','password',urls)
{bunch of commands}
disconnect()
And the values of the list URLS would be define by the environment.
The way I see it you have 3 choices :
Have 1 script per environment, more or less identical save for the URLS list
Have 1 script but with a conditionnal branching on sys.argv[1] (the environment as a parameter) and create the list there.
Have 1 script which use a parameter file for each environment according to the environment. Each parameter file containing the list in question.
Something like that :
propENV = sys.argv[1]
propPath = "/path1/path2"
propFile = "%s/%s" %(propPath,propENV)
loadProperties(propFile)
I would probably use the properties file option myself as it is more flexible from an operational standpoint...at least IMHO.
I am using a python script as an external auth option in ejabberd 2.1.6.
I wanted to start encrypting the clear text passwords that come across in the auth verification, so that they are not being stored in plain text in the backend database. When I add the following code to my python script and restart ejabberd, it hangs:
import hashlib
clear = "barfoo"
salt = "foobar"
hash = hashlib.md5( salt + clear ).hexdigest()
Does hashlib require specific priviledges to run?
When I run it as a normal user (ejabberd) it works without issue. When the python script is run within the external auth of ejabberd it hangs.
I've attempted to have it write out the 'hash' to a file and it never gets there ... if i run it as the 'ejabberd' user, it writes out to file fine.
I've tried to find information about restrictions for using this library on ubuntu without any success. Any ideas?
-sd
** 22.02.2011: Here is the full script adapted from https://git.process-one.net/ejabberd/mainline/blobs/raw/2.1.x/doc/dev.html#htoc8 :
#!/usr/bin/python
import sys
from struct import *
import hashlib
def from_ejabberd():
input_length = sys.stdin.read(2)
(size,) = unpack('>h', input_length)
return sys.stdin.read(size).split(':')
def to_ejabberd(bool):
answer = 0
if bool:
answer = 1
token = pack('>hh', 2, answer)
sys.stdout.write(token)
sys.stdout.flush()
def auth(username, server, password):
clear = "barfoo"
salt = "foobar"
hash = hashlib.md5( salt + clear ).hexdigest()
if (password == hash): return True
else: return False
def isuser(username, server):
return True
def setpass(username, server, password):
return True
while True:
data = from_ejabberd()
success = False
if data[0] == "auth":
success = auth(data[1], data[2], data[3])
elif data[0] == "isuser":
success = isuser(data[1], data[2])
elif data[0] == "setpass":
success = setpass(data[1], data[2], data[3])
to_ejabberd(success)
I have been messing with the same problem. I Can't really track down the problem with openssl binings in _hashlib. Whatever the problem is, i will have to patch python distribution sources. Not really an feasible solution. So i ended up using a pycrypto wrapper for the crypto functions which don't block in this case.
pip install pycrypto
from Crypto.Hash import MD5
m = MD5.new()
m.update("%s%s" % (salt ,clear))
h.hexdigest()
I looked into the hashlib source and while it does not seem to require too much, it does import .so files as modules and one of them hits openssl. It all looks pretty safe but if ejabberd tries to fence itself against calls to 3rd party code (or if you have SELinux or something else to that effect running), stuff can conceivably get weird. I got this in a REPL:
>>> import _md5
>>> _md5.__file__
'/usr/lib/python2.7/lib-dynload/_md5module.so'
Try this on your box and then try putting
_md5 = imp.load_dynamic('_md5', '/usr/lib/python2.7/lib-dynload/_md5module.so')
Or just
import _md5
(with the appropriate path updated to yours) in your code before the offending line and with some trace statement afterwards. Try the same with _hashlib instead of _md5 (hashlib defaults to _hashlib which wraps openssl, but if it doesn't load or doesn't have the needed hash it falls back to _md5, _sha etc.). If it's not the imports that fail/hang then you can try calling _md5.new(salt + clear) and _hashlib.openssl_md5(salt + clear) and see if it's one of them.
If it is the import at fault, then possibly a similar problem was tackled here
I don't know ejabberd, so I can't relate their solution to your problem, unfortunately.
I do have to say it, though: in all python implementations I know, = instead of == in a condition would raise a SyntaxError and that's that - the program would never even enter the main while loop.
hashlib does not require anything special. What means hangs? Where does it hang? Use pdb.set_trace() to step trough the code or use 'strace' or 'ltrace' to investigate API calls.
Try to use logging module, it can help you to watch input and output data, also check scripts permissions, let executing by user which is ejabberd, or for debug just set chmod 777 external.py.