Develop Reference

Post request in JupyterLab server extension breaks when calling imported function

I'm working on creating a JupyterLab server-side extension from this template and am having trouble calling an imported python function in the post method of handlers.py. The post request is sent from my index.ts file with a body containing the relative path of the active notebook when the user presses the "Measure Energy Usage" button. Sending it works fine if I leave out the call to measure_cell2.measure() in handlers.py. However, when I try to call that function, I get these errors on the right from the post request:
In the terminal running jupyter lab, I get this error:
[W 2021-07-13 12:42:47.633 ServerApp] 404 POST /jlab-ext-example/hello?1626198167480 (127.0.0.1) 7.70ms referer=http://localhost:8888/lab/tree/jupyter_lab_ext/jlab_ext_example/nbtest2.ipynb
I know that abs_path is the correct path, because when I leave out the measure function call in handlers.py, simply print the path returned by os.path.abspath, and then run the measure function directly from the python console or the terminal using that path, it produces the desired output. The post method in handlers.py doesn't seem to be executing at all since print(abs_path) is never executed and it comes before the call to measure. What could be causing this error?
JupyterLab 3.0.16,
Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-143-generic x86_64),
Python 3.9.5,
Node v14.8.0,
tsc 2.7.2
index.ts:
import {
ILabShell,
JupyterFrontEnd,
JupyterFrontEndPlugin
} from '#jupyterlab/application';
import { IRetroShell } from '#retrolab/application';
import { ICommandPalette } from '#jupyterlab/apputils';
import { ILauncher } from '#jupyterlab/launcher';
import { requestAPI } from './handler';
import { ToolbarButton } from '#jupyterlab/apputils';
import { DocumentRegistry } from '#jupyterlab/docregistry';
import { INotebookModel, NotebookPanel } from '#jupyterlab/notebook';
import { IDisposable } from '#lumino/disposable';
import { Title, Widget } from '#lumino/widgets';
var nbPath: string; // Holds relative path of notebook to convert
// Create button on the toolbar
export class ButtonExtension implements DocumentRegistry.IWidgetExtension<NotebookPanel, INotebookModel> {
constructor(app: JupyterFrontEnd) {
this.app = app;
}
readonly app: JupyterFrontEnd
// Create the toolbar button
createNew(panel: NotebookPanel, context: DocumentRegistry.IContext<INotebookModel>): IDisposable {
let mybutton = new ToolbarButton({
label: 'Measure Energy Usage',
onClick: async () => {
// Post request to Jupyter server containing relative
// path to .ipynb file
const dataToSend = { file: nbPath };
try {
const reply = await requestAPI<any>('hello', {
body: JSON.stringify(dataToSend),
method: 'POST'
});
console.log(reply);
} catch (reason) {
console.error(
`Error on POST /jlab-ext-example/hello ${dataToSend}.\n${reason}`
);
}
}
});
// Add the toolbar button to the notebook toolbar
panel.toolbar.insertItem(10, 'MeasureEnergyUsage', mybutton);
console.log("MeasEnerUsage activated");
// The ToolbarButton class implements `IDisposable`, so the
// button *is* the extension for the purposes of this method.
return mybutton;
}
}
/**
* Initialization data for the server-extension-example extension.
*/
const extension: JupyterFrontEndPlugin<void> = {
id: 'server-extension-example',
autoStart: true,
optional: [ILauncher],
requires: [ICommandPalette],
activate: async (
app: JupyterFrontEnd
) => {
console.log('JupyterLab extension server-extension-example is activated!');
const your_button = new ButtonExtension(app);
app.docRegistry.addWidgetExtension('Notebook', your_button);
// Get name and relative path of active notebook tab
var shell = app.shell as ILabShell | IRetroShell;
// When user changes tabs, update nbPath to be the relative path for current notebook
const onTitleChanged = (title: Title<Widget>) => {
nbPath = title.caption;
nbPath = nbPath.slice(nbPath.indexOf("Path: ") + 6, nbPath.indexOf("Last Saved") - 1);
console.log(nbPath);
};
// Watch for user changing tabs in JupyterLab
shell.currentChanged.connect((_: any, change: any) => {
const { oldValue, newValue } = change;
if (oldValue) {
oldValue.title.changed.disconnect(onTitleChanged);
}
if (newValue) {
newValue.title.changed.connect(onTitleChanged);
}
});
}
};
export default extension;
handlers.py:
import os
import json
import subprocess
from notebook.base.handlers import APIHandler
from notebook.utils import url_path_join
import tornado
from tornado.web import StaticFileHandler
import measure_cell2
class RouteHandler(APIHandler):
# The following decorator should be present on all verb methods (head, get, post,
# patch, put, delete, options) to ensure only authorized user can request the
# Jupyter server
#tornado.web.authenticated
def get(self):
self.finish(json.dumps({"data": "At the /jlab-ext-example/hello endpoint; finding and converting notebook"}))
#tornado.web.authenticated
def post(self):
# receive .ipynb file name and relative path from index.ts on client-side
# input_data is a dictionary with a key "file"
input_data = self.get_json_body()
# convert notebook to .py file with nbconvert
subprocess.run(["jupyter", "nbconvert", input_data["file"], "--to", "script"])
# get name and absolute path of converted .py file
python_file_name = input_data["file"]
extension_index = python_file_name.find(".")
python_file_name = python_file_name[0:extension_index] + ".py"
abs_path = os.path.abspath(python_file_name)
print(abs_path)
# measure code energy usage
measure_cell2.measure(abs_path)
# confirm conversion and measurement occurred
data = {"greetings": "Converted notebook to the python file {}; now measuring energy usage".format(python_file_name)}
self.finish(json.dumps(data))
def setup_handlers(web_app, url_path):
host_pattern = ".*$"
base_url = web_app.settings["base_url"]
# Prepend the base_url so that it works in a JupyterHub setting
route_pattern = url_path_join(base_url, url_path, "hello")
handlers = [(route_pattern, RouteHandler)]
web_app.add_handlers(host_pattern, handlers)
# Prepend the base_url so that it works in a JupyterHub setting
doc_url = url_path_join(base_url, url_path, "public")
doc_dir = os.getenv(
"JLAB_SERVER_EXAMPLE_STATIC_DIR",
os.path.join(os.path.dirname(__file__), "public"),
)
handlers = [("{}/(.*)".format(doc_url), StaticFileHandler, {"path": doc_dir})]
web_app.add_handlers(".*$", handlers)
measure_cell2.py:
import requests
def measure(file_path):
base_url = 'http://localhost:9898'
# send file and get ID
url_id = base_url + '/id'
print(file_path)
files = {'file': open(file_path, 'rb')} # opens file into buffer; rb = read buffer
getID = requests.post(url_id, files=files)
ID = getID.json().get('id')
print("Running Code at ", file_path)
print(ID)
# send ID and get gpu results
url_results_gpu = base_url + '/results/gpu?id={}'.format(ID)
gpu_results = requests.get(url_results_gpu)
print(gpu_results.text)
# send ID and get cpu results
url_results_cpu = base_url + '/results/cpu?id={}'.format(ID)
cpu_results = requests.get(url_results_cpu)
print(cpu_results.text)

How to restart kernel and rerun all cells in Python Jupyter notebook through code or by an algorithm?

Need a Python Script that will restart the kernel and rerun all the cells automatically without any human intervention.
I tried the following code below but needs a human intervention since it uses a toggle button.
from IPython.display import HTML, Javascript, display
def initialize():
display(HTML(
'''
<script>
code_show = false;
function restart_run_all(){
IPython.notebook.kernel.restart();
setTimeout(function(){
IPython.notebook.execute_all_cells();
}, 10000)
}
function code_toggle() {
if (code_show) {
$('div.input').hide(200);
} else {
$('div.input').show(200);
}
code_show = !code_show
}
</script>
<button onclick="code_toggle()">Click to toggle</button>
<button onclick="restart_run_all()">Click to Restart and Run all Cells</button>
'''
))
initialize()

modifying the code a bit seems to be working for me:
from IPython.display import HTML, Javascript, display
def initialize():
display(HTML(
'''
<script>
code_show = false;
function restart_run_all(){
IPython.notebook.kernel.restart();
setTimeout(function(){
IPython.notebook.execute_all_cells();
}, 10000)
}
function code_toggle() {
if (code_show) {
$('div.input').hide(200);
} else {
$('div.input').show(200);
}
code_show = !code_show
}
restart_run_all();
</script>
'''
))
Note: I just removed the buttons and called the restart_run_all() function before </ script>

How to execute Python code generated by Blockly right in the browser?

I was following the example Blockly Code Generators and was able to generate Python codes. But when I run the Python code, I get an error. It seems the error is 'eval(code)' below, what should I do if I want to execute the Python code right inside the browser? Thanks for any help!
Blockly.JavaScript.addReservedWords('code');
var code = Blockly.JavaScript.workspaceToCode(workspace);
try {
eval(code);
} catch (e) {
alert(e);
}
here is the snapshot Unfortunately i dont have enough points to post the image here

Can you try this with a simple code , like - print('Hello World!')
According to the image , the issue could be with indentation , and indentation is very important in python, othewise it can cause syntax errors .
You should have also changed the code to -
Blockly.Python.addReservedWords('code');
var code = Blockly.JavaScript.workspaceToCode(workspace);
try {
eval(code);
} catch (e) {
alert(e);
}

Try to eval with a Python interp written is JS, like http://brython.info/.

as you using eval function of javascript you are actually saying to print the page as print() means to print the page in javascript.
The solution to this is there in the Blockly itself as to rn the code they call this function.
function() {
Blockly.JavaScript.INFINITE_LOOP_TRAP = 'checkTimeout();\n';
var timeouts = 0;
var checkTimeout = function() {
if (timeouts++ > 1000000) {
throw MSG['timeout'];
}
};
var code = Blockly.JavaScript.workspaceToCode(Code.workspace);
Blockly.JavaScript.INFINITE_LOOP_TRAP = null;
try {
eval(code);
} catch (e) {
alert(MSG['badCode'].replace('%1', e));
}
The above code is from code.js in demos/code/code.js line 553.

Unable to get browser console logs from a remote chrome browser [duplicate]

I want to build an automation testing, so I have to know the errors that appear in the console of chrome.
there is an option to get the error lines that appear in the console?
In order to see the console: right click somewhere in the page, click "inspect element" and then go to "console".

I don't know C# but here's Java code that does the job, I hope you can translate it to C#
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.logging.LogEntries;
import org.openqa.selenium.logging.LogEntry;
import org.openqa.selenium.logging.LogType;
import org.openqa.selenium.logging.LoggingPreferences;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
public class ChromeConsoleLogging {
private WebDriver driver;
#BeforeMethod
public void setUp() {
System.setProperty("webdriver.chrome.driver", "c:\\path\\to\\chromedriver.exe");
DesiredCapabilities caps = DesiredCapabilities.chrome();
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.BROWSER, Level.ALL);
caps.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);
driver = new ChromeDriver(caps);
}
#AfterMethod
public void tearDown() {
driver.quit();
}
public void analyzeLog() {
LogEntries logEntries = driver.manage().logs().get(LogType.BROWSER);
for (LogEntry entry : logEntries) {
System.out.println(new Date(entry.getTimestamp()) + " " + entry.getLevel() + " " + entry.getMessage());
//do something useful with the data
}
}
#Test
public void testMethod() {
driver.get("http://mypage.com");
//do something on page
analyzeLog();
}
}
Pay attention to setUp method in above code. We use LoggingPreferences object to enable logging. There are a few types of logs, but if you want to track console errors then LogType.BROWSER is the one that you should use. Then we pass that object to DesiredCapabilities and further to ChromeDriver constructor and voila - we have an instance of ChromeDriver with logging enabled.
After performing some actions on page we call analyzeLog() method. Here we simply extract the log and iterate through its entries. Here you can put assertions or do any other reporting you want.
My inspiration was this code by Michael Klepikov that explains how to extract performance logs from ChromeDriver.

You can get logs this way:
Driver().Manage().Logs.GetLog();
By specifying what log you are interested in you can get the browser log, that is:
Driver().Manage().Logs.GetLog(LogType.Browser);
Also remember to setup your driver accordingly:
ChromeOptions options = new ChromeOptions();
options.SetLoggingPreference(LogType.Browser, LogLevel.All);
driver = new ChromeDriver("path to driver", options);

This is the c# code for logging the brower log from chrome.
private void CheckLogs()
{
List<LogEntry> logs = Driver.Manage().Logs.GetLog(LogType.Browser).ToList();
foreach (LogEntry log in logs)
{
Log(log.Message);
}
}
here is my code for the actual log:
public void Log(string value, params object[] values)
{
// allow indenting
if (!String.IsNullOrEmpty(value) && value.Length > 0 && value.Substring(0, 1) != "*")
{
value = " " + value;
}
// write the log
Console.WriteLine(String.Format(value, values));
}

As per issue 6832 logging is not implemented yet for C# bindings. So there might not be an easy way to get this working as of now.

Here is a solution to get Chrome logs using the C#, Specflow and Selenium 4.0.0-alpha05.
Pay attention that the same code doesn't work with Selenium 3.141.0.
[AfterScenario]
public void AfterScenario(ScenarioContext context)
{
if (context.TestError != null)
{
GetChromeLogs(context); //Chrome logs are taken only if test fails
}
Driver.Quit();
}
private void GetChromeLogs()
{
var chromeLogs = Driver.Manage().Logs.GetLog(LogType.Browser).ToList();
}

public void Test_DetectMissingFilesToLoadWebpage()
{
try
{
List<LogEntry> logs = driver.Manage().Logs.GetLog(LogType.Browser).ToList();
foreach (LogEntry log in logs)
{
while(logs.Count > 0)
{
String logInfo = log.ToString();
if (log.Message.Contains("Failed to load resource: the server responded with a status of 404 (Not Found)"))
{
Assert.Fail();
}
else
{
Assert.Pass();
}
}
}
}
catch (NoSuchElementException e)
{
test.Fail(e.StackTrace);
}
}
You could do something like this in C#. It is a complete test case. Then print the console output as String i.e logInfo in your report. For some reason, Log(log.Message) from the solution above this one gave me build errors.So, I replaced it.

C# bindings to the Chrome console logs are finally available in Selenium 4.0.0-alpha05. Selenium 3.141.0 and prior do not have support.
Before instantiating a new ChromeDriver object, set the logging preference in a ChromeOptions object and pass that into ChromeDriver:
ChromeOptions options = new ChromeOptions();
options.SetLoggingPreference(LogType.Browser, LogLevel.All);
ChromeDriver driver = new ChromeDriver(options);
Then, to write the Chrome console logs to a flat file:
public void WriteConsoleErrors()
{
string strPath = "C:\\ConsoleErrors.txt";
if (!File.Exists(strPath))
{
File.Create(strPath).Dispose();
}
using (StreamWriter sw = File.AppendText(strPath))
{
var entries = driver.Manage().Logs.GetLog(LogType.Browser);
foreach (var entry in entries)
{
sw.WriteLine(entry.ToString());
}
}
}

driver.manage().logs().get("browser")
Gets all logs printed on the console. I was able to get all logs except Violations. Please have a look here Chrome Console logs not printing Violations

Autocomplete for Python in Codemirror?

I am trying to set up an autocomplete feature for Codemirror for the Python language. Unfortunately, it seems that Codemirror only includes the files necessary for Javascript key term completion.
Has anyone built Python hint file for CodeMirror similar to the JavaScript Version?
(Edit for future reference: link to similar question on CodeMirror Google Group)

I'm the original author of the Python parser for Codemirror (1 and 2). You are correct that the Python parser does not offer enough information for autocomplete. I tried to build it into the parser when Codemirror 2 came around but it proved too difficult for my JS skills at the time.
I have far more skills now but far less time. Perhaps someday I'll get back to it. Or if someone wants to take it up, I would be glad to help.

Add python-hint.js, show-hint.js, show-hint.css. Then
var editor = CodeMirror.fromTextArea(your editor instance codes here;
editor.on('inputRead', function onChange(editor, input) {
if (input.text[0] === ';' || input.text[0] === ' ' || input.text[0] === ":") {
return;
}
editor.showHint({
hint: CodeMirror.pythonHint
});
});

< script >
var editor = CodeMirror.fromTextArea(document.getElementById("code"), {
mode: {
name: "python",
version: 3,
singleLineStringErrors: false
},
lineNumbers: true,
indentUnit: 4,
extraKeys: {
"Ctrl-Space": "autocomplete"
},
matchBrackets: true
});
CodeMirror.commands.autocomplete = function (cm) {
CodeMirror.simpleHint(cm, CodeMirror.pythonHint);
}
</script>

I start the python autocomplete with a js based on pig-hint from codemirror 3.
You can get the python-hint.js from here.
to work, you need in your html:
include simple-hint,js and python-hint.js, simple-hint.css plus codemirror.js
add this script:
<script>
CodeMirror.commands.autocomplete = function(cm) {
CodeMirror.simpleHint(cm, CodeMirror.pythonHint);
}
</script>
python-hint.js is a basic js I have created today and not reviewed in depth.

You can initialize this way also, adding extraKeys parameter to CodeMirror initialization:
CodeMirror(function(elt) {
myTextArea.parentNode.replaceChild(elt, myTextArea);
}, {
mode: "python",
lineNumbers: true,
autofocus: true,
extraKeys: {"Ctrl-Space": "autocomplete"}
});