I want to programmatically create several cells in a Jupyter notebook.
With this function I can create one cell
def create_new_cell(contents):
from IPython.core.getipython import get_ipython
shell = get_ipython()
shell.set_next_input(contents, replace=False)
But if I try to call it several times, for instance, from a for loop, like so
for x in ['a', 'b', 'c']:
create_new_cell(x)
It will only create one cell with the last item in the list. I've tried to find if there's a "flush" function or something similar but did not succeed.
Does anyone know how to properly write several cells programmatically?
I dug a bit more in the code of the shell.payload_manager and found out that in the current implementation of the set_next_input it does not pass the single argument to the shell.payload_manager.write_payload function. That prevents the notebook from creating several cells, since they all have the same source (the set_next_input function, in this case).
That being said, the following function works. It's basically the code from write_payload function setting the single parameter to False.
def create_new_cell(contents):
from IPython.core.getipython import get_ipython
shell = get_ipython()
payload = dict(
source='set_next_input',
text=contents,
replace=False,
)
shell.payload_manager.write_payload(payload, single=False)
Hope this helps someone out there ;)
from IPython.display import display, Javascript
def add_cell(text, type='code', direct='above'):
text = text.replace('\n','\\n').replace("\"", "\\\"").replace("'", "\\'")
display(Javascript('''
var cell = IPython.notebook.insert_cell_{}("{}")
cell.set_text("{}")
'''.format(direct, type, text)));
for i in range(3):
add_cell(f'# heading{i}', 'markdown')
add_cell(f'code {i}')
codes above will add cells as follows：
Related
How can I call functions defined in a different Synapse notebook after running the notebook with mssparkutils.notebook.run()?
example:
#parameters
value = "test"
from notebookutils import mssparkutils
mssparkutils.notebook.run("function definitions", 60, {"param": value})
df = load_cosmos_data() #defined in 'function definitions' notebook
This fails with: NameError: name 'load_cosmos_data' is not defined
I can use the functions with the %run command, but I need to be able to pass the parameter through to the function definitions notebook. %run doesn't allow me to pass a variable as a parameter.
After going through this Official Microsoft Documentation,
When referencing other notebook, after the exit from the referenced
notebook with exit() or without that, the source notebook script will
be executed and they will become two different notebooks which have no
relationship between them. We can’t access any variable from that
notebook, and it applies to the functions of that notebook as well.
In general programming languages as well, we can’t access the variables of a function which are local to it after its return. It is only possible when we return that variable.
Unfortunately, the exit() method doesn’t support returning values other than strings from the referenced notebook.
From the above code, assuming that you need to access the dataframe which is returning from the function load_cosmos_data() in referenced notebook. You can do it using the temporary views.
Please follow the demonstration below:
In the referenced notebook call the function and store the returned dataframe in a variable and create a temporary view for that. You can store this temporary view as dataframe in the source notebook.
Function Notebook:
Code:
from pyspark.sql.types import StructType,StructField, StringType, IntegerType
def load_data():
data2 = [(24,"Rakesh","Govindula"),
(16,"Virat","Kohli")]
schema = StructType([ \
StructField("id",IntegerType(),True), \
StructField("firstname",StringType(),True), \
StructField("lastname",StringType(),True)
])
df = spark.createDataFrame(data=data2,schema=schema)
return df
df2=load_data()
df2.show()
df2.createOrReplaceTempView("dataframeview")
mssparkutils.notebook.exit("dataframeview")
Source Notebook:
Code:
value="test"
from notebookutils import mssparkutils
view_name=mssparkutils.notebook.run("/function_notebook", 60, {"param": value})
df=spark.sql("select * from {0}".format(view_name))
df.show()
With this approach you can pass the parameter through to function notebook and can access the dataframe returned from the function as well.
Please go through this SO Thread if you face any issues when returning values from synapse notebook.
In the Pygsheet reference doc here it shows a wrap_strategy property.
wrap_strategy
How to wrap text in this cell. Possible wrap strategies: ‘OVERFLOW_CELL’, ‘LEGACY_WRAP’, ‘CLIP’, ‘WRAP’. Reference: api docs
But in actual code if I were to cell.wrap_strategy = 'WRAP' I get an error TypeError: 'str' object is not callable
Actual code snippet:
for cell in wsheet.range("L3:L20").pop():
cell.wrap_strategy('WRAP')
I believe your goal as follows.
You want to set the wrap strategy using pygsheets.
Modification points:
When I saw the script of wrap_strategy of pygsheets, it seems that this is the Class Cell and in this case, I think that in the case of for cell in wsheet.range("L3:L20").pop():, you can use cell.wrap_strategy = 'WRAP'. (In this case, it sets to "L20".)
From this, how about modifying your script as follows?
Modified script:
for cell in wsheet.range("L3:L20").pop():
cell.wrap_strategy = "WRAP"
or, as other direction, how about using get_values as follows?
for cell in wsheet.get_values("L3", "L20", returnas="range")._data.pop():
cell.wrap_strategy = "WRAP"
Note:
If above modification was not the direct solution of your issue, can you provide your whole script without your personal information? By this, I would like to confirm it.
Reference:
wrap_strategy
Added:
From your following replying,
I was expecting it to wrap L3 to L20. But it seems only L20 is being read in this for loop. Would you know how to make it so?
When I saw your script in your question, you wanted to use the last element of wsheet.range("L3:L20") because of pop(). So I followed to it. From your replying, when you want to set the wap strategy for "L3:L20" by modifying your script, how about the following sample script?
Sample 1:
for cell in wsheet.range("L3:L20"):
cell[0].wrap_strategy = "WRAP"
Sample 2:
for cell in wsheet.get_values("L3", "L20", returnas="range")._data:
cell[0].wrap_strategy = "WRAP"
Like the question says, I want to know if/how you can set a Databricks widget input using a variable instead of a hard-coded value. I have two notebooks. One needs apply a filter to some values. The other needs to run some code, then optionally (as dictated by another widget) apply that same filter.
Here's some example code (modified for simplicity/privacy).
In Notebook2 we have:
start = dbutils.widgets.get("startDate")
filter_condition = None
if start:
filter_condition = f"GeneratedDate >= '{start}'"
foo = important_function(filter_condition)
%run ./Notebook1 $run_training="True" $num_trials=100 $filter_string=filter_condition
where I want filter_condition to be the above-defined variable and not a string.
In Notebook1, there's some code like:
if run_training=="True":
bar = optimize_model(datasets, grid, int(num_trials))
elif run_training=="False":
baz = apply_filter(filter_string)
else:
# Throw error
You're probably looking for the notebook.run function of the databricks utilities package, rather than the %run command:
dbutils.notebook.run(path='./Notebook1',
timeout_seconds=300,
arguments={'run_training':'True',
'num_trials':100,
'filter_string':filter_condition})
The notebook will be run as a "ephemeral" job. Note that the notebook will run in a separate notebook environment, so any variables etc created will not be brought back into the notebook you ran it from. Your input arguments come through as widget variables, which can be accessed using:
num_trails = dbutils.widgets.get('num_trails')
etc. I think you are already doing that though.
I am working on a mini GUI project , I am currently struggling to figure out how to get selected value from the list and then return that value to the main function so that I can use that value in somewhere else . Can someone help me please !!!!
####
self.device_list_store = gtk.ListStore(str,str,str,str,str)
for device in self.get_dev_list():
self.device_list_store.append(list(device))
device_list_treeview = gtk.TreeView(self.device_list_store)
selected_row = device_list_treeview.get_selection()
selected_row.connect("changed",self.item_selected)
####
def item_selected(self,selection):
model,row = selection.get_selected()
if row is not None:
selected_device = model[row][0]
at the moment ,the item_selected function is not returning anything , I want to return selected_device back to the main function so I can use it in other functions as well .
EDIT: I've edited code above to remove formatting errors #jcoppens
As you can see in the documentation, the item_selected function is called with one parameter, tree_selection. But if you define the function inside a class, it requires the self parameter too, which is normally added automatically. In your (confusing) example, there is no class defined, so I suspect the problem is your program which is incomplete.
Also, I suspect you don't want device_list_treeview = gtk.T... in the for loop:
for device in self.get_dev_list():
self.device_list_store.append(list(device))
device_list_treeview = gtk.TreeView(self.device_list_store)
And I suspect you want selected_device = mod... indented below the if:
if row is not None:
selected_device = model[row][0]
Please convert your example in a complete program, and formatted correctly.
BTW: item_selected is not a good name for the signal handler. It is also called if the item is unselected (which is why the signal is called 'changed')!
And important: Even though you should first read the basic Python tutorials and Gtk tutorials, you should then consider using lazka's excellent reference for all the Python APIs. There's a link on the page to download it completely and have it at hand in your computer.
My question is: How can I write an IPython cell magic which has access to the namespace of the IPython notebook?
IPython allows writing user-defined cell magics. My plan is creating a plotting function which can plot one or multiple arbitrary Python expressions (expressions based on Pandas Series objects), whereby each line in the cell string is a separate graph in the chart.
This is the code of the cell magic:
def p(line, cell):
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame()
line_list = cell.split('\n')
counter = 0
for line in line_list:
df['series' + str(counter)] = eval(line)
counter += 1
plt.figure(figsize = [20,6])
ax = plt.subplot(111)
df.plot(ax = ax)
def load_ipython_extension(ipython):
ipython.register_magic_function(p, 'cell')
The function receives the entire cell contents as a string. This string is then split by line breaks and evaluated using eval(). The result is added to a Pandas DataFrame. Finally the DataFrame is plotted using matplotlib.
Usage example: First define the Pandas Series object in IPython notebook.
import pandas as pd
ts = pd.Series([1,2,3])
Then call the magic in IPython notebook (whereby the whole code below is one cell):
%%p
ts * 3
ts + 1
This code fails with the following error:
NameError: name 'ts' is not defined
I suspect the problem is that the p function only receives ts * 3\n ts + 1 as a string and that it does not have access to the ts variable defined in the namespace of IPython notebook (because the p function is defined in a separate .py file).
How does my code have to be changed so the cell magic has access to the ts variable defined in the IPython notebook (and therefore does not fail with the NameError)?
Use the #needs_local_scope decorator decorator. Documentation is a bit missing, but you can see how it is used, and contributing to docs would be welcome.
You could also use shell.user_ns from Magics. For example something like:
from IPython.core.magic import Magics
class MyClass(Magics):
def myfunc(self):
print(self.shell.user_ns)
See how it's used in code examples: here and here.