I'm working on a project where I'm importing a notebook as a module into another module.
I use the example on the IPython GH and it works fine.
However, now I'm trying to add a feature for which I need to access the source code (inspect.getsourcelines()) of a function/class-method.
The feature works when I execute it inside one notebook.
However, as soon as I import the function as a module, inspect cannot read the sources anymore.
This is because the code is executed via exec(code, mod.__dict__) which, as expected and Python default, will not allow the sources to be read.
I searched for a while but didn't find a conclusive answer yet, so here my questions:
Why is in the example the source code executed with exec? (My suspicion: because the code is beforehand transformed)
EDIT: Why is run_cell not used in the example?
In general, what happens when I execute a Jupyter cell? Does the code get transformed there? EDIT: I assume it's run_cell, am I right?
Is it possible to modify the module-import-example to use the standard "execute-cell" command and allow using inspect to look at the sources?
So I still haven't found a fully acceptable solution.
For anybody interested: As a workaround I'm using the %run filename command which imports everything into the local notebook (and also prints all outputs that it executes [1])
The problem: everything is 'local' and not in a submodule. That means any variables defined in the imported notebook are accessible and overwrite existing ones. This can become a bit annoying. (But right now I'm at least happy that it works somewhat).
[1] %%capture blocks the output and makes it at least clean.
Related
I have a question regarding simple imports that I cannot get my head around.
Take a look at the attached screenshot to see my project layout.
The file somefile.py imports the class SayHello from a file called someclass.py and calls it. someotherfile.py does the exact same thing. They both use from someclass import SayHello.
In Pycharm both files run. However, From the command line or from VSCode somefile.py runs, but someotherfile.py errors out with the following error:
ModuleNotFound: No module named 'someclass'.
I believe it has something to do with PYTHONPATH/environment variables or something like that, but every explanation I have read has confused me thus far (Even this one which I thought was going to set me strait Relative imports for the billionth time).
Can someone explain in simple terms what is happening here? What is Pycharm doing by default that other editors are not such that my imported modules are found? How can I make someotherfile.py work in VSCode without modifying it?
Thanks in advance!
Pycharm adds your project directory into python paths by default. See configuration of "Pycharm run" that you execute and you shall see few checkboxes like
If those checked Pycharm creates PYTHONPATH environment variable for you that instructs Python where to look for someclass module.
You will have to configure VSCode to define PYTHONPATH environemnt variable for python command you run and include your root project directory path on it.
TLDR: Mess with the line starting with ./automated pointing it to various directories in your project until it works haha.
Long rambling answer: Alright now that I am not in a frenzy from trying to figure this out and it has been a day, I feel like I can make a conherint response (lets see if that is true).
So my original answer was an attempt to simplify my problem into what I thought was the issue due to a ModuleNotFound error I was getting. I had been trying to get unittests in Python inside of Visual Studio code to work (hint hint just use Pycharm it just works out of the box in there), but the integrated testing feature or whatever could not find my tests giving ModuleNotFound as the reason.
The solution to my problem actually just concerned the line ./automated-software-testsing-with-python/blog.
In the below screenshot the relevant part is ./automated-software-testing-with-python/blog.
This is it when it is correctly configured (My tests work Woo hoo!)
Now you can go ahead and read the official documentation for what this line is doing, but I am convinced that the documentation is wrong. Due to how my code is structured and how my files are named, what the documentation says its looking for definitely does not exist. But that is another can of worms...
I actually got it to work by accident. Originally when you go though the wizard to set up what's in that screenshot above, it defaulted to ./automated-software-testing-with-python which did not work. I then manually edited it to something else that was not the below screenshot hoping to get it to work.
It was only when I pointed it to that blog directory on accident (thinking I was in a different file due to trying to debug this for hours and hours in a blind rage) that it ended up working.
I did a bunch of PYTHONPATH manipulation and Environment Variable mumbo jumbo, and I originally thought that that had an effect, but when I cloned my repot to my other machine (which did not have any of that Environment Variable PYTHONPATH stuff going on) it worked too (again, provided that the line in question pointed to blog.
Ok hopefully that might help somone someday, so I'll end it there. Not to end on a bitter sounding zinger, but I will never cease be amazed by how doing such simple things such as configuring the most basic unit test can be so difficult in our profession haha. Well now I can start working. Thanks for the help to those who answered!
As many others, I am running into the issue of PyCharm not using the newer version of a function in a different file when the function is imported into a .ipynb. The behaviour is as follows: I run the ipynb, importing the function. I change the function in a seperate .py file. I rerun the ipynb and the unchanged version of the function is still in use. I tried to fix this with the proposed fix found here: https://stackoverflow.com/a/60099037/5587736. However, this doesn't change anything. I need help running through this, because this fix seems to work for others.
Suppose I am working on some python code say in foo.py.
Is it reasonable to have a notebook named foo.ipynb and import foo in that notebook?
This seems to work, but is it a good idea?
Thanks
Yes, technically you could that do with no issue, as long as yourself is comfortable with having those two files using the same name and different extension.
However, I would like to mention that at some point of time, people might want or need to change a notebook file to a single python script file. When that happens, it might be confusing. Thus, personally, if I have a foo.py, I would name my notebook as foo_notebook.ipynb, even if it seems reductant.
Is it possible to change the code while debugging in VSCode and that the change will take effect immediately without rerun the code?
I'm using Microsoft Python extension.
It depends. There's no such thing as a hot reload in Python. The closest you can come is importlib.reload(), but realize that only reloads the module and not the objects which already exist in memory. IOW it typically doesn't do what you want in code (it's usually used in the REPL).
So, apparently that the closest way of doing it is to use Jupyter capabilities (that can be use via VSCode).
With Jupyter you can split your code to multiple cells and run every each one of them separately without run all from beginning.
This is a general question so codes not included here.
Essentially, I wrote python functions that I need to use in sublime and make sure I have saved after every modification. A jupyter notebook is running along side while I am making changes to the python file.
The problem is, as I intend to do debugging within the jupyter notebook, even though I ran the import after every modification of the python file, the effect of the modification does not show up in the jupyter notebook.
Can anyone tell me why is this? Do I have to shut down the localhost everytime to import the most recent version of the python file? Is there a way to avoid this?
Thanks a lot!
If a module has already been imported, importing it again won't reread the file from disk. You would have to use reload(module). (See more information about similar issues here.) It's not totally clear from your statement if that's the source of your problem but it's a reasonable guess.