Develop Reference

Import function from a cython python module not working

I have made a module:
However I got this error when I import the count_kmers function:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'kmer_cython' has no attribute 'count_kmers'
I import my stuff like this:
from Seq_utils.fasta_parser import parse_fasta
from kmer_cython import count_kmers
from Alphabet.alphabet import iupac_dna
My function was defined like this:
It is a simple function that worked fine when I had it spread in the middle of other functions in my directory. Then I decide to organize it and now I wish to run it, but I can't.
I have set the PYTHONPATH in my terminal and other codes that I made a module are working.
Any tip to solve this problem?
Thank you all for your time.
Paulo

python find global variables of specific file without importing it

I'm making a module file, and I need to get the global variables of the file that is using it.
I've tried importing it, and then getting the global variables of of that, but that gets me this error:
Traceback (most recent call last):
File "C:\Users\isaac\OneDrive\Coding\python\logger\file.py", line 1, in <module>
import logger
File "C:\Users\isaac\OneDrive\Coding\python\logger\logger.py", line 3, in <module>
import file as bla
File "C:\Users\isaac\OneDrive\Coding\python\logger\file.py", line 3, in <module>
log = logger.Logger('logfile', 'file')
AttributeError: partially initialized module 'logger' has no attribute 'Logger' (most likely due to a circular import)
Is there a way to get the global variables with open(...) or something like that?

There are two solutions to this. One is to take the global variables out of file and move them to a third file, say globals.py which can be imported by both.
In terms of accessing global variables by opening the file, you can try and parse the file yourself, or indeed use something like AST. I imagine though that the aforementioned technique will be adequate for your use case.

Changing sys.modules caused an unexpected KeyError

Some automated tests in a larger system need to be able to import a module, and then restore sys.modules to its original condition.
But this code fragment:
import sys
sys.modules = dict(sys.modules)
import pickle
causes this KeyError in Python 3.6-3.8:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "[...]/python3.6/pickle.py", line 1562, in <module>
from _pickle import (
KeyError: '_compat_pickle'
It seems as if only pickle and modules that depend on it like multiprocessing are affected. I've investigated _compat_pickle - it's a module for pickling compatibility with Python 2 - but nothing jumps out that would cause this.
Is there a safe way to restore sys.modules back to an earlier state? And what is the mechanism behind this unexpected KeyError?

The problem is that sys.modules is a lie (I think). It is not actually the true source of the modules dict. That is stored on a C level in the current interpreter, and sys.modules is just a copy to that. _pickle is special, since it imports a module from C source, which I assume leads to this error (mismatch between what tstate->interp->modules says is imported and what sys.modules thinks is imported).
This might be considered a bug in python. I am not sure if a bug report already exists. Here is the bug report: https://bugs.python.org/issue12633 .
You could just save which keys are in modules before and after the code, and delete all other entries afterwards.

Python circular reference importing doesn't work when using "as"

Let me first establish what working scenario.
main.py
module/file1.py
module/file2.py
main.py
import module.file1
print(module.file1)
module/file1.py
import module.file2
module/file2.py
import module.file1
Running python3 main.py gives me the following, which is fine.
<module 'module.file1' from '/project/module/file1.py'>
Now, if I change module/file2.py to have the following:
import module.file1 as testtt
I get this new output (error):
Traceback (most recent call last):
File "main.py", line 1, in <module>
import module.file1
File "/project/module/file1.py", line 1, in <module>
import module.file2
File "/project/module/file2.py", line 2, in <module>
import module.file1 as testtt
AttributeError: module 'module' has no attribute 'file2'
I'm guessing that python doesn't fully evaluate the imported module when simply importing, causing the circular reference to blow up only when you immediately use it within either of the two files.
I'd imagine I also would not get the error if I used the module in a function, since that would be evaluate when the function is actually called, like this:
import module.file1
def test():
print(module.file1)
What is the recommendation here? Should I just work to remove the circular reference? It seems like code smell anyway (existing code base).

Its an implementation detail. The import statement uses the __import__ function to do the work of finding and importing the module and then binds its returned module to the as testtt variable.
When doing a nested import like import module.file1 as testtt, __import__ returns the base module ("module"). Since the importer still needs to bind "file1" to the local namespace, it has to look up the submodule name "file1" on that object. Since the import of file1 is still in progress, it hasn't been bound to the "module" module yet.
It works in the import module.file1 case because file1 isn't bound to the local namespace and doesn't need a lookup.
There are many pitfalls with circular imports that will bedevil you throughout your code's life cycle. Good luck!

"import" is an executable statement, so you can just do the import inside the function
def test():
import module.file1
print(module.file1)

Python. Import a function and then modified it. Later returns error in the same python session

I have two python scripts, one has all functions I have defined (functions.py) and the other only runs those functions (running_functions.py).
I imported the functions into running_functions script using from functions import*
My problem is when I ran running_functions into python console using execfile('running_functions.py') at first worked like a charm, but if I don't close the python session and do some modifications into one function in functions.py (for example changing the number of parameters that getLabels() takes (from 4 to 5)) saved then and then I ran again running_functions.py with the same comand or when I called getLabels() I get the error:
With execfile()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "running_functions.py", line 82, in <module>
predict_labels = getLabels(pred_labels, ids_tr ,labels_tr,filenames_tr, filenames_ts)
TypeError: getLabels() takes exactly 4 arguments (5 given)
Calling the function
>>> predict_labels = getLabels(pred_labels, ids_tr ,labels_tr,filenames_tr, filenames_ts)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: getLabels() takes exactly 4 arguments (5 given)
To get it work again I have to close python session and then run again execfile() or rename functions.py or do little pythons scripts with modified function.
This is very annoying because all the code takes around 10 or 15 minutes and I don't like have a lot of little scripts. So, how can I avoid this error?
I wouldn't like to close every time the session and wouldn't like to use in each function pickle module. Is it wrong the way I imported the functions? And why python returns this error? Sorry for this silly questions

I would recommend skimming over how python imports work. In general it's considered bad practice to use glob imports like from module import *. It's not transparent and makes it difficult to take advantage of reload.
I would recommend rewriting your code to do the following :
import functions
functions.getLabels(...)
and then after you change getLabels or something, you can from the shell run the following :
reload(functions)
and that will re-import your changes without having to restart the python kernel.