I'm a rookie using Dask and I installed the new version 2.12.0 on my MacBook MacOS High Sierra 10.13.6. When I try to start the distributed mode with the code below:
from dask.distributed import Client
c = Client()
I got the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-1-67101dbde0b6> in <module>()
1 from dask.distributed import Client
----> 2 c = Client()
~/anaconda3/lib/python3.6/site-packages/distributed/client.py in __init__(self, address, loop, timeout, set_as_default, scheduler_file, security, asynchronous, name, heartbeat_interval, **kwargs)
554 if set_as_default:
555 self._previous_get = _globals.get('get')
--> 556 dask.set_options(get=self.get)
557 self._previous_shuffle = _globals.get('shuffle')
558 dask.set_options(shuffle='tasks')
AttributeError: module 'dask' has no attribute 'set_options'
Also, if I instantiate the c = Client(set_as_default=False) the cluster seems to successfully raise because I get the connection information for the dashboard (see this image). But as I navigate through the dashboard the following error is triggered.
According to the documentation lifting a single machine cluster seems to be trivial but I don't know what could be wrong. I would be very grateful if someone could guide me on how to solve this incident ;)
Your source file is named dask.py, which introduces a naming conflict between your module, dask.py, and the dask package, with your module taking precedence in later import statements. Therefore, when distributed/client.py attempts to call dask.set_options, this fails with AttributeError as your module does not provide this function.
After running this dask.py module, python caches the compiled python code. Therefore, even when you renamed your module to something else, the import name conflict still exists. To resolve this, you should delete the __pycache__ directory after which you should find that your module executes successfully.
Related
I am running a Windows 10 machine with python 3.11 installed, and am facing an unexpected error.
The issue arose when I was attempting to lock my mouse, I've been tackling this issue for a while now, and I've finally found a good lead.
I tried this code:
from Xlib.display import Display
from Xlib import X
import contextlib
from time import sleep
DISPLAY=":0"
disp = Display(DISPLAY)
screen = disp.screen()
root = screen.root
#contextlib.contextmanager
root.grab_pointer(owner_events=True,
event_mask=0,
time=X.CurrentTime,
pointer_mode=X.GrabModeAsync,
keyboard_mode=X.GrabModeAsync,
confine_to=0,
cursor=0)
sleep(5)
disp.ungrab_pointer(time=X.CurrentTime)
It seems to be Display(DISPLAY) causing the issue, the whole error being:
Exception has occurred: AttributeError
module 'socket' has no attribute 'AF_UNIX'
File "G:\Downloads\PY SCRIPTS\mouselock.py", line 8, in <module>
disp = Display(DISPLAY)
Leaving Display empty ( as "Display()" ) will not work, though I have seen it used in others' scripts- it does not work in my case. This has led me to believe this is an issue with me operating on Windows 10, and/or python 3.11.
Anything I can do to resolve this issue?
Any help is appreciated.
Some automated tests in a larger system need to be able to import a module, and then restore sys.modules to its original condition.
But this code fragment:
import sys
sys.modules = dict(sys.modules)
import pickle
causes this KeyError in Python 3.6-3.8:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "[...]/python3.6/pickle.py", line 1562, in <module>
from _pickle import (
KeyError: '_compat_pickle'
It seems as if only pickle and modules that depend on it like multiprocessing are affected. I've investigated _compat_pickle - it's a module for pickling compatibility with Python 2 - but nothing jumps out that would cause this.
Is there a safe way to restore sys.modules back to an earlier state? And what is the mechanism behind this unexpected KeyError?
The problem is that sys.modules is a lie (I think). It is not actually the true source of the modules dict. That is stored on a C level in the current interpreter, and sys.modules is just a copy to that. _pickle is special, since it imports a module from C source, which I assume leads to this error (mismatch between what tstate->interp->modules says is imported and what sys.modules thinks is imported).
This might be considered a bug in python. I am not sure if a bug report already exists. Here is the bug report: https://bugs.python.org/issue12633 .
You could just save which keys are in modules before and after the code, and delete all other entries afterwards.
I am trying to create my own module (mi_modulo.py) and move there all the functions that I have defined in my Jupyter Notebook script, so that it looks cleaner.
However, when I am trying to use these functions that I have already moved to the module, I am not able to use them all, and I get the following message:
module 'mi_modulo' has no attribute 'train4_data_import'
I have installed Anaconda 3.0 and I am running Python 3.7.0 through Jupyter Notebooks. (Forgive me if the expressions sound awkward, I know a bit of Python, but I am not really into all the installation, software, IDE, etc details.)
## mi_modulo.py ##
def train4_data_import(file_name):
df = pandas.read_excel(file_name)
force = df["Signal 1"].values[13:]
acceleration1 = df["Signal 2"].values[13:]
acceleration2 = df["Signal 3"].values[13:]
return force, acceleration1, acceleration2
def hola_mundo():
print("whatever")
## script ##
import pandas
import mi_modulo as mi
mi.hola_mundo()
mi.train4_data_import("Tren4.xlsx")
And this is what I get:
(I was going to show an image but I am not sure how to do that with this stackoverflow new form style)
whatever
AttributeError Traceback (most recent call last)
<ipython-input-18-69a38929f7e6> in <module>()
3 mi.hola_mundo()
4
----> 5 mi.train4_data_import()
AttributeError: module 'mi_modulo' has no attribute 'train4_data_import'
I don't understand why it is able to read one function but not the other.
----------------------------- EDIT 1 ----------------------------
Doing what U9-Forward suggests:
import pandas
from mi_modulo import *
hola_mundo()
train4_data_import("Tren4.xlsx")
I get now the following error:
whatever
NameError Traceback (most recent call last)
<ipython-input-25-e1885200beb7> in <module>()
3 hola_mundo()
4
----> 5 train4_data_import("Tren4.xlsx")
NameError: name 'train4_data_import' is not defined
In jupyter-notebook, sometimes you need to restart the kernel to import all the unsaved module you have. Also, you need to import all the dependency for the custom module within that module.
It is probably because you didn't press Ctrl+S or hit the save button on the file, it will probably work if you do that:
Ctrl+S
Or save button.
then run script.py and see it working :-)
I don't understand why python (at least 2.7) is not validating the exception handling mechanism for errors.
Example:
try:
some code selecting data from pymongo
except pymongo.errors.OperationFailure:
exception
In this case, if the exception is not called for the first time, python will not validate if I actually did import the pymongo lib.
Any idea why?
If I'm reading your question right, you want to know why except pymongo.errors.OperationFailure doesn't cause an error when the module is loaded if you haven't already imported the pymongo module.
Like most things in Python, the arguments to except clauses are evaluated at runtime. In fact, they can be expressions! Python does not validate them at "compile" time any more than it validates any other names at that time.
The reason is that Python is a dynamic language. Imports can be done conditionally, or performed based on names that are not known at "compile" time, and modules and other namespaces can be replaced, modified, or removed by code. As a result, Python literally cannot know whether pymongo.errors.OperationFailure is a valid name at that point in your code's execution without running your code.
According to PyMongo documentation, exception pymongo.errors.OperationFailure will be "raised when a database operation fails". AS such, your exceptblock gets evaluated only when such an error is raised.
I'm assuming that by "validation of pymongo's existence" you are referring to somethine like:
try:
import pymongo
except:
print("PyMongo not found!")
sys.exit(-1)
This method is often used to provide fallbacks (and backwards compatibity) not to "validate" imports. For instance in the case of json encoder/decoder, we can try whether we have simplejson library available and use jsonlibrary as a fallback as follows:
try:
import simplejson as json
except ImportError:
import json
Assuming that in the beginning of your script, you already have import pymongo, I don't see a reason why you should be checking or "validating" that pymongo has been imported: import pymongo will already raise an ImportError if pymongo library is not found.
First of all, pymongo.errors.OperationFailure may be defined anywhere, not only as a part of pymongo module, but also as a property of property of pymongo object defined in the same file.
Thus when handling exceptions Python should not check if specific module has been imported.
But if you do something like that:
import pymongo
you will see that import error is actually raised if module is not found:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import pymongo
ImportError: No module named pymongo
If my answer is not enough for you and you want to know more about why inclusion of pymongo.errors.OperationFailure does not throw any error when you run your script for the first time, even though you do not have any import pymongo statement in your code, then please see kindall's answer on Python being a dynamic language.
I just started experimenting with a new technique I name (for the moment at least) "module duck typing".
Example:
Main Module
import somepackage.req ## module required by all others
import abc
import Xyz
Module abc
__all__=[]
def getBus():
""" Locates the `req` for this application """
for mod_name in sys.modules:
if mod_name.find("req") > 0:
return sys.modules[mod_name].__dict__["Bus"]
raise RuntimeError("cannot find `req` module")
Bus=getBus()
In module abc I do not need to explicitly import req: it could be anywhere in the package hierarchy. Of course this requires some discipline...
With this technique, it is easy to relocate packages within the hierarchy.
Are there pitfalls awaiting me? e.g. moving to Python 3K
Updated: after some more testing, I decided to go back to inserting package dependencies directly in sys.path.
There might be all kinds of modules imported that contain "req" and you don't know if it's the module you are actually looking for:
>>> import urllib.request
>>> import tst
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tst.py", line 12, in <module>
Bus=getBus()
File "tst.py", line 9, in getBus
return sys.modules[mod_name].__dict__["Bus"]
KeyError: 'Bus'
The whole point of packages is that there are namespaces for module hierarchies. Looking up module names "from any package" just causes your code to break randomly if the user happens to import some library that happens to contain a module with a conflicting name.
This technique is dangerous and error prone. It could work with your tests until the day that someone imports a new something.req and gets a confusing, far-off error. (This is in the best case scenario; the current implementation would jump on many other modules.) If you restructure packages, it's easy enough to at that time modify your code in an automated fashion without any use of magic. Python makes it possible to do all sorts of magical, dynamic things, but that doesn't mean we should.
I think this is more like duck typing. I would also recommend using a more unique identifier than "Bus"
def getBus():
""" Locates the Bus for this application """
for mod in sys.modules.values():
if hasattr(mod, 'Bus') and type(mod.Bus) is...: # check other stuff about mod.Bus
return mod.Bus
raise RuntimeError("cannot find Bus")