Develop Reference

Getting weird error on running python in cmd then continue normally

When I run python on cmd I get this weird error:
Failed calling sys.__interactivehook__
Traceback (most recent call last):
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site.py", line 446, in register_readline
import readline
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site-packages\readline.py", line 34, in <module>
rl = Readline()
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site-packages\pyreadline\rlmain.py", line 422, in __init__
BaseReadline.__init__(self)
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site-packages\pyreadline\rlmain.py", line 62, in __init__
mode.init_editing_mode(None)
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site-packages\pyreadline\modes\emacs.py", line 633, in init_editing_mode
self._bind_key('space', self.self_insert)
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site-packages\pyreadline\modes\basemode.py", line 162, in _bind_key
if not callable(func):
File "C:\Users\Khaled\AppData\Local\Programs\Python\Python310\lib\site-packages\pyreadline\py3k_compat.py", line 8, in callable
return isinstance(x, collections.Callable)
AttributeError: module 'collections' has no attribute 'Callable'
But then I continue using python normally:
>>>
I did not find any effects of this problem python is running normally but I do not want it to appear and to know what does it do.

assert group is none

I am trying to run a preprocessing pipeline using nipype and I get the following error message:
Traceback (most recent call last):
File "preprocscript.py", line 211, in <module>
preproc.run('MultiProc', plugin_args={'n_procs': 8})
File "/sw/anaconda/3/lib/python3.6/site-packages/nipype/pipeline/engine/workflows.py", line 579, in run
runner = plugin_mod(plugin_args=plugin_args)
File "/sw/anaconda/3/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py", line 162, in __init__
initargs=(self._cwd,)
File "/sw/anaconda/3/lib/python3.6/multiprocessing/pool.py", line 175, in __init__
self._repopulate_pool()
File "/sw/anaconda/3/lib/python3.6/multiprocessing/pool.py", line 236, in _repopulate_pool
self._wrap_exception)
File "/sw/anaconda/3/lib/python3.6/multiprocessing/pool.py", line 250, in _repopulate_pool_static
wrap_exception)
File "/sw/anaconda/3/lib/python3.6/multiprocessing/process.py", line 73, in __init__
assert group is None, 'group argument must be None for now'
AssertionError: group argument must be None for now
and I am not sure what exactly might be wrong in my code that leads to this or if this is an issue with my software.
I am on a linux system and use python 3.6.

The module you are using has a ProcessPoolExecuter being used in it. In Python 3.7 they added some additional arguments to that class, namely initargs which is what is being called in nipype multiprocess module you are using. Unfortunately it is not backwards compatible to 3.6 and they did not write in another way to use that module.
Your options are to upgrade or not use the multiprocessing portion of nipype.

CloudDataflow can not use "google.cloud.datastore" package?

I want to put datastore with transaction on CloudDataflow.
So, I wrote below.
def exe_dataflow():
....
from google.cloud import datastore
# call from pipeline
def ds_test(content):
datastore_client = datastore.Client()
kind = 'test_out'
name = 'change'
task_key = datastore_client.key(kind, name)
for _ in range(3):
with datastore_client.transaction():
current_value = client.get(task_key)
current_value['v'] += content['v']
datastore_client.put(task)
# pipeline
....
| 'datastore test' >> beam.Map(ds_test)
But, Error occured and log message was displayed as below.
(7b75e0ef2db229da): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 582, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 167, in execute
op.start()
...(SNIP)...
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 767, in _import_module
return getattr(__import__(module, None, None, [obj]), obj)
AttributeError: 'module' object has no attribute 'datastore'
CloudDataflow can not use "google.cloud.datastore" package?
add 2018/2/28.
I add --requirements_file to MyOption
options = MyOptions(flags = ["--requirements_file", "./requirements.txt"])
and I make requirements.txt
google-cloud-datastore==1.5.0
But, Another error occurred.
(366397598dcf7f02): Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 582, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 167, in execute
op.start()
...(SNIP)...
File "my_dataflow.py", line 66, in to_entity
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore/__init__.py", line 60, in <module>
from google.cloud.datastore.batch import Batch
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore/batch.py", line 24, in <module>
from google.cloud.datastore import helpers
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore/helpers.py", line 29, in <module>
from google.cloud.datastore_v1.proto import datastore_pb2
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore_v1/__init__.py", line 17, in <module>
from google.cloud.datastore_v1 import types
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore_v1/types.py", line 21, in <module>
from google.cloud.datastore_v1.proto import datastore_pb2
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore_v1/proto/datastore_pb2.py", line 17, in <module>
from google.cloud.datastore_v1.proto import entity_pb2 as google_dot_cloud_dot_datastore__v1_dot_proto_dot_entity__pb2
File "/usr/local/lib/python2.7/dist-packages/google/cloud/datastore_v1/proto/entity_pb2.py", line 28, in <module>
dependencies=[google_dot_api_dot_annotations__pb2.DESCRIPTOR,google_dot_protobuf_dot_struct__pb2.DESCRIPTOR,google_dot_protobuf_dot_timestamp__pb2.DESCRIPTOR,google_dot_type_dot_latlng__pb2.DESCRIPTOR,])
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/descriptor.py", line 824, in __new__
return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "google/cloud/datastore_v1/proto/entity.proto":
google.datastore.v1.PartitionId.project_id: "google.datastore.v1.PartitionId.project_id" is already defined in file "google/cloud/proto/datastore/v1/entity.proto".
...(SNIP)...
google.datastore.v1.Entity.properties: "google.datastore.v1.Entity.PropertiesEntry" seems to be defined in "google/cloud/proto/datastore/v1/entity.proto", which is not imported by "google/cloud/datastore_v1/proto/entity.proto". To use it here, please add the necessary import.

The recommended way to interact with Cloud Datastore from a Cloud Dataflow Pipeline is to use the Datastore I/O API, which is available through the Dataflow SDK and provides some methods to read and write data to a Cloud Datastore database.
You can find detailed documentation for the Datastore I/O package for Dataflow SDK 2.x for Python in this other link. The datastore.v1.datastoreio module is the specific module that you want to use. There is plenty of information in the links I am sharing, but in short, it is a connector to Datastore that uses PTransform to read / write / delete a PCollection from Datastore using the classes ReadFromDatastore() / WriteToDatastore() / DeleteFromDatastore() respectively.
You should try using it instead of implementing the calls yourself. I suspect this may be the reason for the error you are seeing, as a Datastore implementation already exists in the Dataflow SDK:
"google.datastore.v1.PartitionId.project_id" is already defined in file "google/cloud/proto/datastore/v1/entity.proto".
UPDATE:
It looks like those three classes collect several mutations and executes them in a single transaction. You can check that in the code describing the classes.
If the aim is to retrieve (get()) and then update (put()) a Datastore entity, you can probably work with the write_mutations() function, which is described in the documentation, and you can work with a full batch of mutations performing the operations you are interested in.

How to setup MongoDB with Django

I've followed numerous tutorials on how to setup a Django project with MongoDB but still get the same errors when running the command for testing:
python manage.py runsever
Here is the link I've been following:
http://www.allbuttonspressed.com/blog/django/2010/05/MongoDB-backend-for-Django-nonrel-released
Here are the errors I get from the above command:
Validating models...
Unhandled exception in thread started by <bound method Command.inner_run of <django.contrib.staticfiles.management.commands.runserver.Command object at 0x10d266210>>
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/django/core/management/commands/runserver.py", line 91, in inner_run
self.validate(display_num_errors=True)
File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 266, in validate
num_errors = get_validation_errors(s, app)
File "/Library/Python/2.7/site-packages/django/core/management/validation.py", line 23, in get_validation_errors
from django.db import models, connection
File "/Library/Python/2.7/site-packages/django/db/__init__.py", line 40, in <module>
backend = load_backend(connection.settings_dict['ENGINE'])
File "/Library/Python/2.7/site-packages/django/db/__init__.py", line 34, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/Library/Python/2.7/site-packages/django/db/utils.py", line 92, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Library/Python/2.7/site-packages/django/db/utils.py", line 24, in load_backend
return import_module('.base', backend_name)
File "/Library/Python/2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "build/bdist.macosx-10.8-intel/egg/django_mongodb_engine/__init__.py", line 13, in <module>
AttributeError: 'tuple' object has no attribute 'insert'
I have no idea why this isn't working and ideally I just want a database setup to use specifically for storing JSON documents. Is there an easier alternative to connecting Django with another database that is ideal for storing JSON information?

For clarification:
INSTALLED_APPS should be tuple.
He gets AttributeError: 'tuple' object has no attribute 'insert' because mongodb does not handle tuple INSTALLED_APPS correctly. The default in mongodb is insert list. This is already fix before but now it's gone.
There suggestion are:
changing your settings.INSTALLED_APPS to a list instead of a tuple. (which is bad for me)
correct their code by adding the patch

There is a new package called djongo for fully nosql support with mongoDB:
- https://nesdis.github.io/djongo/
you cann install ist in your venv:
pip install djongo

help('modules') crashing? Not sure how to fix

I was trying to install a module for opencv and added an opencv.pth file to the folder beyond my sites.py file. I have since deleted it and no change.
When I try to run help('modules'), I get the following error:
Please wait a moment while I gather a
list of all available modules...
/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/words/im/init.py:8:
UserWarning: twisted.im will be
undergoing a rewrite at some point in
the future.
warnings.warn("twisted.im will be
undergoing a rewrite at some point in
the future.")
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pkgutil.py:110:
DeprecationWarning: The wxPython
compatibility package is no longer
automatically generated or actively
maintained. Please switch to the wx
package as soon as possible.
import(name) Traceback (most recent call last): File "",
line 1, in File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site.py",
line 348, in call
return pydoc.help(*args, **kwds) File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pydoc.py",
line 1644, in call
self.help(request) File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pydoc.py",
line 1681, in help
elif request == 'modules': self.listmodules() File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pydoc.py",
line 1802, in listmodules
ModuleScanner().run(callback) File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pydoc.py",
line 1853, in run
for importer, modname, ispkg in pkgutil.walk_packages(): File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pkgutil.py",
line 110, in walk_packages
import(name) File "/BinaryCache/wxWidgets/wxWidgets-11~262/Root/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/wxaddons/init.py",
line 180, in import_hook File
"/Library/Python/2.5/site-packages/ctypes_opencv/init.py",
line 19, in
from ctypes_opencv.cv import * File
"/BinaryCache/wxWidgets/wxWidgets-11~262/Root/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/wxaddons/init.py",
line 180, in import_hook File
"/Library/Python/2.5/site-packages/ctypes_opencv/cv.py",
line 2567, in ('desc', CvMat_r, 1), # CvMat* desc File
"/Library/Python/2.5/site-packages/ctypes_opencv/cxcore.py",
line 114, in cfunc
return CFUNCTYPE(result, *atypes)((name, dll), tuple(aflags)) AttributeError: dlsym(0x2674d10, cvCreateFeatureTree): symbol not found
What gives?!

This happens because help('modules') imports all modules, which can result in a lot of unsentineled code being executed. There's nothing you can do short of reporting bugs in every single package that causes this (opencv in this case) and wait for them to fix it.