Develop Reference

python pathlib resolve does not support Chinese character on windows

On my windows,
pathlib.Path('R:/人')
has no problem to give
WindowsPath('R:/人')
However,
pathlib.Path('R:/人').resolve()
gives
--------------------------------------------------------------------------- OSError Traceback (most recent call
last) in
----> 1 pathlib.Path('R:/人').resolve()
~\anaconda3\lib\pathlib.py in resolve(self, strict) 1178 if
self._closed: 1179 self._raise_closed()
-> 1180 s = self._flavour.resolve(self, strict=strict) 1181 if s is None: 1182 # No symlink resolution => for
consistency, raise an error if
~\anaconda3\lib\pathlib.py in resolve(self, path, strict)
203 while True:
204 try:
--> 205 s = self._ext_to_normal(_getfinalpathname(s))
206 except FileNotFoundError:
207 previous_s = s
OSError: [WinError 1] 函数不正确。: 'R:\人'
What is the problem with resolve in pathlib?
On the other hand,
pathlib.Path('R:/人').absolute()
correctly gives
WindowsPath('R:/人')

Word2Vec error when loading in GoogleNews data

I am following a tutorial here: https://towardsdatascience.com/multi-class-text-classification-model-comparison-and-selection-5eb066197568
I am at the part "Word2vec and Logistic Regression". I have downloaded the "GoogleNews-vectors-negative300.bin.gz" file and I am tyring to apply it to my own text data. However when I get to the following code:
%%time
from gensim.models import Word2Vec
wv = gensim.models.KeyedVectors.load_word2vec_format("/data/users/USERS/File_path/classifier/GoogleNews_Embedding/GoogleNews-vectors-negative300.bin.gz", binary=True)
wv.init_sims(replace=True)
I run into the following error:
/data/users/msmith/env/lib64/python3.6/site-packages/smart_open/smart_open_lib.py:398: UserWarning: This function is deprecated, use smart_open.open instead. See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function
'See the migration notes for details: %s' % _MIGRATION_NOTES_URL
---------------------------------------------------------------------------
EOFError Traceback (most recent call last)
<timed exec> in <module>
~/env/lib64/python3.6/site-packages/gensim/models/keyedvectors.py in load_word2vec_format(cls, fname, fvocab, binary, encoding, unicode_errors, limit, datatype)
1492 return _load_word2vec_format(
1493 cls, fname, fvocab=fvocab, binary=binary, encoding=encoding, unicode_errors=unicode_errors,
-> 1494 limit=limit, datatype=datatype)
1495
1496 def get_keras_embedding(self, train_embeddings=False):
~/env/lib64/python3.6/site-packages/gensim/models/utils_any2vec.py in _load_word2vec_format(cls, fname, fvocab, binary, encoding, unicode_errors, limit, datatype)
383 with utils.ignore_deprecation_warning():
384 # TODO use frombuffer or something similar
--> 385 weights = fromstring(fin.read(binary_len), dtype=REAL).astype(datatype)
386 add_word(word, weights)
387 else:
/usr/lib64/python3.6/gzip.py in read(self, size)
274 import errno
275 raise OSError(errno.EBADF, "read() on write-only GzipFile object")
--> 276 return self._buffer.read(size)
277
278 def read1(self, size=-1):
/usr/lib64/python3.6/_compression.py in readinto(self, b)
66 def readinto(self, b):
67 with memoryview(b) as view, view.cast("B") as byte_view:
---> 68 data = self.read(len(byte_view))
69 byte_view[:len(data)] = data
70 return len(data)
/usr/lib64/python3.6/gzip.py in read(self, size)
480 break
481 if buf == b"":
--> 482 raise EOFError("Compressed file ended before the "
483 "end-of-stream marker was reached")
484
EOFError: Compressed file ended before the end-of-stream marker was reached
Any idea whats gone wrong/ how to overcome this issue?
Thanks in advance!

Google AdManager getCurrentNetwork() error

I have set up ad manager credentials. I'm trying to access the Admanager API, im getting the following error.
from googleads import ad_manager
client = ad_manager.AdManagerClient.LoadFromStorage()
network_service = client.GetService('NetworkService', version='v201902')
current_network = network_service.getCurrentNetwork()
the error im facing is:
Fault Traceback (most recent call last)
~\Anaconda3\lib\site-packages\googleads\common.py in MakeSoapRequest(*args)
1381 return soap_service_method(
-> 1382 *packed_args, _soapheaders=soap_headers)['body']['rval']
1383 except zeep.exceptions.Fault as e:
~\Anaconda3\lib\site-packages\zeep\proxy.py in __call__(self, *args, **kwargs)
41 self._proxy._client, self._proxy._binding_options,
---> 42 self._op_name, args, kwargs)
43
~\Anaconda3\lib\site-packages\zeep\wsdl\bindings\soap.py in send(self, client, options, operation, args, kwargs)
131
--> 132 return self.process_reply(client, operation_obj, response)
133
~\Anaconda3\lib\site-packages\zeep\wsdl\bindings\soap.py in process_reply(self, client, operation, response)
193 if response.status_code != 200 or fault_node is not None:
--> 194 return self.process_error(doc, operation)
195
~\Anaconda3\lib\site-packages\zeep\wsdl\bindings\soap.py in process_error(self, doc, operation)
287 actor=None,
--> 288 detail=etree_to_string(doc))
289
Fault: Unknown fault occured
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-16-991c0839fc99> in <module>()
----> 1 current_network = network_service.getCurrentNetwork()
~\Anaconda3\lib\site-packages\googleads\common.py in MakeSoapRequest(*args)
1385 if e.detail is not None:
1386 underlying_exception = e.detail.find(
-> 1387 '{%s}ApiExceptionFault' % self._GetBindingNamespace())
1388 fault_type = self.zeep_client.get_element(
1389 '{%s}ApiExceptionFault' % self._GetBindingNamespace())
TypeError: a bytes-like object is required, not 'str'

You are probably missing permissions with your configured service account. Make sure the account has access to ad-manager and scopes are configured properly.

I suggest to do it this way:
class Adx:
def __init__(self):
self.GOOGLEADS_YAML = 'googleads.yaml'
self.GOOGLEADS_VERSION = 'v202111'
self.google_keys = self.GOOGLEADS_YAML
def activate(self):
ad_manager_client = ad_manager.AdManagerClient.LoadFromStorage(self.google_keys)

jupyter notebook: NameError: name 'main' is not defined

I'm working with a Jupyter notebook. I'm trying to import a file called main.py which contains 2 lines:
import os
ROOT_DIR = os.path.dirname(os.path.abspath(__file__)) # This is your Project Root
My code:
import os
%load main.py
from main import ROOT_DIR
I'm getting
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
....\site-packages\IPython\core\interactiveshell.py in find_user_code(self, target, raw, py_only, skip_encoding_cookie, search_ns)
3282 try: # User namespace
-> 3283 codeobj = eval(target, self.user_ns)
3284 except Exception:
<string> in <module>()
NameError: name 'main' is not defined
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-3-1f13faed25b1> in <module>()
1 import os
----> 2 get_ipython().run_line_magic('load', 'main.py')
3 from main import ROOT_DIR
....\site-packages\IPython\core\interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
2129 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2130 with self.builtin_trap:
-> 2131 result = fn(*args,**kwargs)
2132 return result
2133
<decorator-gen-48> in load(self, arg_s)
....\site-packages\IPython\core\magic.py in <lambda>(f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):
....\site-packages\IPython\core\magics\code.py in load(self, arg_s)
342 search_ns = 'n' in opts
343
--> 344 contents = self.shell.find_user_code(args, search_ns=search_ns)
345
346 if 's' in opts:
....\site-packages\IPython\core\interactiveshell.py in find_user_code(self, target, raw, py_only, skip_encoding_cookie, search_ns)
3284 except Exception:
3285 raise ValueError(("'%s' was not found in history, as a file, url, "
-> 3286 "nor in the user namespace.") % target)
3287
3288 if isinstance(codeobj, str):
ValueError: 'main.py' was not found in history, as a file, url, nor in the user namespace.
What am I doing wrong?

Unexpected EOFError raised by pickle.load(). Is the file corrupted or what?

I am experiencing an odd problem while loading a particular file with pickle.
I am using PyPy on OSX. The files were originally generated with the same configuration using, obviously, pickle.dump().
The code below iterates over the pickled files and attempt to read them. Most of the files got loaded successfully but some (two found up to now) does not.
Each file is around 64MB and I have plenty of RAM. What can be the cause of the error?
poolFiles = get_all_files('../test','.pool')
counter = Counter()
for p in poolFiles:
print p
with open(p,'rb') as f:
bots = pickle.load(f)
for botID, genome in bots.iteritems():
counter.update({botID})
../test/best_performers.pool
../test/best_performers_01.pool
../test/best_performers_02.pool
../test/best_performers_03.pool
../test/best_performers_04.pool
../test/best_performers_05.pool
../test/best_performers_06.pool
../test/best_performers_07.pool
---------------------------------------------------------------------------
EOFError Traceback (most recent call last)
<ipython-input-4-22bb8d4b58db> in <module>()
5 print p
6 with open(p,'rb') as f:
----> 7 bots = pickle.load(f)
8 for botID, genome in bots.iteritems():
9 counter.update({botID})
/usr/local/Cellar/pypy/5.3.0/libexec/lib-python/2.7/pickle.pyc in load(file)
1425
1426 def load(file):
-> 1427 return Unpickler(file).load()
1428
1429 def loads(str):
/usr/local/Cellar/pypy/5.3.0/libexec/lib-python/2.7/pickle.pyc in load(self)
899 while 1:
900 key = read(1)
--> 901 dispatch[key](self)
902 except _Stop, stopinst:
903 return stopinst.value
/usr/local/Cellar/pypy/5.3.0/libexec/lib-python/2.7/pickle.pyc in load_eof(self)
921
922 def load_eof(self):
--> 923 raise EOFError
924 dispatch[''] = load_eof
925
EOFError: