Develop Reference

Sklearn's roc_auc_score error when in debug mode but runs fine while running normally?

I'm using roc_auc_score to evaluate AUC between two arrays, the truth and the estimation. My code runs fine when I'm executing it normally on PyCharms; however, when I use the debug mode, the below strange errors pop up. I tried pausing the code prior to the roc_auc_score line and attempted to just run it using the debug console with just 2 small arrays. Same issue. Strangely, everything is fine using the normal Python console. Any ideas?
Traceback (most recent call last):
File "C:\Users\User\anaconda3\envs\project\lib\site-packages\numpy\core\getlimits.py", line 649, in __init__
self.dtype = numeric.dtype(int_type)
TypeError: 'NoneType' object is not callable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm 2022.1.2\plugins\python\helpers\pydev\pydevd.py", line 1491, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2022.1.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/User/PycharmProjects/project/main-resnet.py", line 707, in <module>
results = train_net(net=net,
File "C:/Users/User/PycharmProjects/project/main-resnet.py", line 495, in train_net
train_auc = roc_auc_score(training_true, training_estimated)
File "C:\Users\User\anaconda3\envs\project\lib\site-packages\sklearn\metrics\_ranking.py", line 566, in roc_auc_score
y_true = label_binarize(y_true, classes=labels)[:, 0]
File "C:\Users\User\anaconda3\envs\project\lib\site-packages\sklearn\preprocessing\_label.py", line 546, in label_binarize
Y = sp.csr_matrix((data, indices, indptr), shape=(n_samples, n_classes))
File "C:\Users\User\anaconda3\envs\project\lib\site-packages\scipy\sparse\compressed.py", line 66, in __init__
idx_dtype = get_index_dtype((indices, indptr),
File "C:\Users\User\anaconda3\envs\project\lib\site-packages\scipy\sparse\sputils.py", line 153, in get_index_dtype
int32min = np.iinfo(np.int32).min
File "C:\Users\User\anaconda3\envs\project\lib\site-packages\numpy\core\getlimits.py", line 651, in __init__
self.dtype = numeric.dtype(type(int_type))
TypeError: 'NoneType' object is not callable
python-BaseException

Well... guess I was simply googling the wrong thing :P
It seems to be an issue between PyCharms and the python version (https://youtrack.jetbrains.com/issue/PY-52137).
Switching from Python 3.10.1 to 3.9 should work.

File used by another process error: Python

I am trying to change the contents of a destination file with the help of some source file. When the work is done, I am making sure that I have closed all the files in code (ofcourse, I have made sure that it is not opened in any editor too.)
def write_to_file(self, _source_path, _destination_path):
f_source = open(_source_path, 'r')
f_destination = open(_destination_path, 'r')
f_temp = open(self.temp_path, 'w+')
while source_line or destination_line:...
f_temp.close()
f_destination.close()
f_source.close()
shutil.move(self.temp_path, _destination_path)
return
I am getting below error:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\Program Files\JetBrains\PyCharm\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "C:\Program Files\JetBrains\PyCharm\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/rohit.tayal/PycharmProjects/AutoNVMerge/MainActivity.py", line 151, in <module>
script.write_to_file(source_path, destination_path)
File "C:/Users/rohit.tayal/PycharmProjects/AutoNVMerge/MainActivity.py", line 62, in write_to_file
shutil.move(self.temp_path, _destination_path)
File "C:\Anaconda\envs\AutoNVMerge\lib\shutil.py", line 581, in move
os.unlink(src)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\auto_nv_merge.xml'
C:\auto_nv_merge.xml is my temp file.

When working with
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\auto_nv_merge.xml'
you have 2 options, be a hero and try to hunt down the process that is holding that file hostage or just restart your computer.
I usually go with the latter, restart

Attribute error in python with Google Speech API

'str' object has no attribute 'long_running_recognize'. This error is occurring at the last line of the code shown. I jump into the code and see no obvious reason for it to fail. Jumps out at the wrap_method below.
if "long_running_recognize" not in self._inner_api_calls:
self._inner_api_calls[
"long_running_recognize"
] = google.api_core.gapic_v1.method.wrap_method(
self.transport.long_running_recognize,
default_retry=self._method_configs["LongRunningRecognize"].retry,
default_timeout=self._method_configs["LongRunningRecognize"].timeout,
client_info=self._client_info,
)
The only issue that I could conceive here is maybe the audio file is too large(45mb...53 min). But the code break happens in a location that doesn't appear to relate to this.
import io
import os
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types
with open("path\\My First Project-edb18653fedf.json") as f:
GOOGLE_APPLICATION_CREDENTIALS = f.read()
client = speech.SpeechClient(GOOGLE_APPLICATION_CREDENTIALS)
file_path = "audio\\102.flac"
with io.open(file_path, 'rb') as audio_file:
content = audio_file.read()
audio = types.RecognitionAudio(content=content)
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.FLAC,
sample_rate_hertz=32000,
language_code='en-US')
operation = client.long_running_recognize(config, audio)
Connected to pydev debugger (build 191.7479.30)
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 1758, in <module>
main()
File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 1752, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py", line 1147, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/shear/Documents/PythonScripts/GStTEnv/SpeechToText/CloudSpeechText.py", line 105, in <module>
operation = client.long_running_recognize(config, audio)
File "C:\Users\shear\GStTEnv\lib\site-packages\google\cloud\speech_v1\gapic\speech_client.py", line 314, in long_running_recognize
self.transport.long_running_recognize,
AttributeError: 'str' object has no attribute 'long_running_recognize'

Did you try to prove out your theory about the length of the audio? What happens if you trim the audio and try it?
Does it happen if you try a curl request at the command line instead of code?
I’d like to see if I can help. To diagnose further, would you be able to provide:
- Exact steps to reproduce
- Pointer to github repo if available
- As much of the code as you can provide
- Exact error messages
- Language Code (config)
- All configuration options applicable
- Sample audio file with all personal data removed (on Google Drive or Cloud Storage)

Pandas unable to open this Excel file

I am trying to use python pandas to open an Excel file. Code is simple as shown below;
import pandas as pd
df = pd.read_excel('../TestXLWings.xlsm', sheetname="TestSheet")
I got an error below;
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.2\helpers\pydev\pydevd.py", line 1599, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.2\helpers\pydev\pydevd.py", line 1026, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm Community Edition 2017.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/testing/Dropbox/Test-XLwings/test.py", line 3, in <module>
df = pd.read_excel('../TestXLWings.xlsm', sheetname="TestSheet")
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\excel.py", line 203, in read_excel
io = ExcelFile(io, engine=engine)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\io\excel.py", line 260, in __init__
self.book = xlrd.open_workbook(io)
File "C:\ProgramData\Anaconda3\lib\site-packages\xlrd\__init__.py", line 441, in open_workbook
ragged_rows=ragged_rows,
File "C:\ProgramData\Anaconda3\lib\site-packages\xlrd\book.py", line 87, in open_workbook_xls
ragged_rows=ragged_rows,
File "C:\ProgramData\Anaconda3\lib\site-packages\xlrd\book.py", line 595, in biff2_8_load
raise XLRDError("Can't find workbook in OLE2 compound document")
xlrd.biffh.XLRDError: Can't find workbook in OLE2 compound document
My Excel file is xlsm and protected by password. What does OLE2 compound document mean exactly? Does pandas have problems opening this kind of Excel files? I am using python v3.6

I will answer my own question. In one of the comments from ayhan, Excel-protected files cannot be read by xlrd. One solution is to remove the protection.
I need the command to unprotect an Excel file from python
Another solution to read the Excel-protected file is to use xlwings. I have verified that xlwings is able to read protected Excel files when the Excel file is opened.

I would create a new excel file and remove sensitivity label in excel. Then be able to read the file with pd.

python memory error when loading MNIST.pkl.gz

I am new to Python and I have downloaded the code DBN.py but there is a problem:when I was trying to load the dataset MNIST.pkl.gz.there is always an meomory error..
my code is very simple:
import cPickle, gzip, numpy
# Load the dataset
f = gzip.open('C:\Users\MAC\Desktop\mnist.pkl.gz', 'rb')
train_set, valid_set, test_set = cPickle.load(f)
f.close()
and the error is as follows:
Traceback (most recent call last):
File "<ipython-input-17-528eea6bbfdd>", line 1, in <module>
runfile('C:/Users/MAC/Documents/Python Scripts/untitled0.py', wdir='C:/Users/MAC/Documents/Python Scripts')
File "C:\Users\MAC\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 699, in runfile
execfile(filename, namespace)
File "C:\Users\MAC\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 74, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "C:/Users/MAC/Documents/Python Scripts/untitled0.py", line 19, in <module>
train_set, valid_set, test_set = cPickle.load(f)
File "C:\Users\MAC\Anaconda\lib\gzip.py", line 268, in read
self._read(readsize)
File "C:\Users\MAC\Anaconda\lib\gzip.py", line 320, in _read
self._add_read_data( uncompress )
File "C:\Users\MAC\Anaconda\lib\gzip.py", line 338, in _add_read_data
self.extrabuf = self.extrabuf[offset:] + data
MemoryError
I really have no idea,is it because the memory of my computer is too small? it is on windows 7,32 bits

I suspect the problem to be Spyder in this case.
As to why, I have no idea but either the process isn't allowed to allocate enugh memory outside of it's own script or it simply gets stuck in a loop some how.
Try running your code without Spyder by pasting your code into myscript.py for instance and open a terminal and navigate to the folder where you saved your script and run python myscript.py and see if that works or gives the same output.
This is based on a conversation in the comments above.