Develop Reference

Editorconfig for python docstring

I am using the pycharm IDE and I would like to automatize the indentation of the python docstrings with the .editorconfig file. I get to control almost everything with the following configuration (just in case it is useful):
[{*.py,*.pyw}]
ij_python_align_collections_and_comprehensions = true
ij_python_align_multiline_imports = true
ij_python_align_multiline_parameters = true
ij_python_align_multiline_parameters_in_calls = true
ij_python_blank_line_at_file_end = true
ij_python_blank_lines_after_imports = 1
ij_python_blank_lines_after_local_imports = 0
ij_python_blank_lines_around_class = 1
ij_python_blank_lines_around_method = 1
ij_python_blank_lines_around_top_level_classes_functions = 2
ij_python_blank_lines_before_first_method = 0
ij_python_call_parameters_new_line_after_left_paren = false
ij_python_call_parameters_right_paren_on_new_line = false
ij_python_call_parameters_wrap = normal
ij_python_dict_alignment = 0
ij_python_dict_new_line_after_left_brace = false
ij_python_dict_new_line_before_right_brace = false
ij_python_dict_wrapping = 1
ij_python_from_import_new_line_after_left_parenthesis = false
ij_python_from_import_new_line_before_right_parenthesis = false
ij_python_from_import_parentheses_force_if_multiline = false
ij_python_from_import_trailing_comma_if_multiline = false
ij_python_from_import_wrapping = 1
ij_python_hang_closing_brackets = false
ij_python_keep_blank_lines_in_code = 1
ij_python_keep_blank_lines_in_declarations = 1
ij_python_keep_indents_on_empty_lines = false
ij_python_keep_line_breaks = true
ij_python_method_parameters_new_line_after_left_paren = false
ij_python_method_parameters_right_paren_on_new_line = false
ij_python_method_parameters_wrap = normal
ij_python_new_line_after_colon = false
ij_python_new_line_after_colon_multi_clause = true
ij_python_optimize_imports_always_split_from_imports = false
ij_python_optimize_imports_case_insensitive_order = false
ij_python_optimize_imports_join_from_imports_with_same_source = false
ij_python_optimize_imports_sort_by_type_first = true
ij_python_optimize_imports_sort_imports = true
ij_python_optimize_imports_sort_names_in_from_imports = false
ij_python_space_after_comma = true
ij_python_space_after_number_sign = true
ij_python_space_after_py_colon = true
ij_python_space_before_backslash = true
ij_python_space_before_comma = false
ij_python_space_before_for_semicolon = false
ij_python_space_before_lbracket = false
ij_python_space_before_method_call_parentheses = false
ij_python_space_before_method_parentheses = false
ij_python_space_before_number_sign = true
ij_python_space_before_py_colon = false
ij_python_space_within_empty_method_call_parentheses = false
ij_python_space_within_empty_method_parentheses = false
ij_python_spaces_around_additive_operators = true
ij_python_spaces_around_assignment_operators = true
ij_python_spaces_around_bitwise_operators = true
ij_python_spaces_around_eq_in_keyword_argument = false
ij_python_spaces_around_eq_in_named_parameter = false
ij_python_spaces_around_equality_operators = true
ij_python_spaces_around_multiplicative_operators = true
ij_python_spaces_around_power_operator = true
ij_python_spaces_around_relational_operators = true
ij_python_spaces_around_shift_operators = true
ij_python_spaces_within_braces = false
ij_python_spaces_within_brackets = false
ij_python_spaces_within_method_call_parentheses = false
ij_python_spaces_within_method_parentheses = false
ij_python_use_continuation_indent_for_arguments = false
ij_python_use_continuation_indent_for_collection_and_comprehensions = false
ij_python_use_continuation_indent_for_parameters = true
ij_python_wrap_long_lines = false
It would be rather useful to manage all the indentation config from the same file. That is, adding the docstring configuration to the .editorconfig. Someone knows if it is possible to control the docstring style by using .editorconfig?

Sklearn KernelDensity gives identical results for two different models

I'm having trouble with KernelDensity from sklearn. I put in two completely different arrays to create two different models, but the two models have identical results (scores). They should have different results for different models, shouldn't they?
Here's my code:
from sklearn.neighbors import KernelDensity
import numpy as np
kde = KernelDensity(kernel="gaussian", bandwidth=15)
def reproducible_example():
X1 = np.array([9,18,28,35,54,59,65,83,89,116,119,124,144])
X2 = np.array([39,51,57,61,66,81,88,103,120,126,130,132,134])
model1 = kde.fit(X1[:, np.newaxis])
model2 = kde.fit(X2[:, np.newaxis])
X_plot = np.linspace(0, 129, 130)
score1 = model1.score_samples(X_plot[:, np.newaxis])
score2 = model2.score_samples(X_plot[:, np.newaxis])
print(np.exp(score1) == np.exp(score2))
reproducible_example()
The expected output is False, as the two different models should return two different scores. Instead, the output is this:
[ True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True True True
True True True True True True True True True True]
Indicating that the two results are identical. How is that possible?

Pandas dropna not working as expected

I have a dataframe and I used dropna() on it successfully as shown:
proc_train.isnull().any()
id False
perc_premium_paid_by_cash_credit False
age_in_days False
Income False
Count_3-6_months_late False
Count_6-12_months_late False
Count_more_than_12_months_late False
application_underwriting_score False
no_of_premiums_paid False
premium False
renewal False
sourcing_channel_B False
sourcing_channel_C False
sourcing_channel_D False
sourcing_channel_E False
Urban/Rural False
prem_to_inc_ratio False
late36_612 False
late36_12more False
late612_12more False
perc_times_prem False
Then I try to take a selection of the data to use as input variables:
X_train = proc_train.loc[:, proc_train.columns != 'renewal']
X_train = X.loc[:, X.columns != 'id']
but it then gives all the null values back:
X_train.isnull().any()
perc_premium_paid_by_cash_credit False
age_in_days False
Income False
Count_3-6_months_late True
Count_6-12_months_late True
Count_more_than_12_months_late True
application_underwriting_score True
no_of_premiums_paid False
premium False
sourcing_channel_B False
sourcing_channel_C False
sourcing_channel_D False
sourcing_channel_E False
Urban/Rural False
prem_to_inc_ratio False
late36_612 True
late36_12more True
late612_12more True
perc_times_prem False
Why does this happen and what would be a better way to run this?

This section:
X_train = X.loc[:, X.columns != 'id']
should be
X_train = X_train.loc[:, X_train.columns != 'id']
which produces the same all-False result for isnull().any() as before.

Maximum limit in the length of expression evaluated by eval() in python

Consider the example
a = "( False or False ) and not ( False and True and False ) and not ( False and True and False ) "
print eval(a)
b = "( False or False or False or False or False or False or True or False or False or False or False or False or False or False or False or False or False or False ) and not False and not False and not ( False and False ) and not ( False and False ) and not ( False and False ) and not ( False and False ) and not ( False and False ) and not ( False and False ) and not ( False and True ) and not ( False and False ) and not ( False and False ) and not ( False and False ) and not ( False and False ) and not ( False and False) and not False"
print eval(b)
First one gives proper output. but for second eventhough synax is correct it is giving
SyntaxError: EOL while scanning string literal
because of length. I need to evaluate large expressions in my program. Any suggestions?

Try to find the limit empirically:
b = 'False or False'
while True:
try:
b = b + b[5:]
print len(b), eval(b)
except:
print len(b)
break
I stopped it at len(b) == 288MiB. Interestingly, python used up to 5.5GiB of RAM at the 288MiB level.

New line character overwrites the following byte [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I've been writing a code that inserts new line character('\n') at the end of the '#include' statements of the C language.
ex)
[BEFORE]
#include <stdio.h> #include <stdlib.h>
[AFTER]
#include <stdio.h>
#include <stdlib.h>
So I made a function for that. The varaible 'raw' stands for the string buffer of raw data, and idx is an index variable which moves through the string variable 'raw'.
def insert(raw, idx, new): # inserts string(new) to the original string(raw). Location of insertion can be specified by variable idx.
return raw[:idx] + new + raw[idx:]
Using the function above, I also wrote a code like this:
includeFlag1 = False # be ready to detect '<' of the include statement.
includeFlag2 = False # Turned on when the '<' is detected. Turned off when the include statement ends.
elif raw[idx:idx+8] == '#include' : # include statement is detected.
includeFlag1 = True
elif raw[idx] == '<' and includeFlag1 == True:
includeFlag2 = True
elif raw[idx] == '>' and includeFlag1 == True:
raw = insert(raw, idx+1, '\n') # adds the new-line charcter at the end of the include statement.
includeFlag2 = False # the include statement has just ended.
includeFlag1 = False
idx = idx + 1
The real problem is that when I run the code above, the new line character overwrites the '#' character, which is the starting point of the '#include <>' statement.
For more information, please look at the debugging log below.
The log is recorded like this:
raw[idx] [icdFlag] includeFlag1 includeFlag2
===================================================================
# [icdFlag] False False
i [icdFlag] True False
n [icdFlag] True False
c [icdFlag] True False
l [icdFlag] True False
u [icdFlag] True False
d [icdFlag] True False
e [icdFlag] True False
[icdFlag] True False
< [icdFlag] True False
s [icdFlag] True True
t [icdFlag] True True
d [icdFlag] True True
i [icdFlag] True True
o [icdFlag] True True
. [icdFlag] True True
h [icdFlag] True True
> [icdFlag] True True
[icdFlag] False False
i [icdFlag] False False
n [icdFlag] False False
c [icdFlag] False False
l [icdFlag] False False
u [icdFlag] False False
d [icdFlag] False False
e [icdFlag] False False
[icdFlag] False False
< [icdFlag] False False
s [icdFlag] False False
t [icdFlag] False False
d [icdFlag] False False
l [icdFlag] False False
i [icdFlag] False False
b [icdFlag] False False
. [icdFlag] False False
h [icdFlag] False False
> [icdFlag] False False
# [icdFlag] False False
i [icdFlag] True False
n [icdFlag] True False
c [icdFlag] True False
l [icdFlag] True False
u [icdFlag] True False
d [icdFlag] True False
e [icdFlag] True False
[icdFlag] True False
< [icdFlag] True False
s [icdFlag] True True
t [icdFlag] True True
r [icdFlag] True True
i [icdFlag] True True
n [icdFlag] True True
g [icdFlag] True True
. [icdFlag] True True
h [icdFlag] True True
> [icdFlag] True True
[icdFlag] False False
i [icdFlag] False False
n [icdFlag] False False
c [icdFlag] False False
l [icdFlag] False False
u [icdFlag] False False
d [icdFlag] False False
e [icdFlag] False False
[icdFlag] False False
< [icdFlag] False False
t [icdFlag] False False
i [icdFlag] False False
m [icdFlag] False False
e [icdFlag] False False
. [icdFlag] False False
h [icdFlag] False False
> [icdFlag] False False
v [icdFlag] False False
o [icdFlag] False False
i [icdFlag] False False
d [icdFlag] False False
[icdFlag] False False
g [icdFlag] False False
e [icdFlag] False False
t [icdFlag] False False
_ [icdFlag] False False
u [icdFlag] False False
s [icdFlag] False False
e [icdFlag] False False
r [icdFlag] False False
_ [icdFlag] False False
i [icdFlag] False False
n [icdFlag] False False
f [icdFlag] False False
o [icdFlag] False False
( [icdFlag] False False
c [icdFlag] False False
h [icdFlag] False False
a [icdFlag] False False
r [icdFlag] False False
[icdFlag] False False
* [icdFlag] False False
Of course, I've already checked whether the original C source code wasn't missing the '#' characters, but it didn't. I wonder why this kind of thing happens.

I'd probably just break it up into lines and split by the "include".
processedLines = [] # put into an array
lines = raw.split('\n')
for line in lines:
if "#include" in line:
includes = line.split("#include ") # "#include <foo.h> #include <bar.h>" becomes ["", "<foo.h>", "<bar.h>"]
for include in includes[1:] # skip the blank at the beginning
processedLines.append("#include " + include)
else
processedLines.append(line)

Perhaps this?
#!/usr/bin/env python
my_code = """
# Other stuff
#include <stdio.h> #include <stdlib.h> #include <other.h>
# Other stuff
"""
import os
import re
for line in my_code.splitlines():
include_list = re.findall(r"include <\w+\.h>", line)
if include_list:
print(os.linesep.join(include_list))
else:
print(line)