I'm in need of a function returning only the significant part of a value with respect to a given error. Meaning something like this:
def (value, error):
""" This function takes a value and determines its significant
accuracy by its error.
It returns only the scientific important part of a value and drops the rest. """
magic magic magic....
return formated value as String.
What i have written so far to show what I mean:
import numpy as np
def signigicant(value, error):
""" Returns a number in a scintific format. Meaning a value has an error
and that error determines how many digits of the
value are signifcant. e.g. value = 12.345MHz,
error = 0.1MHz => 12.3MHz because the error is at the first digit.
(in reality drop the MHz its just to show why.)"""
xx = "%E"%error # I assume this is most ineffective.
xx = xx.split("E")
xx = int(xx[1])
if error <= value: # this should be the normal case
yy = np.around(value, -xx)
if xx >= 0: # Error is 1 or bigger
return "%i"%yy
else: # Error is smaller than 1
string = "%."+str(-xx) +"f"
return string%yy
if error > value: # This should not be usual but it can happen.
return "%g"%value
What I don't want is a function like numpys around or round. Those functions take a value and want to know what part of this value is important. The point is that in general I don't know how many digits are significant. It depends in the size of the error of that value.
Another example:
value = 123, error = 12, => 120
One can drop the 3, because the error is at the size of 10. However this behaviour is not so important, because some people still write 123 for the value. Here it is okay but not perfectly right.
For big numbers the "g" string operator is a usable choice but not always what I need. For e.g.
If the error is bigger than the value.( happens e.g. when someone wants to measure something that does not exist.)
value = 10, error = 100
I still wish to keep the 10 as the value because I done know it any better. The function should return 10 then and not 0.
The stuff I have written does work more or less, but its clearly not effective or elegant in any way. Also I assume this question does concern hundreds of people because every scientist has to format numbers in that way. So I'm sure there is a ready to use solution somewhere but I haven't found it yet.
Probably my google skills aren't good enough but I wasn't able to find a solution to this in two days and now I ask here.
For testing my code I used this the following but more is needed.
errors = [0.2,1.123,1.0, 123123.1233215,0.123123123768]
values = [12.3453,123123321.4321432, 0.000321 ,321321.986123612361236,0.00001233214 ]
for value, error in zip(values, errors):
print "Teste Value: ",value, "Error:", error
print "Result: ", signigicant(value, error)
import math
def round_on_error(value, error):
significant_digits = 10**math.floor(math.log(error, 10))
return value // significant_digits * significant_digits
Example:
>>> errors = [0.2,1.123,1.0, 123123.1233215,0.123123123768]
>>> values = [12.3453,123123321.4321432, 0.000321 ,321321.986123612361236,0.00001233214 ]
>>> map(round_on_error, values, errors)
[12.3, 123123321.0, 0.0, 300000.0, 0.0]
And if you want to keep a value that is inferior to its error
if (value < error)
return value
else
def round_on_error(value, error):
significant_digits = 10**math.floor(math.log(error, 10))
return value // significant_digits * significant_digits
Related
I am trying to run a simple LP pyomo Concrete model with Gurobisolver :
import pyomo.environ as pyo
from pyomo.opt import SolverFactory
model = pyo.ConcreteModel()
nb_years = 3
nb_mins = 2
step = 8760*1.5
delta = 10000
#Range of hour
model.h = pyo.RangeSet(0,8760*nb_years-1)
#Individual minimums
model.min = pyo.RangeSet(0, nb_mins-1)
model.mins = pyo.Var(model.min, within=model.h, initialize=[i for i in model.min])
def maximal_step_between_mins_constraint_rule(model, min):
next_min = min + 1 if min < nb_mins-1 else 0
if next_min == 0: # We need to take circularity into account
return 8760*nb_years - model.mins[min] + model.mins[next_min] <= step + delta
return model.mins[next_min] - model.mins[min] <= step + delta
def minimal_step_between_mins_constraint_rule(model, min):
next_min = min + 1 if min < nb_mins-1 else 0
if next_min == 0: # We need to take circularity into account
return 8760*nb_years - model.mins[min] + model.mins[next_min] >= step - delta
return model.mins[next_min] - model.mins[min] >= step - delta
model.input_list = pyo.Param(model.h, initialize=my_input_list, within=pyo.Reals, mutable=False)
def objective_rule(model):
return sum([model.input_list[model.mins[min]] for min in model.min])
model.maximal_step_between_mins_constraint= pyo.Constraint(model.min, rule=maximal_step_between_mins_constraint_rule)
model.minimal_step_between_mins_constraint= pyo.Constraint(model.min, rule=minimal_step_between_mins_constraint_rule)
model.objective = pyo.Objective(rule=objective_rule, sense=pyo.minimize)
opt = SolverFactory('gurobi')
results = opt.solve(model, options={'Presolve':2})
Basically I am trying to find two hours in my input list (which looks like this) spanning over 3 years of data, with constraints on the distance separating them, and where the sum of both value is minimized by the model.
I implemented my list as a parameter of fixed value, however even if mutable is set to False running my model produces this error :
ERROR: Rule failed when generating expression for Objective objective with
index None: RuntimeError: Error retrieving the value of an indexed item
input_list: index 0 is not a constant value. This is likely not what you
meant to do, as if you later change the fixed value of the object this
lookup will not change. If you understand the implications of using non-
constant values, you can get the current value of the object using the
value() function.
ERROR: Constructing component 'objective' from data=None failed: RuntimeError:
Error retrieving the value of an indexed item input_list: index 0 is not a
constant value. This is likely not what you meant to do, as if you later
change the fixed value of the object this lookup will not change. If you
understand the implications of using non-constant values, you can get the
current value of the object using the value() function.
Any idea why I get this error and how to fix it ?
Obviously, changing the objective function to sum([pyo.value(model.input_list[model.mins[min]]) for min in model.min]) is not a solution to my problem.
I also tried not to use pyomo parameters (with something like sum([input_list[model.mins[min]] for min in model.min]), but pyomo can't iterate over it and raises the following error :
ERROR: Constructing component 'objective' from data=None failed: TypeError:
list indices must be integers or slices, not _GeneralVarData
You have a couple serious syntax and structure problems in your model. Not all of the elements are included in the code you provide, but you (minimally) need to fix these:
In this snippet, you are initializing the value of each variable to a list, which is invalid. Start with no variable initializations:
model.mins = pyo.Var(model.min, within=model.h, initialize=[i for i in model.min])
In this summation, you appear to be using a variable as the index for some data. This is an invalid construct. The value of the variable is unkown when the model is built. You need to reformulate:
return sum([model.input_list[model.mins[min]] for min in model.min])
My suggestion: Start with a very small chunk of your data and pprint() your model and read it carefully for quality before you attempt to solve.
model.pprint()
This is a question I've had before: I have two arrays representing the inputs and corresponding outputs of a function. I need to find the input for a specific output that falls between data points. How do I do that?
For example:
import numpy as np
B = np.arange(0,10,1)
def fun(b):
return b*3/5
A = fun(B)
How to get the value of "B" for fun to return 3.75?
This technique uses linear interpolation to approximate.
I start with this function:
def interpABS(A,B,Aval):
if Aval>max(A) or Aval<min(A):
print('Error: Extrapolating beyond given data')
else:
if len(A)==len(B):
for i in np.arange(1,len(A),1):
ihi = i
ilo = i-1
if A[i]>Aval:
break
Alo = A[ilo]
Blo = B[ilo]
Ahi = A[ihi]
Bhi = B[ihi]
out = Blo + (Bhi-Blo)*(Aval-Alo)/(Ahi-Alo)
return out
else:
print('Error: inputs of different sizes')
Note: I'm kind of an amateur and don't know how to set up exceptions, so instead the error outputs are just print commands on a different path from the rest of the function. Those more experienced than I am may recommend improvements.
Use the output array from your function as A, and the corresponding input array as B, then input your target value as Aval. interpABS will return the an approximate input for your original function to get the target value
So, for our example above, interpABS(A,B,3.75) will return a value of 6.25
This can be useful even if Aval is a value of A to find the corresponding B value, since the math simplifies to Blo + 0. For example, changing Aval in the above example will give 5.0, which is part of the original input set B.
My specific concerns are
When parsing a parameter, is it intuitive for a future maintainer to understand code that depends on throwing an error?
Is it expensive to be throwing exceptions as a matter of course for the
default case? (seems like it might be according to https://stackoverflow.com/a/9859202/776940 )
Context
I have a parameter counter that determines the name of a counter to increment, and optionally can increment by a positive or negative integer separated from the counter name by an =. If no increment value is provided, the default magnitude of the increment is 1. The function is fed by breaking up a comma delimited list of counters and increments, so a valid input to the whole process can look like:
"counter1,counter2=2,counter3=-1"
which would increment "counter1" by 1, increment "counter2" by 2 and decrement "counter3" by 1.
How I Originally Wrote It
counterDescriptor = counterValue.split('=')
if len(counterDescriptor) == 1:
counterName = counterDescriptor[0]
counterIncr = 1
elif len(counterDescriptor) == 2:
counterName = counterDescriptor[0]
counterIncr = int(counterDescriptor[1])
else:
counterName, counterIncr = ('counterParsingError', 1)
which strikes me, as I recently came back to look at it, as overly verbose and clunky.
Is this a more or less Pythonic way to code that behavior?
def cparse(counter):
try:
desc,mag = counter.split('=')
except ValueError:
desc = counter
mag = ''
finally:
if mag == '':
mag = 1
return desc, int(mag)
With these test cases, I see:
>>> cparse("byfour=4")
('byfour', 4)
>>> cparse("minusone=-1")
('minusone', -1)
>>> cparse("equalAndNoIncr=")
('equalAndNoIncr', 1)
>>> cparse("noEqual")
('noEqual', 1)
These test cases that would have been caught how I originally wrote it (above) won't get caught this way:
>>> cparse("twoEquals=2=3")
('twoEquals=2=3', 1)
>>> cparse("missingComma=5missingComma=-5")
('missingComma=5missingComma=-5', 1)
and this last test case doesn't get caught by either way of doing it. Both make the int() vomit:
>>> cparse("YAmissingComma=5NextCounter")
ValueError: invalid literal for int() with base 10: '5NextCounter'
I'm glad I discovered this problem by asking this question. The service that consumes this value would eventually choke on it. I suppose I could change the one line return desc, int(mag) of the function to this:
if desc.find("=")<0 and (mag=='0' or (mag if mag.find('..') > -1 else mag.lstrip('-+').rstrip('0').rstrip('.')).isdigit()):
return desc, int(mag)
else:
return 'counterParsingError: {}'.format(desc), 1
(hat tip to https://stackoverflow.com/a/9859202/776940 for figuring out that this was the fastest way offered in that discussion to determine if a string is an integer)
I would consider that pythonic, though you might perhaps prefer:
def cparse(counter):
if "=" not in counter:
# early exit for this expected case
return (counter, 1)
desc, mag = counter.split("=", maxsplit=1)
# note the use of the optional maxsplit to prevent ValueErrors on "a=b=c"
# and since we've already tested and short-circuited out of the "no equals" case
# we can now consider this handled completely without nesting in a try block.
try:
mag = int(mag)
except ValueError:
# can't convert mag to an int, this is unexpected!
mag = 1
return (desc, mag)
You can tweak this to ensure you get the right output while parsing strings like a=b=c. If you expect to receive ('a', 1) then keep the code as-is. If you expect ('a=b', 1) you can use counter.rsplit instead of counter.split.
Hey so I am just working on some coding homework for my Python class using JES. Our assignment is to take a sound, add some white noise to the background and to add an echo as well. There is a bit more exacts but I believe I am fine with that. There are four different functions that we are making: a main, an echo equation based on a user defined length of time and amount of echos, a white noise generation function, and a function to merge the noises.
Here is what I have so far, haven't started the merging or the main yet.
#put the following line at the top of your file. This will let
#you access the random module functions
import random
#White noise Generation functiton, requires a sound to match sound length
def whiteNoiseGenerator(baseSound) :
noise = makeEmptySound(getLength(baseSound))
index = 0
for index in range(0, getLength(baseSound)) :
sample = random.randint(-500, 500)
setSampleValueAt(noise, index, sample)
return noise
def multipleEchoesGenerator(sound, delay, number) :
endSound = getLength(sound)
newEndSound = endSound +(delay * number)
len = 1 + int(newEndSound/getSamplingRate(sound))
newSound = makeEmptySound(len)
echoAmplitude = 1.0
for echoCount in range (1, number) :
echoAmplitude = echoAmplitude * 0.60
for posns1 in range (0, endSound):
posns2 = posns1 + (delay * echoCount)
values1 = getSampleValueAt(sound, posns1) * echoAmplitude
values2 = getSampleValueAt(newSound, posns2)
setSampleValueAt (newSound, posns2, values1 + values2)
return newSound
I receive this error whenever I try to load it in.
The error was:
Inappropriate argument value (of correct type).
An error occurred attempting to pass an argument to a function.
Please check line 38 of C:\Users\insanity180\Desktop\Work\Winter Sophomore\CS 140\homework3\homework_3.py
That line of code is:
setSampleValueAt (newSound, posns2, values1 + values2)
Anyone have an idea what might be happening here? Any assistance would be great since I am hoping to give myself plenty of time to finish coding this assignment. I have gotten a similar error before and it was usually a syntax error however I don't see any such errors here.
The sound is made before I run this program and I defined delay and number as values 1 and 3 respectively.
Check the arguments to setSampleValueAt; your sample value must be out of bounds (should be within -32768 - 32767). You need to do some kind of output clamping for your algorithm.
Another possibility (which indeed was the error, according to further input) is that your echo will be out of the range of the sample - that is, if your sample was 5 seconds long, and echo was 0.5 seconds long; or the posns1 + delay is beyond the length of the sample; the length of the new sound is not calculated correctly.
Edit: I did a first version, which Eike helped me to advance quite a bit on it. I'm now stuck to a more specific problem, which I will describe bellow. You can have a look at the original question in the history
I'm using pyparsing to parse a small language used to request specific data from a database. It features numerous keyword, operators and datatypes as well as boolean logic.
I'm trying to improve the error message sent to the user when he does a syntax error, since the current one is not very useful. I designed a small example, similar to what I'm doing with the language aforementioned but much smaller:
#!/usr/bin/env python
from pyparsing import *
def validate_number(s, loc, tokens):
if int(tokens[0]) != 0:
raise ParseFatalException(s, loc, "number musth be 0")
def fail(s, loc, tokens):
raise ParseFatalException(s, loc, "Unknown token %s" % tokens[0])
def fail_value(s, loc, expr, err):
raise ParseFatalException(s, loc, "Wrong value")
number = Word(nums).setParseAction(validate_number).setFailAction(fail_value)
operator = Literal("=")
error = Word(alphas).setParseAction(fail)
rules = MatchFirst([
Literal('x') + operator + number,
])
rules = operatorPrecedence(rules | error , [
(Literal("and"), 2, opAssoc.RIGHT),
])
def try_parse(expression):
try:
rules.parseString(expression, parseAll=True)
except Exception as e:
msg = str(e)
print("%s: %s" % (msg, expression))
print(" " * (len("%s: " % msg) + (e.loc)) + "^^^")
So basically, the only things which we can do with this language, is writing series of x = 0, joined together with and and parenthesis.
Now, there are cases, when and and parenthesis are used, where the error reporting is not very good. Consider the following examples:
>>> try_parse("x = a and x = 0") # This one is actually good!
Wrong value (at char 4), (line:1, col:5): x = a and x = 0
^^^
>>> try_parse("x = 0 and x = a")
Expected end of text (at char 6), (line:1, col:1): x = 0 and x = a
^^^
>>> try_parse("x = 0 and (x = 0 and (x = 0 and (x = a)))")
Expected end of text (at char 6), (line:1, col:1): x = 0 and (x = 0 and (x = 0 and (x = a)))
^^^
>>> try_parse("x = 0 and (x = 0 and (x = 0 and (x = 0)))")
Expected end of text (at char 6), (line:1, col:1): x = 0 and (x = 0 and (x = 0 and (xxxxxxxx = 0)))
^^^
Actually, it seems that if the parser can't parse (and parse here is important) something after a and, it doesn't produce good error messages anymore :(
And I mean parse, since if it can parse 5 but the "validation" fails in the parse action, it still produces a good error message. But, if it can't parse a valid number (like a) or a valid keyword (like xxxxxx), it stops producing the right error messages.
Any idea?
Pyparsing will always have somewhat bad error messages, because it backtracks. The error message is generated in the last rule that the parser tries. The parser can't know where the error really is, it only knows that there is no matching rule.
For good error messages you need a parser that gives up early. These parsers are less flexible than Pyparsing, but most conventional programming languages can be parsed with such parsers. (C++ and Scala IMHO can't.)
To improve error messages in Pyparsing use the - operator, it works like the + operator, but it does not backtrack. You would use it like this:
assignment = Literal("let") - varname - "=" - expression
Here is a small article on improving error reporting, by Pyparsing's author.
Edit
You could also generate good error messages for the invalid numbers in the parse actions that do the validation. If the number is invalid you raise an exception that is not caught by Pyparsing. This exception can contain a good error message.
Parse actions can have three arguments [1]:
s = the original string being parsed (see note below)
loc = the location of the matching substring
toks = a list of the matched tokens, packaged as a ParseResults object
There are also three useful helper methods for creating good error messages [2]:
lineno(loc, string) - function to give the line number of the location within the string; the first line is line 1, newlines start new rows.
col(loc, string) - function to give the column number of the location within the string; the first column is column 1, newlines reset the column number to 1.
line(loc, string) - function to retrieve the line of text representing lineno(loc, string). Useful when printing out diagnostic messages for exceptions.
Your validating parse action would then be like this:
def validate_odd_number(s, loc, toks):
value = toks[0]
value = int(value)
if value % 2 == 0:
raise MyFatalParseException(
"not an odd number. Line {l}, column {c}.".format(l=lineno(loc, s),
c=col(loc, s)))
[1] http://pythonhosted.org/pyparsing/pyparsing.pyparsing.ParserElement-class.html#setParseAction
[2] HowToUsePyparsing
Edit
Here [3] is an improved version of the question's current (2013-4-10) script. It gets the example errors right, but other error are indicated at the wrong position. I believe there are bugs in my version of Pyparsing ('1.5.7'), but maybe I just don't understand how Pyparsing works. The issues are:
ParseFatalException seems not to be always fatal. The script works as expected when I use my own exception.
The - operator seems not to work.
[3] http://pastebin.com/7E4kSnkm