[ot-users] Sample transformation

regis lebrun regis_anne.lebrun_dutfoy at yahoo.fr
Thu Oct 5 15:40:58 CEST 2017

Hi Pamphile,

1) The problem:
The problem you get is due to the fact that in your version of OpenTURNS (1.7 I suppose), the GaussProductExperiment class has a different way to handle the input distribution than the other WeightedExperiment classes: it generates the quadrature rule of the *standard representatives* of the marginal distributions instead of the marginal distributions. It does not change the rate of convergence of the PCE algorithm and allows to use specific algorithms for distributions with known orthonormal polynomials. It is not explained in the documentation and if you ask the doe for its distribution it will give you the initial distribution instead of the standardized one.

2) The mathematical background:
The generation of quadrature rules for arbitrary 1D distributions is a badly conditioned problem. Even if the quadrature rule is well-defined (existence of moments of any order, distribution characterized by these moments), the application that maps the recurrence coefficients of the orthogonal polynomials to their value can have a very large condition number. As a result, the adaptive integration used to compute the recurrence coefficients of order n, based on the values of the polynomials of degree n-1 and n-2, can lead to wrong values and all the process falls down.

3) The current state of the software:
Since version 1.8, OpenTURNS no more generates the quadrature rule of the standard representatives, but the quadrature rule of the actual marginal distributions. The AdaptiveStieltjesAlgorithm class, introduced in release 1.8, is much more robust than the previous orthonormalization algorithms and is able to handle even stiff problems. There are still difficult situations (distributions with discontinuous PDF inside of the range, fixed in OT 1.9, or really badly conditioned distributions, hopefully fixed when ticket#885 will be solved) but most usual situations are under control even with marginal degrees of order 20.

4) The (probable) bug in your code and the way to solve it
You must be aware of the fact that the distribution you put into your WeightedExperiment object will be superseded by the distribution corresponding to your OrthogonalBasisFactory inside of the FunctionalChaosAlgorithm. If you need to have the input sample before to run the functional chaos algorithm, then you have to build your transformation by hand. Assuming that you already defined your projection basis called 'myBasis', your marginal integration degrees 'myDegrees' and your marginal distributions 'myMarginals', you have to write (in OT 1.7):

# Here the explicit cast into a NumericalMathFunction is to be able to evaluate the transformation over a sample
myTransformation = ot.NumericalMathFunction(ot.MarginalTransformationEvaluation([myBasis.getDistribution().getMarginal(i) for i in range(dimension), myMarginals))
sample = myTransformation(ot.GaussProductExperiment(myBasis.getDistribution(), myDegrees).generate())

You should avoid to cast OT objects into np objects as much as possible, and if you cannot avoid these casts you should do them only in the sections where they are needed. They can be expansive for large objects, and if the sample you get from generate() is used only as an argument of a NumericalMathFunction, then it will be converted back into a NumericalSample!

Best regards

De : roy <roy at cerfacs.fr>
À : users <users at openturns.org> 
Envoyé le : Jeudi 5 octobre 2017 11h13
Objet : [ot-users] Sample transformation


I am facing consistency concerns in the API regarding distributions and sampling.

The initial goal was to get the sampling for Polynomial Chaos as I must not use the model variable.
So for least square strategy I do something like this:

proj_strategy = ot.LeastSquaresStrategy(montecarlo_design)
sample = np.array(proj_strategy.getExperiment().generate())

sample is correct as the bounds of each feature lie in the corresponding ranges.

But now if I want to use IntegrationStrategy:

ot.IntegrationStrategy(ot.GaussProductExperiment(dists, list))
sample = np.array(proj_strategy.getExperiment().generate())

sample’s outputs lie between [-1, 1] which does not corresponds to the distribution I have initially.

So I used the conversion class but it does not work well with GaussProductExperiment as it requires [0, 1] instead of [-1, 1].

Thus I use this hack:

# Convert from [-1, 1] -> input distributions
marg_inv_transf = ot.MarginalTransformationEvaluation(distributions, 1)
sample = (proj_strategy.getExperiment().generate() + 1) / 2.

Is it normal that the distribution classes are not returning in the same intervals?

Thanks for your support!

Pamphile ROY
Chercheur doctorant en Quantification d’Incertitudes
CERFACS - Toulouse (31) - France
+33 (0) 5 61 19 31 57
+33 (0) 7 86 43 24 22

OpenTURNS users mailing list
users at openturns.org

More information about the users mailing list