Efficient Predictive Inferences for Future Outcomes Under Parametric Uncertainty of Underlying Models
Nicholas A. Nechval^{1, *}, Natalija Ribakova^{2}, Gundars Berzins^{3}
1Department of Mathematics, Baltic International Academy, Riga, Latvia
^{2}Department of Marketing, University of Latvia, Riga, Latvia
^{3}Department of Management, University of Latvia, Riga, Latvia
Email address:
To cite this article:
Nicholas A. Nechval, Natalija Ribakova, Gundars Berzins. Efficient Predictive Inferences for Future Outcomes Under Parametric Uncertainty of Underlying Models. American Journal of Theoretical and Applied Statistics. Special Issue: Novel Ideas for Efficient Optimization of Statistical Decisions and Predictive Inferences under Parametric Uncertainty of Underlying Models with Applications. Vol. 5, No. 2-1, 2016, pp. 49-55. doi: 10.11648/j.ajtas.s.2016050201.17
Abstract: Predictive inferences (predictive distributions, prediction and tolerance limits) for future outcomes on the basis of the past and present knowledge represent a fundamental problem of statistics, arising in many contexts and producing varied solutions. In this paper, new-sample prediction based on a previous sample (i.e., when for predicting the future outcomes in a new sample there are available the observed data only from a previous sample), within-sample prediction based on the early data from a current experiment (i.e., when for predicting the future outcomes in a sample there are available the early data only from that sample), and new-within-sample prediction based on both the early data from that sample and the data from a previous sample (i.e., when for predicting the future outcomes in a new sample there are available both the early data from that sample and the data from a previous sample) are considered. It is assumed that only the functional form of the underlying distributions is specified, but some or all of its parameters are unspecified. In such cases ancillary statistics and pivotal quantities, whose distribution does not depend on the unknown parameters, are used. In order to construct predictive inferences for future outcomes, the invariant embedding technique representing the exact pivotal-based method is proposed.In particular, this technique can be used for optimization of inventory management problems. A practical example is given.
Keywords: Future Outcomes, Parametric Uncertainty, Predictive Inferences
1. Introduction
Prediction of future outcomes based on the past and current data is the most prevalent form of statistical inference. Predictive inferences for future outcomes are widely used in risk management, finance, insurance, economics, hydrology, material sciences, telecommunications, and many other industries. Practical problems often require the computation of predictive distributions, prediction and tolerance limits for future values of random quantities. Consider the following examples: 1) A consumer purchasing a refrigerator would like to have a lower limit for the failure time of the unit to be purchased (with less interest in distribution of the population of units purchased by other consumers); 2) Financial managers in manufacturing companies need upper prediction limits on future warranty costs. A large number of problems in inventory management, production planning and scheduling, location, transportation, finance, and engineering design require that decisions be made in the presence of uncertainty. Most of the inventory management literature assumes that demand distributions are specified explicitly. However, in many practical situations, the true demand distributions are not known, and the only information available may be a time-series of historic demand data. When the demand distribution is unknown, one may either use a parametric approach (where it is assumed that the demand distribution belongs to a parametric family of distributions) or a non-parametric approach (where no assumption regarding the parametric form of the unknown demand distribution is made). Under the parametric approach, one may choose to estimate the unknown parameters or choose a prior distribution for the unknown parameters and apply the Bayesian approach to incorporating the demand data available. Scarf [1] and Karlin [2] consider a Bayesian framework for the unknown demand distribution. Specifically, assuming that the demand distribution belongs to the family of exponential distributions, the demand process is characterized by the prior distribution on the unknown parameter. Further extension of this approach is presented in [3]. Parameter estimation is considered in [4]. Liyanage and Shanthikumar [5] propose the concept of operational statistics and apply it to a single period newsvendor inventory control problem. In this paper we consider the case, where it is known that the demand distribution function belongs to a parametric family of distribution functions. However, unlike in the Bayesian approach, we do not assume any prior knowledge on the parameter values. Conceptually, it is useful to distinguish between "new-sample" prediction, "within-sample" prediction, and "new-within-sample" prediction. For new-sample prediction, data from a past sample are used to make predictions of future outcomes from the same process or population. For within-sample prediction, the problem is to predict future outcomes in a sample or process based on early data from that sample or process. For new-within-sample prediction, the problem is to predict future outcomes in a sample or process based on early data from that sample or process as well as on past sample data from the same process or population. Various solutions have been proposed for the prediction problem, that is, the problem of making inferences on a random sample {Y_{j}; j = 1, …, m} given independent observations {X_{i}; i=1, …, n} drawn from the same distribution. The Y_{j}’s and the X_{i}’s are commonly featured as "future outcomes" and "past outcomes" respectively. Inferences usually bear on some reduction Z of the Y_{j}’s - possibly a minimal sufficient statistic - and consist of either prediction intervals or likelihood or predictive distribution for Z, depending on different authors. Kaminsky and Nelson [6] discussed point and interval prediction of order statistics. Best linear unbiased predictors based on location-scale family of distributions are reviewed. Prediction intervals based on such predictors as well as those based on pivotals are studied. A brief discussion of Bayesian prediction is also given. Predictive distributions are found in the Bayesian framework (see Aitchison and Sculthorpe [7]). Lawless [8] applied the conditional method, which was first suggested by Fisher [9] and promoted further by a number of others (Nechval et al. [10]; Murthy et al. [11]), to different problems relating to the Weibull and extreme value distributions. In practice the proposed methods have limited applications and it is the purpose of this paper to obtain predictive inferences concerning Z via the simple invariant embedding technique [12-16]. The obtained results are given below.
2. Prediction Situations
Let us assume that the random variable X follows the exponential distribution with the probability density function
(1)
and the cumulative distribution function
(2)
where σ is the scale parameter (σ > 0).
2.1. New-Sample Prediction
Theorem 1. Let X_{1} ≤... ≤ X_{k} be the first k ordered observations from the previous sample of size n from the exponential distribution (1) and Y_{r} be the rth order statistic in a set of m future ordered observations Y_{1}≤…≤Y_{m} also from the distribution (1). Then the probability distribution function of the ancillary statistic Y_{r }/S_{k} is given by
(3)
where
(4)
Proof. The joint density of X_{1} ≤... ≤ X_{k} is given by
(5)
It is known that
(6)
is the sufficient statistic for σ. Then
(7)
is the pivotal quantity, the probability density function of which is given by
(8)
Using the invariant embedding technique [12-16], we reduce
(9)
to
(10)
where
(11)
It follows from (10) that
(12)
This ends the proof.
Corollary 1.1. A lower one-sided new-sample α prediction limit h on the rth order statistic Y_{r} in a set of m future ordered observations Y_{1}≤…≤Y_{m} is given by
(13)
where h satisfies the equation
(14)
(Observe that an upper one-sided 1-α prediction limit h may be obtained from a lower one-sided α prediction limit by replacing α by 1-α.)
2.2. Within-Sample Prediction
Theorem 2. Let Y_{1} ≤... ≤ Y_{l} be the first l ordered observations (order statistics) in a sample of size m from a continuous distribution with some probability density function f_{θ }(x) and distribution function F_{θ }(x), where θ is a parameter (in general, vector). Then the joint probability density function of Y_{1} ≤... ≤ Y_{l} and the rth order statistics Y_{r} (1 ≤ l < r ≤ m) is given by
(15)
where
(16)
(17)
represents the conditional probability density function of Y_{r} given Y_{l}=y_{l}.
Proof. The joint density of Y_{1} ≤... ≤ Y_{l} and Y_{r} is given by
(18)
It follows from (18) that
(19)
i.e., the conditional distribution of Y_{r}, given Y_{i}_{ }= y_{i} for all i = 1,…, l, is the same as the conditional distribution of Y_{r}, given only Y_{l}_{ }= y_{l}, which is given by (17). This ends the proof.
Corollary 2.1. The conditional probability distribution function of Y_{r} given Y_{l}=y_{l} is
(20)
Corollary 2.2. Let Y_{1} ≤... ≤ Y_{l} be the first l order statistics in a sample of size m from the exponential distribution (1). Then the conditional probability distribution function of Y_{r} given Y_{l}=y_{l} is
(21)
Theorem 3. Let Y_{1} ≤... ≤ Y_{l} be the first l ordered observations (order statistics) in a sample of size m from the exponential distribution (1), where the parameter σ is unknown. Then the probability distribution function of the ancillary statistic Y_{r }/Y_{l} is given by
(22)
Proof. We reduce (20) to
(23)
where Y_{r} /Y_{l} is the ancillary statistic whose distribution does not depend on the parameter σ, Y_{l}_{ }/σ is the pivotal quantity. Using the probability density function of Y_{l }/σ, we eliminate the unknown parameter σ from the problem as
(24)
where
(25)
represents the probability density function of the pivotal quantity Y_{l}_{ }/σ. This ends the proof.
Corollary 3.1. A lower one-sided within-sample α prediction limit h on the rth order statistic Y_{r} in a set of m future ordered observations Y_{1}≤…≤Y_{m} is given by
(26)
where h satisfies the equation
(27)
Theorem 4. Let Y_{1} ≤... ≤ Y_{l} be the first l ordered observations (order statistics) in a sample of size m from the exponential distribution (1), where the parameter σ is unknown. Then the probability distribution function of the ancillary statistic (Y_{r }-Y_{l})/S_{l} is given by
(28)
where
(29)
is the sufficient statistic for σ.
Proof. Using the technique of invariant embedding [12-16], we reduce (20) to
(30)
where (Y_{r}- Y_{l})/S_{l} is the ancillary statistic whose distribution does not depend on the parameter σ; S_{l }/σ is the pivotal quantity. Since the probability density function of S_{l }/σ is known, we eliminate the unknown parameter σ from the problem as
(31)
where
(32)
This ends the proof.
Corollary 4.1. A lower one-sided within-sample α prediction limit h on the rth order statistic Y_{r} in a set of m future ordered observations Y_{1}≤…≤Y_{m} is given by
(33)
where h satisfies the equation
(34)
2.3. New-Within-Sample Prediction
Theorem 5. Let X_{1} ≤... ≤ X_{k} be the first k ordered observations from a previous sample of size n from the exponential distribution (1) and Y_{1} ≤... ≤ Y_{l} be the first l ordered early observations from a new sample of size m also from the distribution (1), where the parameter σ is unknown. Then the probability distribution function of the ancillary statistic (Y_{r }-Y_{l})/(S_{k}+S_{l}) is given by
(35)
Proof. For the proof we refer to Theorems 1 and 4.
Corollary 5.1. A lower one-sided within-sample α prediction limit h on the rth order statistic Y_{r} in a set of m future ordered observations Y_{1}≤…≤Y_{m} is given by
(36)
where h satisfies the equation
(37)
3. Practical Example
Consider a single-period newsvendor model. Single-period stocking decisions often occur in practice; these require the decision maker to choose the stocking level of an item for which demand exists for only a single period. Several factors affect this decision: the distribution of demand, the cost and price of the item, the salvage value of the item, and the loss of customer goodwill due to stockouts. The newsvendor model addresses this problem and develops a formula for what is usually called the ‘critical-fractile’. The optimum order quantity is calculated using the critical-fractile of the distribution of the demand for the period. Underlying the mathematical simplicity of the critical-fractile formula is a powerful and intuitively appealing insight for the determination of the order quantity. The order quantity depends only on the optimum balance between two types of costs. The first is the cost per unit associated with the unavailability of stock to meet the manifest demand (i.e., underage cost). The second is the unit cost associated with excess inventory at the end of the period for which there is no demand (i.e., overage cost). Following Hadley and Whitin [17], we review the single-period newsvendor model and provide a broader interpretation to the structure of its solution. The notation, we use for the newsvendor model, is given in Table 1.
Y | Random variable for single-period demand |
f_{θ} (y) | Probability density function of single-period demand |
F_{θ} (y) | Probability distribution function of single-period demand |
θ | Parameter (in general, vector) |
c_{1} | Unit selling price |
c | Unit procurement cost, which is independent of the procured amount |
g | Unit salvage value for unsold items remaining at the end of the period |
c_{2} | Unit stockout penalty cost (over and above any lost profit) |
u | Variable representing the order quantity |
u* | Optimal order quantity |
Q(u) | Expected profit as a function of the order quantity |
c_{o} | Unit overage cost |
c_{u} | Unit underage cost |
Different versions of the problem may equivalently consider expected opportunity cost minimization or expected profit maximization. We examine the latter and write the expected profit as
(38)
Assuming that c_{1 }+ c_{2} > g (which is generally true in most situations), we can show that the expected profit function Q(u) is concave. We, therefore, set the first derivative equal to zero to find a maximizing solution. The value of u that maximizes (38) is the one that satisfies
(39)
Clearly the lost contribution margin (c_{1} - c) plus the stockout penalty (c_{2}) represents the ‘underage cost’. Similarly the item cost (c) minus the salvage value (g) represents the ‘overage cost’. If we refer to the underage cost as c_{u} and the overage cost as c_{o}, we may rewrite (39) as
(40)
We should choose the order quantity u^{*} such that the cumulative distribution function (cdf) of u^{*} equals the ratio of the underage cost to the sum of the underage and overage costs. A relatively high underage cost results in a higher order quantity, whereas a relatively high overage cost leads to a lower order quantity, as one would expect. If the single-period demand Y follows the exponential distribution (1) then
(41)
(42)
and
(43)
Parametric uncertainty. Consider the case when the parameter σ is unknown. Let X_{1} ≤... ≤ X_{n} be the past observations (order statistics) of single-period demands from the exponential distribution (1). Then
(44)
is a sufficient statistic for σ; S_{ }is distributed with
(45)
To find the best invariant decision rule we use the invariant embedding technique [12-16] to transform (41) to the form, which is depended only on the pivotal quantity V=S/σ and the ancillary factor h=u/S. In statistics, a pivotal quantity or pivot is a function of observations and unobservable parameters whose probability distribution does not depend on unknown parameters. Note that a pivotal quantity need not be a statistic-the function and its value can depend on parameters of the model, but its distribution must not. If it is a statistic, then it is known as an ancillary statistic.
Transformation of Q(u) is given by
(46)
We find the expected profit for the statistical decision u=h S as
(47)
where
(48)
The value of η that maximizes (47) is given by
(49)
Thus,
(50)
Comparison of decision rules. For comparison, consider the maximum likelihood decision rule that may be obtained from (42) as
(51)
where is the maximum likelihood estimator of σ,
(52)
Since and belong to the same class
(53)
it follows from the above that is inadmissible in relation to . If, say, c= 2, c_{1}=490, c_{2}=2, g=1 (in terms of money), and n=1, we have that
(54)
Thus, in this case, the use of leads to a reduction in the risk of about 7 % as compared with . The absolute risk will be proportional to σ and may be considerable.
Predictive inference. It will be noted that the predictive probability density function of the single-period demand Y, which is compatible with (38), is given by
(55)
Using (55), the predictive profit is determined as
(56)
which can be reduced to
(57)
It follows from (57) that
(58)
Thus, u^{BI} can be found immediately from (56) as
(59)
4. Conclusion
The technique proposed in this paper represents a simple and computationally attractive statistical method based on the constructive use of the invariance principle in mathematical statistics. The main advantage of this technique consists in that it allows one to eliminate unknown parameters from the problem and to use the previous and current data of observations for obtaining predictive inferences as completely as possible. We have illustrated the technique for the exponential distribution. Applications to other log-location-scale distributions could follow directly.
References