Rao–Blackwell theorem

Rao–Blackwell theorem This article needs additional citations for verification. Aiutaci a migliorare questo articolo aggiungendo citazioni a fonti affidabili. Il materiale non fornito può essere contestato e rimosso. Trova fonti: "Rao–Blackwell theorem" – news · newspapers · books · scholar · JSTOR (Maggio 2014) (Scopri come e quando rimuovere questo messaggio modello) Nelle statistiche, the Rao–Blackwell theorem, sometimes referred to as the Rao–Blackwell–Kolmogorov theorem, is a result which characterizes the transformation of an arbitrarily crude estimator into an estimator that is optimal by the mean-squared-error criterion or any of a variety of similar criteria.

The Rao–Blackwell theorem states that if g(X) is any kind of estimator of a parameter θ, then the conditional expectation of g(X) given T(X), where T is a sufficient statistic, is typically a better estimator of θ, and is never worse. Sometimes one can very easily construct a very crude estimator g(X), and then evaluate that conditional expected value to get an estimator that is in various senses optimal.

The theorem is named after Calyampudi Radhakrishna Rao and David Blackwell. The process of transforming an estimator using the Rao–Blackwell theorem is sometimes called Rao–Blackwellization. The transformed estimator is called the Rao–Blackwell estimator.[1][2][3] Contenuti 1 Definizioni 2 Il teorema 2.1 Mean-squared-error version 2.2 Convex loss generalization 3 Proprietà 4 Esempio 5 Idempotence 6 Completeness and Lehmann–Scheffé minimum variance 7 Guarda anche 8 Riferimenti 9 External links Definitions An estimator δ(X) is an observable random variable (cioè. a statistic) used for estimating some unobservable quantity. Per esempio, one may be unable to observe the average height of all male students at the University of X, but one may observe the heights of a random sample of 40 di loro. The average height of those 40—the "sample average"—may be used as an estimator of the unobservable "population average". A sufficient statistic T(X) is a statistic calculated from data X to estimate some parameter θ for which no other statistic which can be calculated from data X provides any additional information about θ. It is defined as an observable random variable such that the conditional probability distribution of all observable data X given T(X) does not depend on the unobservable parameter θ, such as the mean or standard deviation of the whole population from which the data X was taken. In the most frequently cited examples, il "unobservable" quantities are parameters that parametrize a known family of probability distributions according to which the data are distributed. In altre parole, a sufficient statistic T(X) for a parameter θ is a statistic such that the conditional distribution of the data X, given T(X), does not depend on the parameter θ. A Rao–Blackwell estimator δ1(X) of an unobservable quantity θ is the conditional expected value E(δ(X) | T(X)) of some estimator δ(X) given a sufficient statistic T(X). Call δ(X) il "original estimator" and δ1(X) il "improved estimator". It is important that the improved estimator be observable, cioè. that it does not depend on θ. Generally, the conditional expected value of one function of these data given another function of these data does depend on θ, but the very definition of sufficiency given above entails that this one does not. The mean squared error of an estimator is the expected value of the square of its deviation from the unobservable quantity being estimated of θ. The theorem Mean-squared-error version One case of Rao–Blackwell theorem states: The mean squared error of the Rao–Blackwell estimator does not exceed that of the original estimator.

In altre parole, {nome dell'operatore dello stile di visualizzazione {e} ((delta _{1}(X)-teta )^{2})leq operatorname {e} ((delta (X)-teta )^{2}).} The essential tools of the proof besides the definition above are the law of total expectation and the fact that for any random variable Y, e(Y2) cannot be less than [e(Y)]2. That inequality is a case of Jensen's inequality, although it may also be shown to follow instantly from the frequently mentioned fact that {displaystyle 0leq operatorname {Var} (Y)=nome operatore {e} ((Y-operatorname {e} (Y))^{2})=nome operatore {e} (Y^{2})-(nome operatore {e} (Y))^{2}.} Più precisamente, the mean square error of the Rao-Blackwell estimator has the following decomposition[4] {nome dell'operatore dello stile di visualizzazione {e} [(delta _{1}(X)-teta )^{2}]=nome operatore {e} [(delta (X)-teta )^{2}]-nome operatore {e} [nome operatore {Var} (delta (X)mid T(X))]} Da {nome dell'operatore dello stile di visualizzazione {e} [nome operatore {Var} (delta (X)mid T(X))]geq 0} , the Rao-Blackwell theorem immediately follows.

Convex loss generalization The more general version of the Rao–Blackwell theorem speaks of the "expected loss" or risk function: {nome dell'operatore dello stile di visualizzazione {e} (l(delta _{1}(X)))leq operatorname {e} (l(delta (X)))} dove il "loss function" L may be any convex function. If the loss function is twice-differentiable, as in the case for mean-squared-error, then we have the sharper inequality[4] {nome dell'operatore dello stile di visualizzazione {e} (l(delta (X)))-nome operatore {e} (l(delta _{1}(X)))geq {frac {1}{2}}nome operatore {e} _{T}sinistra[inf_{X}L''(X)nome operatore {Var} (delta (X)mid T)Giusto].} Properties The improved estimator is unbiased if and only if the original estimator is unbiased, as may be seen at once by using the law of total expectation. The theorem holds regardless of whether biased or unbiased estimators are used.

The theorem seems very weak: it says only that the Rao–Blackwell estimator is no worse than the original estimator. In pratica, però, the improvement is often enormous.[5] Example Phone calls arrive at a switchboard according to a Poisson process at an average rate of λ per minute. This rate is not observable, but the numbers X1, ..., Xn of phone calls that arrived during n successive one-minute periods are observed. It is desired to estimate the probability e−λ that the next one-minute period passes with no phone calls.

An extremely crude estimator of the desired probability is {displaystyle delta _{0}= sinistra{{inizio{matrice}1&{testo{Se}} X_{1}=0,\0&{testo{altrimenti,}}fine{matrice}}right.} cioè., it estimates this probability to be 1 if no phone calls arrived in the first minute and zero otherwise. Despite the apparent limitations of this estimator, the result given by its Rao–Blackwellization is a very good estimator.

The sum {stile di visualizzazione S_{n}=somma _{io=1}^{n}X_{io}=X_{1}+cdot +X_{n}} can be readily shown to be a sufficient statistic for λ, cioè., the conditional distribution of the data X1, ..., Xn, depends on λ only through this sum. Perciò, we find the Rao–Blackwell estimator {displaystyle delta _{1}=nome operatore {e} (delta _{0}mid S_{n}=s_{n}).} After doing some algebra we have {stile di visualizzazione {inizio{allineato}delta _{1}&=operatorname {e} sinistra(mathbf {1} _{{X_{1}=0}}{Bigg |}somma _{io=1}^{n}X_{io}=s_{n}Giusto)\&=Pleft(X_{1}=0{Bigg |}somma _{io=1}^{n}X_{io}=s_{n}Giusto)\&=Pleft(X_{1}=0,sum _{i=2}^{n}X_{io}=s_{n}Giusto)times Pleft(somma _{io=1}^{n}X_{io}=s_{n}Giusto)^{-1}\&=e^{-lambda }{frac {sinistra((n-1)lambda a destra)^{S_{n}}e^{-(n-1)lambda }}{S_{n}!}}times left({frac {(nlambda )^{S_{n}}e^{-nlambda }}{S_{n}!}}Giusto)^{-1}\&={frac {sinistra((n-1)lambda a destra)^{S_{n}}e^{-nlambda }}{S_{n}!}}volte {frac {S_{n}!}{(nlambda )^{S_{n}}e^{-nlambda }}}\&=left(1-{frac {1}{n}}Giusto)^{S_{n}}fine{allineato}}} Since the average number of calls arriving during the first n minutes is nλ, one might not be surprised if this estimator has a fairly high probability (if n is big) of being close to {stile di visualizzazione a sinistra(1-{1 over n}Giusto)^{nlambda }approx e^{-lambda }.} So δ1 is clearly a very much improved estimator of that last quantity. Infatti, since Sn is complete and δ0 is unbiased, δ1 is the unique minimum variance unbiased estimator by the Lehmann–Scheffé theorem.

Idempotence Rao–Blackwellization is an idempotent operation. Using it to improve the already improved estimator does not obtain a further improvement, but merely returns as its output the same improved estimator.

Completeness and Lehmann–Scheffé minimum variance If the conditioning statistic is both complete and sufficient, and the starting estimator is unbiased, then the Rao–Blackwell estimator is the unique "best unbiased estimator": see Lehmann–Scheffé theorem.

An example of an improvable Rao–Blackwell improvement, when using a minimal sufficient statistic that is not complete, was provided by Galili and Meilijson in 2016.[6] Permettere {stile di visualizzazione X_{1},ldot ,X_{n}} be a random sample from a scale-uniform distribution {displaystyle Xsim Uleft((1-K)teta ,(1+K)theta right),} with unknown mean {stile di visualizzazione E[X]=theta } and known design parameter {parente dello stile di visualizzazione (0,1)} . In the search for "best" possible unbiased estimators for {stile di visualizzazione theta ,} it is natural to consider {stile di visualizzazione X_{1}} as an initial (crude) unbiased estimator for {stile di visualizzazione theta } and then try to improve it. Da {stile di visualizzazione X_{1}} is not a function of {displaystyle T=left(X_{(1)},X_{(n)}Giusto)} , the minimal sufficient statistic for {stile di visualizzazione theta } (dove {stile di visualizzazione X_{(1)}=min(X_{io})} e {stile di visualizzazione X_{(n)}=max(X_{io})} ), it may be improved using the Rao–Blackwell theorem as follows: {stile di visualizzazione {cappello {teta }}_{RB}=E_{teta }sinistra[X_{1}|X_{(1)},X_{(n)}Giusto]={frac {X_{(1)}+X_{(n)}}{2}}.} Tuttavia, the following unbiased estimator can be shown to have lower variance: {stile di visualizzazione {cappello {teta }}_{LV}={frac {1}{2sinistra(k^{2}{frac {n-1}{n+1}}+1Giusto)}}sinistra[(1-K){{X}_{(1)}}+(1+K){{X}_{(n)}}Giusto].} And in fact, it could be even further improved when using the following estimator: {stile di visualizzazione {cappello {teta }}_{BAYES}={frac {n+1}{n}}sinistra[1-{frac {{frac {sinistra({frac {{X}_{(1)}}{1-K}}Giusto)}{sinistra({frac {{X}_{(n)}}{1+K}}Giusto)}}-1}{{{sinistra[{frac {sinistra({frac {{X}_{(1)}}{1-K}}Giusto)}{sinistra({frac {{X}_{(n)}}{1+K}}Giusto)}}Giusto]}^{n+1}}-1}}Giusto]{frac {X_{(n)}}{1+K}}} The model is a scale model. Optimal equivariant estimators can then be derived for loss functions that are invariant.[7] See also Basu's theorem — Another result on complete sufficient and ancillary statistics References ^ Blackwell, D. (1947). "Conditional expectation and unbiased sequential estimation". Annali di statistica matematica. 18 (1): 105–110. doi:10.1214/aoms/1177730497. SIG 0019903. Zbl 0033.07603. ^ Kolmogorov, UN. N. (1950). "Unbiased estimates". Izvestiya Akad. Nauk SSSR. Ser. Stuoia. 14: 303–326. SIG 0036479. ^ Rao, C. Radhakrishna (1945). "Information and accuracy attainable in the estimation of statistical parameters". Bulletin of the Calcutta Mathematical Society. 37 (3): 81–91. ^ Salta su: a b J. G. Liao & A. Berg (22 Giugno 2018). "Sharpening Jensen's Inequality". Lo statistico americano: 1–4. arXiv:1707.08644. doi:10.1080/00031305.2017.1419145. ^ Carpenter, Bob (Gennaio 20, 2020). "Rao-Blackwellization and discrete parameters in Stan". Statistical Modeling, Causal Inference, and Social Science. Retrieved September 13, 2021. The Rao-Blackwell theorem states that the marginalization approach has variance less than or equal to the direct approach. In pratica, this difference can be enormous. ^ Tal Galili & Isaac Meilijson (31 Mar 2016). "An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator". Lo statistico americano. 70 (1): 108–113. doi:10.1080/00031305.2015.1100683. PMC 4960505. PMID 27499547. ^ Taraldsen, Gunnar (2020). "Micha Mandel (2020), "The Scaled Uniform Model Revisited," Lo statistico americano, 74:1, 98–100: Commento". Lo statistico americano. 74 (3): 315–315. doi:10.1080/00031305.2020.1769727. ISSN 0003-1305. External links Nikulin, M.S. (2001) [1994], "Rao–Blackwell–Kolmogorov theorem", Enciclopedia della matematica, EMS Press hide vte Statistics OutlineIndex show Descriptive statistics show Data collection hide Statistical inference Statistical theory PopulationStatisticProbability distributionSampling distribution Order statisticEmpirical distribution Density estimationStatistical model Model specificationLp spaceParameter locationscaleshapeParametric family Likelihood (monotone)Location–scale familyExponential familyCompletenessSufficiencyStatistical functional BootstrapUVOptimal decision loss functionEfficiencyStatistical distance divergenceAsymptoticsRobustness Frequentist inference Point estimation Estimating equations Maximum likelihoodMethod of momentsM-estimatorMinimum distanceUnbiased estimators Mean-unbiased minimum-variance Rao–BlackwellizationLehmann–Scheffé theoremMedian unbiasedPlug-in Interval estimation Confidence intervalPivotLikelihood intervalPrediction intervalTolerance intervalResampling BootstrapJackknife Testing hypotheses 1- & 2-tailsPower Uniformly most powerful testPermutation test Randomization testMultiple comparisons Parametric tests Likelihood-ratioScore/Lagrange multiplierWald Specific tests Z-test (normal)Student's t-testF-test Goodness of fit Chi-squaredG-testKolmogorov–SmirnovAnderson–DarlingLillieforsJarque–BeraNormality (Shapiro–Wilk)Likelihood-ratio testModel selection Cross validationAICBIC Rank statistics Sign Sample medianSigned rank (Wilcoxon) Hodges–Lehmann estimatorRank sum (Mann–Whitney)Nonparametric anova 1-way (Kruskal–Wallis)2-modo (Friedman)Ordered alternative (Jonckheere–Terpstra)Van der Waerden test Bayesian inference Bayesian probability priorposteriorCredible intervalBayes factorBayesian estimator Maximum posterior estimator show CorrelationRegression analysis show Categorical / Multivariate / Time-series / Survival analysis show Applications Category Mathematics portalCommons WikiProject Categories: Theorems in statisticsEstimation theory

Se vuoi conoscere altri articoli simili a Rao–Blackwell theorem puoi visitare la categoria Estimation theory.

lascia un commento

L'indirizzo email non verrà pubblicato.

Vai su

Utilizziamo cookie propri e di terze parti per migliorare l'esperienza dell'utente Maggiori informazioni