Bias approximations for likelihood-based estimators

Bias approximation has played an important rôle in statistical inference, and numerous bias calculation techniques have been proposed under different contexts. We provide a unified approach to approximating the bias of the maximum likelihood estimator and the l2 penalized likelihood estimator for both linear and nonlinear models, where the design variables are allowed to be random and the sample size can be a stopping time. The proposed method is based on the Woodroofe–Stein identity and is justified by very weak approximations. The accuracy of the derived bias formulas is assessed by simulation for several examples. The bias of the ridge estimator in high-dimensional settings is also discussed.