The weight-decay technique in learning from data: an optimization point of view

The technique known as “weight decay” in the literature about learning from data is investigated using tools from regularization theory. Weight-decay regularization is compared with Tikhonov’s regularization of the learning problem and with a mixed regularized learning technique. The accuracies of suboptimal solutions to weight-decay learning are estimated for connectionistic models with a-priori fixed numbers of computational units. The Authors were partially supported by a PRIN grant from the Italian Ministry for University and Research, project “Models and Algorithms for Robust Network Optimization”.     

A simple construction of optimal estimation in multivariate marginal Cox regression

The multivariate extension of the Cox model proposed by Wei,Lin and Weissfeld in 1989 has been widely used for analyzing multivariate survival data.Under the model assumption,failure times from an individual are assumed to marginally follow their respective proportional hazards regression relation,leaving the joint distribution completely unspecified.This paper presents a simple approach to efficiency improvement through segmentation of stochastic integrals in the marginal estimating equations and incorporation of the limiting covariance structure.It is shown that when partition of the time interval is done at a suitable rate,the resulting estimator is consistent and asymptotically normal.Through the reproducing kernel Hilbert space arising from the covariance function of the limiting Gaussian process,it is also shown that the proposed estimator is asymptotically optimal within a reasonable class of estimators under marginal specification.Simulations are conducted to assess the finite-sample performance of the proposed method.