An augmented Lagrangian approach with a variable transformation in nonlinear programming

Tangent cone and (regular) normal cone of a closed set under an invertible variable transformation around a given point are investigated, which lead to the concepts of θ⁻¹-tangent cone of a set and θ⁻¹-subderivative of a function. When the notion of θ⁻¹-subderivative is applied to perturbation functions, a class of augmented Lagrangians involving an invertible mapping of perturbation variables are obtained, in which dualizing parameterization and augmenting functions are not necessarily convex in perturbation variables. A necessary and sufficient condition for the exact penalty representation under the proposed augmented Lagrangian scheme is obtained. For an augmenting function with an Euclidean norm, a sufficient condition (resp., a sufficient and necessary condition) for an arbitrary vector (resp., 0) to support an exact penalty representation is given in terms of θ⁻¹-subderivatives. An example of the variable transformation applied to constrained optimization problems is given, which yields several exact penalization results in the literature.