Robust Aggregation for Federated Learning by Minimum γ-Divergence Estimation

Federated learning is a framework for multiple devices or institutions, called local clients, to collaboratively train a global model without sharing their data. For federated learning with a central server, an aggregation algorithm integrates model information sent from local clients to update the parameters for a global model. Sample mean is the simplest and most commonly used aggregation method. However, it is not robust for data with outliers or under the Byzantine problem, where Byzantine clients send malicious messages to interfere with the learning process. Some robust aggregation methods were introduced in literature including marginal median, geometric median and trimmed-mean. In this article, we propose an alternative robust aggregation method, named γ-mean, which is the minimum divergence estimation based on a robust density power divergence. This γ-mean aggregation mitigates the influence of Byzantine clients by assigning fewer weights. This weighting scheme is data-driven and controlled by the γ value. Robustness from the viewpoint of the influence function is discussed and some numerical results are presented.     

On the α-q-Mutual Information and the α-q-Capacities

The measures of information transfer which correspond to non-additive entropies have intensively been studied in previous decades. The majority of the work includes the ones belonging to the Sharma–Mittal entropy class, such as the Rényi, the Tsallis, the Landsberg–Vedral and the Gaussian entropies. All of the considerations follow the same approach, mimicking some of the various and mutually equivalent definitions of Shannon information measures, and the information transfer is quantified by an appropriately defined measure of mutual information, while the maximal information transfer is considered as a generalized channel capacity. However, all of the previous approaches fail to satisfy at least one of the ineluctable properties which a measure of (maximal) information transfer should satisfy, leading to counterintuitive conclusions and predicting nonphysical behavior even in the case of very simple communication channels. This paper fills the gap by proposing two parameter measures named the α-q-mutual information and the α-q-capacity. In addition to standard Shannon approaches, special cases of these measures include the α-mutual information and the α-capacity, which are well established in the information theory literature as measures of additive Rényi information transfer, while the cases of the Tsallis, the Landsberg–Vedral and the Gaussian entropies can also be accessed by special choices of the parameters α and q. It is shown that, unlike the previous definition, the α-q-mutual information and the α-q-capacity satisfy the set of properties, which are stated as axioms, by which they reduce to zero in the case of totally destructive channels and to the (maximal) input Sharma–Mittal entropy in the case of perfect transmission, which is consistent with the maximum likelihood detection error. In addition, they are non-negative and less than or equal to the input and the output Sharma–Mittal entropies, in general. Thus, unlike the previous approaches, the proposed (maximal) information transfer measures do not manifest nonphysical behaviors such as sub-capacitance or super-capacitance, which could qualify them as appropriate measures of the Sharma–Mittal information transfer.     

A Quantitative Comparison between Shannon and Tsallis–Havrda–Charvat Entropies Applied to Cancer Outcome Prediction

In this paper, we propose to quantitatively compare loss functions based on parameterized Tsallis–Havrda–Charvat entropy and classical Shannon entropy for the training of a deep network in the case of small datasets which are usually encountered in medical applications. Shannon cross-entropy is widely used as a loss function for most neural networks applied to the segmentation, classification and detection of images. Shannon entropy is a particular case of Tsallis–Havrda–Charvat entropy. In this work, we compare these two entropies through a medical application for predicting recurrence in patients with head–neck and lung cancers after treatment. Based on both CT images and patient information, a multitask deep neural network is proposed to perform a recurrence prediction task using cross-entropy as a loss function and an image reconstruction task. Tsallis–Havrda–Charvat cross-entropy is a parameterized cross-entropy with the parameter α . Shannon entropy is a particular case of Tsallis–Havrda–Charvat entropy for α=1 . The influence of this parameter on the final prediction results is studied. In this paper, the experiments are conducted on two datasets including in total 580 patients, of whom 434 suffered from head–neck cancers and 146 from lung cancers. The results show that Tsallis–Havrda–Charvat entropy can achieve better performance in terms of prediction accuracy with some values of α .     

A Novel Algebraic Structure of (α,β)-Complex Fuzzy Subgroups

A complex fuzzy set is a vigorous framework to characterize novel machine learning algorithms. This set is more suitable and flexible compared to fuzzy sets, intuitionistic fuzzy sets, and bipolar fuzzy sets. On the aspects of complex fuzzy sets, we initiate the abstraction of (α,β)-complex fuzzy sets and then define -complex fuzzy subgroups. Furthermore, we prove that every complex fuzzy subgroup is an (α,β)-complex fuzzy subgroup and define (α,β)-complex fuzzy normal subgroups of given group. We extend this ideology to define (α,β)-complex fuzzy cosets and analyze some of their algebraic characteristics. Furthermore, we prove that (α,β)-complex fuzzy normal subgroup is constant in the conjugate classes of group. We present an alternative conceptualization of (α,β)-complex fuzzy normal subgroup in the sense of the commutator of groups. We establish the (α,β)-complex fuzzy subgroup of the classical quotient group and show that the set of all (α,β)-complex fuzzy cosets of this specific complex fuzzy normal subgroup form a group. Additionally, we expound the index of -complex fuzzy subgroups and investigate the (α,β)-complex fuzzification of Lagrange’s theorem analog to Lagrange’ theorem of classical group theory.     

Information Measures for Generalized Order Statistics and Their Concomitants under General Framework from Huang-Kotz FGM Bivariate Distribution

In this paper, we study the concomitants of dual generalized order statistics (and consequently generalized order statistics) when the parameters γ1,…,γn are assumed to be pairwise different from Huang–Kotz Farlie–Gumble–Morgenstern bivariate distribution. Some useful recurrence relations between single and product moments of concomitants are obtained. Moreover, Shannon’s entropy and the Fisher information number measures are derived. Finally, these measures are extensively studied for some well-known distributions such as exponential, Pareto and power distributions. The main motivation of the study of the concomitants of generalized order statistics (as an important practical kind to order the bivariate data) under this general framework is to enable researchers in different fields of statistics to use some of the important models contained in these generalized order statistics only under this general framework. These extended models are frequently used in the reliability theory, such as the progressive type-II censored order statistics.     

Changing the Geometry of Representations: α-Embeddings for NLP Tasks

Word embeddings based on a conditional model are commonly used in Natural Language Processing (NLP) tasks to embed the words of a dictionary in a low dimensional linear space. Their computation is based on the maximization of the likelihood of a conditional probability distribution for each word of the dictionary. These distributions form a Riemannian statistical manifold, where word embeddings can be interpreted as vectors in the tangent space of a specific reference measure on the manifold. A novel family of word embeddings, called α-embeddings have been recently introduced as deriving from the geometrical deformation of the simplex of probabilities through a parameter α, using notions from Information Geometry. After introducing the α-embeddings, we show how the deformation of the simplex, controlled by α, provides an extra handle to increase the performances of several intrinsic and extrinsic tasks in NLP. We test the α-embeddings on different tasks with models of increasing complexity, showing that the advantages associated with the use of α-embeddings are present also for models with a large number of parameters. Finally, we show that tuning α allows for higher performances compared to the use of larger models in which additionally a transformation of the embeddings is learned during training, as experimentally verified in attention models.     

Novel plasma biomarker surrogating cerebral amyloid deposition

Alzheimer’s disease (AD) is the most common and devastating dementia. Simple and practical biomarkers for AD are urgently required for accurate diagnosis and to facilitate the development of disease-modifying interventions. The subjects for the study were selected on the basis of PiB amyloid imaging by PET. Forty PiB-positive (PiB+) individuals, including cognitively healthy controls (HC), and mild cognitive impairment and AD individuals, and 22 PiB-negative (PiB−) HC participated. Employing our novel highly sensitive immunoprecipitation-mass spectrometry, we measured plasma amyloid β-proteins (Aβs; Aβ1-40 and Aβ1-42) and Aβ-approximate peptides (AβAPs), which were cleaved from amyloid precursor protein (APP). Among the AβAPs, APP669-711 appeared to be a good reference for deciphering pathological change of Aβ1-42. We evaluated the performance of the ratio of APP669-711 to Aβ1-42 (APP669-711/Aβ1-42) as a biomarker. APP669-711/Aβ1-42 significantly increased in the PiB+ groups. The sensitivity and specificity to discriminate PiB+ individuals from PiB− individuals were 0.925 and 0.955, respectively. Our plasma biomarker precisely surrogates cerebral amyloid deposition.     

RET receptor signaling: Function in development,metabolic disease,and cancer

The RET proto-oncogene encodes a receptor tyrosine kinase whose alterations are responsible for various human cancers and developmental disorders, including thyroid cancer, non-small cell lung cancer, multiple endocrine neoplasia type 2, and Hirschsprung’s disease. RET receptors are physiologically activated by glial cell line-derived neurotrophic factor (GDNF) family ligands that bind to the coreceptor GDNF family receptor α (GFRα). Signaling via the GDNF/GFRα1/RET ternary complex plays crucial roles in the development of the enteric nervous system, kidneys, and urinary tract, as well as in the self-renewal of spermatogonial stem cells. In addition, another ligand, growth differentiation factor-15 (GDF15), has been shown to bind to GFRα-like and activate RET, regulating body weight. GDF15 is a stress response cytokine, and its elevated serum levels affect metabolism and anorexia-cachexia syndrome. Moreover, recent development of RET-specific kinase inhibitors contributed significantly to progress in the treatment of patients with RET-altered cancer. This review focuses on the broad roles of RET in development, metabolic diseases, and cancer.     

A Statistical Journey through the Topological Determinants of the β2 Adrenergic Receptor Dynamics

Activation of G-protein-coupled receptors (GPCRs) is mediated by molecular switches throughout the transmembrane region of the receptor. In this work, we continued along the path of a previous computational study wherein energy transport in the β2 Adrenergic Receptor (β2-AR) was examined and allosteric switches were identified in the molecular structure through the reorganization of energy transport networks during activation. In this work, we further investigated the allosteric properties of β2-AR, using Protein Contact Networks (PCNs). In this paper, we report an extensive statistical analysis of the topological and structural properties of β2-AR along its molecular dynamics trajectory to identify the activation pattern of this molecular system. The results show a distinct character to the activation that both helps to understand the allosteric switching previously identified and confirms the relevance of the network formalism to uncover relevant functional features of protein molecules.     

Information–Theoretic Aspects of Location Parameter Estimation under Skew–Normal Settings

In several applications, the assumption of normality is often violated in data with some level of skewness, so skewness affects the mean’s estimation. The class of skew–normal distributions is considered, given their flexibility for modeling data with asymmetry parameter. In this paper, we considered two location parameter (μ) estimation methods in the skew–normal setting, where the coefficient of variation and the skewness parameter are known. Specifically, the least square estimator (LSE) and the best unbiased estimator (BUE) for μ are considered. The properties for BUE (which dominates LSE) using classic theorems of information theory are explored, which provides a way to measure the uncertainty of location parameter estimations. Specifically, inequalities based on convexity property enable obtaining lower and upper bounds for differential entropy and Fisher information. Some simulations illustrate the behavior of differential entropy and Fisher information bounds.     

Challenge to the suppression of tumor growth by the β4-galactosyltransferase genes

It has been well established that structural changes in glycans attached to proteins and lipids are associated with malignant transformation of cells. We focused on galactose residues among the sugars since they are involved in the galectin-mediated biology, and many carbohydrate antigens are frequently expressed on this sugar. We found changes in the expression of the β4-galactosyltransferase (β4GalT) 2 and 5 genes in cancer cells: decreased expression of the β4GalT2 gene and increased expression of the β4GalT5 gene. The growth of mouse melanoma cells showing enhanced expression of the β4GalT2 gene or reduced expression of the β4GalT5 gene is inhibited remarkably in syngeneic mice. Tumor growth inhibition is probably caused by the induction of apoptosis, inhibition of angiogenesis, and/or reduced MAPK signals. Direct transduction of human β4GalT2 cDNA together with the adenovirus vector into human hepatocellular carcinoma cells grown in SCID mice results in marked growth retardation of the tumors. β4GalT gene-transfer appears to be a potential tool for cancer therapy.     

The biological significance of ω-oxidation of fatty acids

The author focuses on the biological significance of ω-oxidation of fatty acids. Early studies revealed that there is a subsidiary pathway for β-oxidation of fatty acids when β-oxidation is blocked. Many studies demonstrated that the ω-oxidation serves to provide succinyl-CoA for the citric acid cycle and for gluconeogenesis under conditions of starvation and diabetes. Acylglucosylceramides which are composed of linoleic acid, long chain ω-hydroxy fatty acids, eicosasphingenine (or trihydroxyeicosasphingenine) and glucose, are responsible for normal epidermal permeability function in the skin. It is observed that ω- and (ω-1)-oxidation of fatty acids are related to energy metabolism in some laboratory animals such as musk shrews and Mongolian gerbils. Studies confirmed that ω- and (ω-1)-oxidation of fatty acids play crucial roles in the production of insect pheromones of honeybees and in the formation of biopolyesters of higher plants. In addition, the biological significance of ω-oxidation of prostaglandins and leukotrienes is described.     

Okamoto model for necrosis and its expansions,CD38-cyclic ADP-ribose signal system for intracellular Ca2+ mobilization and Reg (Regenerating gene protein)-Reg receptor system for cell regeneration

In pancreatic islet cell culture models and animal models, we studied the molecular mechanisms involved in the development of insulin-dependent diabetes. The diabetogenic agents, alloxan and streptozotocin, caused DNA strand breaks, which in turn activated poly(ADP-ribose) polymerase/synthetase (PARP) to deplete NAD⁺, thereby inhibiting islet β-cell functions such as proinsulin synthesis and ultimately leading to β-cell necrosis. Radical scavengers protected against the formation of DNA strand breaks and inhibition of proinsulin synthesis. Inhibitors of PARP prevented the NAD⁺ depletion, inhibition of proinsulin synthesis and β-cell death. These findings led to the proposed unifying concept for β-cell damage and its prevention (the Okamoto model). The model met one proof with PARP knockout animals and was further extended by the discovery of cyclic ADP-ribose as the second messenger for Ca²⁺ mobilization in glucose-induced insulin secretion and by the identification of Reg (Regenerating gene) for β-cell regeneration. Physiological and pathological events found in pancreatic β-cells have been observed in other cells and tissues.     

Error Exponents and α-Mutual Information

Over the last six decades, the representation of error exponent functions for data transmission through noisy channels at rates below capacity has seen three distinct approaches: (1) Through Gallager’s E0 functions (with and without cost constraints); (2) large deviations form, in terms of conditional relative entropy and mutual information; (3) through the α-mutual information and the Augustin–Csiszár mutual information of order α derived from the Rényi divergence. While a fairly complete picture has emerged in the absence of cost constraints, there have remained gaps in the interrelationships between the three approaches in the general case of cost-constrained encoding. Furthermore, no systematic approach has been proposed to solve the attendant optimization problems by exploiting the specific structure of the information functions. This paper closes those gaps and proposes a simple method to maximize Augustin–Csiszár mutual information of order α under cost constraints by means of the maximization of the α-mutual information subject to an exponential average constraint.     

Identification and quantification of amyloid beta-related peptides in human plasma using matrix-assisted laser desorption/ionization time-of-flight mass spectrometry

Proteolytic processing of the amyloid precursor protein (APP) by β-secretase and γ-secretase leads to the generation and deposition of amyloid β (Aβ) in Alzheimer’s disease (AD). N-terminally or C-terminally truncated Aβ variants have been found in human cerebrospinal fluid and cultured cell media using immunoprecipitation and mass spectrometry. Unfortunately, the profile of plasma Aβ variants has not been revealed due to the difficulty of isolating Aβ from plasma. We present here for the first time studies of Aβ and related peptides in human plasma. Twenty-two Aβ-related peptides including novel peptides truncated before the β-secretase site were detected in human plasma and 20 of the peptides were identified by tandem mass spectrometry. Using an internal standard, we developed a quantitative assay for the Aβ-related peptides and demonstrated plasma dilution linearity and the precision required for their quantitation. The present method should enhance the understanding of APP processing and clearance in AD progression.     

λ-Deformation: A Canonical Framework for Statistical Manifolds of Constant Curvature

This paper systematically presents the λ-deformation as the canonical framework of deformation to the dually flat (Hessian) geometry, which has been well established in information geometry. We show that, based on deforming the Legendre duality, all objects in the Hessian case have their correspondence in the λ-deformed case: λ-convexity, λ-conjugation, λ-biorthogonality, λ-logarithmic divergence, λ-exponential and λ-mixture families, etc. In particular, λ-deformation unifies Tsallis and Rényi deformations by relating them to two manifestations of an identical λ-exponential family, under subtractive or divisive probability normalization, respectively. Unlike the different Hessian geometries of the exponential and mixture families, the λ-exponential family, in turn, coincides with the λ-mixture family after a change of random variables. The resulting statistical manifolds, while still carrying a dualistic structure, replace the Hessian metric and a pair of dually flat conjugate affine connections with a conformal Hessian metric and a pair of projectively flat connections carrying constant (nonzero) curvature. Thus, λ-deformation is a canonical framework in generalizing the well-known dually flat Hessian structure of information geometry.     

ϕ-Informational Measures: Some Results and Interrelations

In this paper, we focus on extended informational measures based on a convex function ϕ: entropies, extended Fisher information, and generalized moments. Both the generalization of the Fisher information and the moments rely on the definition of an escort distribution linked to the (entropic) functional ϕ. We revisit the usual maximum entropy principle—more precisely its inverse problem, starting from the distribution and constraints, which leads to the introduction of state-dependent ϕ-entropies. Then, we examine interrelations between the extended informational measures and generalize relationships such the Cramér–Rao inequality and the de Bruijn identity in this broader context. In this particular framework, the maximum entropy distributions play a central role. Of course, all the results derived in the paper include the usual ones as special cases.     

Metabolism of amyloid β peptide and pathogenesis of Alzheimer’s disease

The conversion of what has been interpreted as “normal brain aging” to Alzheimer’s disease (AD) via transition states, i.e., preclinical AD and mild cognitive impairment, appears to be a continuous process caused primarily by aging-dependent accumulation of amyloid β peptide (Aβ) in the brain. This notion however gives us a hope that, by manipulating the Aβ levels in the brain, we may be able not only to prevent and cure the disease but also to partially control some very significant aspects of brain aging. Aβ is constantly produced from its precursor and immediately catabolized under normal conditions, whereas dysmetabolism of Aβ seems to lead to pathological deposition upon aging. We have focused our attention on elucidation of the unresolved mechanism of Aβ catabolism in the brain. In this review, I describe a new approach to prevent AD development by reducing Aβ burdens in aging brains through up-regulation of the catabolic mechanism involving neprilysin that can degrade both monomeric and oligomeric forms of Aβ. The strategy of combining presymptomatic diagnosis with preventive medicine seems to be the most pragmatic in both medical and socioeconomical terms.     

Kullback–Leibler Divergence of a Freely Cooling Granular Gas

Finding the proper entropy-like Lyapunov functional associated with the inelastic Boltzmann equation for an isolated freely cooling granular gas is a still unsolved challenge. The original H-theorem hypotheses do not fit here and the H-functional presents some additional measure problems that are solved by the Kullback–Leibler divergence (KLD) of a reference velocity distribution function from the actual distribution. The right choice of the reference distribution in the KLD is crucial for the latter to qualify or not as a Lyapunov functional, the asymptotic “homogeneous cooling state” (HCS) distribution being a potential candidate. Due to the lack of a formal proof far from the quasielastic limit, the aim of this work is to support this conjecture aided by molecular dynamics simulations of inelastic hard disks and spheres in a wide range of values for the coefficient of restitution (α) and for different initial conditions. Our results reject the Maxwellian distribution as a possible reference, whereas they reinforce the HCS one. Moreover, the KLD is used to measure the amount of information lost on using the former rather than the latter, revealing a non-monotonic dependence with α.     

Differential Geometric Aspects of Parametric Estimation Theory for States on Finite-Dimensional C∗-Algebras

A geometrical formulation of estimation theory for finite-dimensional C∗-algebras is presented. This formulation allows to deal with the classical and quantum case in a single, unifying mathematical framework. The derivation of the Cramer–Rao and Helstrom bounds for parametric statistical models with discrete and finite outcome spaces is presented.