Learning from Scarce Information: Using Synthetic Data to Classify Roman Fine Ware Pottery

In this article, we consider a version of the challenging problem of learning from datasets whose size is too limited to allow generalisation beyond the training set. To address the challenge, we propose to use a transfer learning approach whereby the model is first trained on a synthetic dataset replicating features of the original objects. In this study, the objects were smartphone photographs of near-complete Roman terra sigillata pottery vessels from the collection of the Museum of London. Taking the replicated features from published profile drawings of pottery forms allowed the integration of expert knowledge into the process through our synthetic data generator. After this first initial training the model was fine-tuned with data from photographs of real vessels. We show, through exhaustive experiments across several popular deep learning architectures, different test priors, and considering the impact of the photograph viewpoint and excessive damage to the vessels, that the proposed hybrid approach enables the creation of classifiers with appropriate generalisation performance. This performance is significantly better than that of classifiers trained exclusively on the original data, which shows the promise of the approach to alleviate the fundamental issue of learning from small datasets.     

On optimal neural network learning

We introduce optimal learning with a neural network, which we define as building a network with a minimal expectation generalisation error. This procedure may be analysed exactly in idealized problems by exploiting the relationship between sampling a space of hypotheses and the replica method of statistical physics. We find that the optimally trained spherical perceptron may learn a linearly separable rule as well as any possible computer, and present simulation results supporting our conclusions. Optimal learning of a well-known unlearnable problem, the “mismatched weight” problem, gives better asymptotic learning than conventional techniques, and may be simulated more easily. Unlike many other perceptron learning schemes, optimal learning extends to more general networks learning more complex rules.     

Exponential speedup of quantum newton optimization algorithm for general polynomials

Quantum optimization algorithms can outperform their classical counterpart and are key in modern technology. The second-order optimization algorithm(the Newton algorithm) is a critical optimization method, speeding up the convergence by employing the second-order derivative of loss functions in addition to their first derivative. Here, we propose a new quantum second-order optimization algorithm for general polynomials with a computational complexity of O(poly(log d)). We use this algorithm to solve the nonlinear equation and learning parameter problems in factorization machines. Numerical simulations show that our new algorithm is faster than its classical counterpart and the first-order quantum gradient descent algorithm. While existing quantum Newton optimization algorithms apply only to homogeneous polynomials, our new algorithm can be used in the case of general polynomials, which are more widely present in real applications.     

PP-waves with torsion: a metric-affine model for the massless neutrino

In this paper we deal with quadratic metric-affine gravity, which we briefly introduce, explain and give historical and physical reasons for using this particular theory of gravity. We then introduce a generalisation of well known spacetimes, namely pp-waves. A classical pp-wave is a 4-dimensional Lorentzian spacetime which admits a nonvanishing parallel spinor field; here the connection is assumed to be Levi-Civita. This definition was generalised in our previous work to metric compatible spacetimes with torsion and used to construct new explicit vacuum solutions of quadratic metric-affine gravity, namely generalised pp-waves of parallel Ricci curvature. The physical interpretation of these solutions we propose in this article is that they represent a conformally invariant metric-affine model for a massless elementary particle. We give a comparison with the classical model describing the interaction of gravitational and massless neutrino fields, namely Einstein–Weyl theory and construct pp-wave type solutions of this theory. We point out that generalised pp-waves of parallel Ricci curvature are very similar to pp-wave type solutions of the Einstein–Weyl model and therefore propose that our generalised pp-waves of parallel Ricci curvature represent a metric-affine model for the massless neutrino.     

First order phase transitions in unbounded spin systemsI: Construction of the phase diagram

The phase diagram and the corresponding infinite volume Gibbs states are constructed for a large class of continuous, unbounded spin models. Our construction relies on a partition of unity mapping our system onto an interacting contour system, a generalisation of Zahradnik's approach to Piragov Sinai theory to interacting contour systems, and a suitable mean field expansion around the minimas of the Hamiltonian.     

Discrete Infomax Codes for Supervised Representation Learning

For high-dimensional data such as images, learning an encoder that can output a compact yet informative representation is a key task on its own, in addition to facilitating subsequent processing of data. We present a model that produces discrete infomax codes (DIMCO); we train a probabilistic encoder that yields k-way d-dimensional codes associated with input data. Our model maximizes the mutual information between codes and ground-truth class labels, with a regularization which encourages entries of a codeword to be statistically independent. In this context, we show that the infomax principle also justifies existing loss functions, such as cross-entropy as its special cases. Our analysis also shows that using shorter codes reduces overfitting in the context of few-shot classification, and our various experiments show this implicit task-level regularization effect of DIMCO. Furthermore, we show that the codes learned by DIMCO are efficient in terms of both memory and retrieval time compared to prior methods.     

Could there be a fourth generation of quarks without more leptons?

We investigate the possibility of adding a fourth generation of quarks. We also extend the Standard Model gauge group by adding another SU(N) component. In order to cancel the contributions of the fourth generation of quarks to the gauge anomalies we must add a generation of fermions coupling to the SU(N) group. This model has many features similar to the Standard Model and, for example, includes a natural generalisation of the Standard Model charge quantisation rule. We discuss the phenomenology of this model and, in particular, show that the infrared quasi-fixed point values of the Yukawa coupling constants put upper limits on the new quark masses close to the present experimental lower bounds.     

Bottleneck Problems: An Information and Estimation-Theoretic View

Information bottleneck (IB) and privacy funnel (PF) are two closely related optimization problems which have found applications in machine learning, design of privacy algorithms, capacity problems (e.g., Mrs. Gerber’s Lemma), and strong data processing inequalities, among others. In this work, we first investigate the functional properties of IB and PF through a unified theoretical framework. We then connect them to three information-theoretic coding problems, namely hypothesis testing against independence, noisy source coding, and dependence dilution. Leveraging these connections, we prove a new cardinality bound on the auxiliary variable in IB, making its computation more tractable for discrete random variables. In the second part, we introduce a general family of optimization problems, termed “bottleneck problems”, by replacing mutual information in IB and PF with other notions of mutual information, namely f-information and Arimoto’s mutual information. We then argue that, unlike IB and PF, these problems lead to easily interpretable guarantees in a variety of inference tasks with statistical constraints on accuracy and privacy. While the underlying optimization problems are non-convex, we develop a technique to evaluate bottleneck problems in closed form by equivalently expressing them in terms of lower convex or upper concave envelope of certain functions. By applying this technique to a binary case, we derive closed form expressions for several bottleneck problems.     

A Manifold Learning Perspective on Representation Learning: Learning Decoder and Representations without an Encoder

Autoencoders are commonly used in representation learning. They consist of an encoder and a decoder, which provide a straightforward method to map n-dimensional data in input space to a lower m-dimensional representation space and back. The decoder itself defines an m-dimensional manifold in input space. Inspired by manifold learning, we showed that the decoder can be trained on its own by learning the representations of the training samples along with the decoder weights using gradient descent. A sum-of-squares loss then corresponds to optimizing the manifold to have the smallest Euclidean distance to the training samples, and similarly for other loss functions. We derived expressions for the number of samples needed to specify the encoder and decoder and showed that the decoder generally requires much fewer training samples to be well-specified compared to the encoder. We discuss the training of autoencoders in this perspective and relate it to previous work in the field that uses noisy training examples and other types of regularization. On the natural image data sets MNIST and CIFAR10, we demonstrated that the decoder is much better suited to learn a low-dimensional representation, especially when trained on small data sets. Using simulated gene regulatory data, we further showed that the decoder alone leads to better generalization and meaningful representations. Our approach of training the decoder alone facilitates representation learning even on small data sets and can lead to improved training of autoencoders. We hope that the simple analyses presented will also contribute to an improved conceptual understanding of representation learning.     

Does a varying speed of light solve the cosmological problems?

We propose a new generalisation of general relativity which incorporates a variation in both the speed of light in vacuum (c) and the gravitational constant (G) and which is both covariant and Lorentz invariant. We solve the generalised Einstein equations for Friedmann universes and show that arbitrary time-variations of c and G never lead to a solution to the flatness, horizon or Λ problems for a theory satisfying the strong energy condition. In order to do so, one needs to construct a theory which does not reduce to the standard one for any choice of time, length and energy units. This can be achieved by breaking a number of invariance principles such as covariance and Lorentz invariance.     

Symmetry conserving generalisation of Hartree Fock Bogoliubov theory

We derive a generalisation of the HFB equations which conserve particle number. This is achieved in using the equation of motion method or alternatively the Green's function technique. The price we have to pay is that there is not only one mean field for the particle numberN but a set of coupled mean field equations for the whole bandN, N±2,N±4... Nevertheless we think that our theory is a quite interesting variant in comparison with the conventional projection technique. We apply our theory to simple models and find that the results are excellent.     

Multiscale finite element methods for high-contrast problems using local spectral basis functions

In this paper we study multiscale finite element methods (MsFEMs) using spectral multiscale basis functions that are designed for high-contrast problems. Multiscale basis functions are constructed using eigenvectors of a carefully selected local spectral problem. This local spectral problem strongly depends on the choice of initial partition of unity functions. The resulting space enriches the initial multiscale space using eigenvectors of local spectral problem. The eigenvectors corresponding to small, asymptotically vanishing, eigenvalues detect important features of the solutions that are not captured by initial multiscale basis functions. Multiscale basis functions are constructed such that they span these eigenfunctions that correspond to small, asymptotically vanishing, eigenvalues. We present a convergence study that shows that the convergence rate (in energy norm) is proportional to (H/Λ₁)^1/2, where Λ₁ is proportional to the minimum of the eigenvalues that the corresponding eigenvectors are not included in the coarse space. Thus, we would like to reach to a larger eigenvalue with a smaller coarse space. This is accomplished with a careful choice of initial multiscale basis functions and the setup of the eigenvalue problems. Numerical results are presented to back-up our theoretical results and to show higher accuracy of MsFEMs with spectral multiscale basis functions. We also present a hierarchical construction of the eigenvectors that provides CPU savings.     

Self-adjoint Operators as Functions I

Observables of a quantum system, described by self-adjoint operators in a von Neumann algebra or affiliated with it in the unbounded case, form a conditionally complete lattice when equipped with the spectral order. Using this order-theoretic structure, we develop a new perspective on quantum observables. In this first paper (of two), we show that self-adjoint operators affiliated with a von Neumann algebra ${\mathcal{N}}$ can equivalently be described as certain real-valued functions on the projection lattice ${\mathcal{P}(\mathcal{N}})$ of the algebra, which we call q-observable functions. Bounded self-adjoint operators correspond to q-observable functions with compact image on non-zero projections. These functions, originally defined in a similar form by de Groote (Observables II: quantum observables, 2005), are most naturally seen as adjoints (in the categorical sense) of spectral families. We show how they relate to the daseinisation mapping from the topos approach to quantum theory (Döring and Isham , New Structures for Physics, Springer, Heidelberg, 2011). Moreover, the q-observable functions form a conditionally complete lattice which is shown to be order-isomorphic to the lattice of self-adjoint operators with respect to the spectral order. In a subsequent paper (Döring and Dewitt, 2012, preprint), we will give an interpretation of q-observable functions in terms of quantum probability theory, and using results from the topos approach to quantum theory, we will provide a joint sample space for all quantum observables.     

A class of discontinuous Petrov–Galerkin methods. Part IV: The optimal test norm and time-harmonic wave propagation in 1D

The phase error, or the pollution effect in the finite element solution of wave propagation problems, is a well known phenomenon that must be confronted when solving problems in the high-frequency range. This paper presents a new method with no phase errors for one-dimensional (1D) time-harmonic wave propagation problems using new ideas that hold promise for the multidimensional case. The method is constructed within the framework of the discontinuous Petrov–Galerkin (DPG) method with optimal test functions. We have previously shown that such methods select solutions that are the best possible approximations in an energy norm dual to any selected test space norm. In this paper, we advance by asking what is the optimal test space norm that achieves error reduction in a given energy norm. This is answered in the specific case of the Helmholtz equation with L²-norm as the energy norm. We obtain uniform stability with respect to the wave number. We illustrate the method with a number of 1D numerical experiments, using discontinuous piecewise polynomial hp spaces for the trial space and its corresponding optimal test functions computed approximately and locally. A 1D theoretical stability analysis is also developed.     

Dobrushin's compactness criterion for euclidean gibbs measures

We prove existence and uniform á priori estimates for Euclidean Gibbs measures corresponding to certain quantum systems with unbounded spins, pair potentials of superquadratic growth, and infinite radius of interaction. The quantum particles are indexed by the elements of a countable, possibly irregular, set L ⊂ ∝^d. We use Dobrushin's criterion and give a direct construction of appropriate compact functions on (infinite dimensional) loop spaces. For the quantum systems on L := ∝^d, with the superquadratic interactions of finite range, a new uniqueness result is established by means of the Dobrushin-Pechersky criterion.     

Linear response of a two-layer electron gas II

In this paper we present some new theoretical results from our study of a two-layer electron gas separated by a variable distance, based on a model recently proposed by us (referred to as the CD model) and also beyond that model. In a recent communication we derived analytical results for the optical and acoustic plasmon modes in the presence of electron impurity collisions on the layers and briefly presented an energy loss function. The present paper contains results of our further exploration of the CD model and a detailed consideration of the energy loss functions. We present plots for the loss functions and discuss the prospects of experimentally observing the elusive acoustic plasmon.     

The Kadomtsev-Petviashvili II equation on the half-plane

The KPII equation is an integrable nonlinear PDE in 2+1 dimensions (two spatial and one temporal), which arises in several physical circumstances, including fluid mechanics, where it describes waves in shallow water. It provides a multidimensional generalisation of the renowned KdV equation. In this work, we employ a novel approach recently introduced by one of the authors in connection with the Davey-Stewartson equation (Fokas (2009) [13]), in order to analyse the initial-boundary value problem for the KPII equation formulated on the half-plane. The analysis makes crucial use of the so-called d-bar formalism, as well as of the so-called global relation. A novel feature of boundary as opposed to initial value problems in 2+1 is that the d-bar formalism now involves a function in the complex plane which is discontinuous across the real axis.