No-reference image quality assessment using structural activity

Presuming that human visual perception is highly sensitive to the structural information in a scene, we propose the concept of structural activity (SA) together with a model of SA indicator in a new framework for no-reference (NR) image quality assessment (QA) in this study. The proposed framework estimates image quality based on the quantification of the SA information of different visual significance. We propose some alternative implementations of SA indicator in this paper as examples to demonstrate the effectiveness of the SA-motivated framework. Comprehensive testing demonstrates that the model of SA indicator exhibits satisfactory performance in comparison with subjective quality scores as well as representative full-reference (FR) image quality measures.     

Screen content image quality assessment based on convolutional neural networks

Screen content image (SCI) is a composite image including textual and pictorial regions resulting in many difficulties in image quality assessment (IQA). Large SCIs are divided into image patches to increase training samples for CNN training of IQA model, and this brings two problems: (1) local quality of each image patch is not equal to subjective differential mean opinion score (DMOS) of an entire image; (2) importance of different image patches is not same for quality assessment. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN) for assessing the perceptual quality of SCIs. Our model conducts two designs solving problems which benefits from two strategies. For the first strategy, to imitate full-reference (FR) CNN-based model behavior, a CNN-based model is designed for both FR and NR IQA, and performance of NR-IQA part improves when the image patch scores predicted by FR-IQA part are adopted as the ground-truth to train NR-IQA part. For the second strategy, image patch qualities of one entire SCI are fused to obtain the SCI quality with an adaptive weighting method taking account the effect of the different image patch contents. Experimental results verify that our model outperforms all test NR IQA methods and most FR IQA methods on the screen content image quality assessment database (SIQAD). On the cross-database evaluation, the proposed method outperforms the existing NR IQA method in terms of at least 2.4 percent in PLCC and 2.8 percent in SRCC, which shows high generalization ability and high effectiveness of our model.     

No-reference image quality assessment using statistical characterization in the shearlet domain

Image and video quality measurements are crucial for many applications, such as acquisition, compression, transmission, enhancement, and reproduction. Nowadays, no-reference (NR) image quality assessment (IQA) methods have drawn extensive attention because it does not rely on any information of original images. However, most of the conventional NR-IQA methods are designed only for one or a set of predefined specific image distortion types, which are unlikely to generalize for evaluating image/video distorted with other types of distortions. In order to estimate a wide range of image distortions, in this paper, we present an efficient general-purpose NR-IQA algorithm which is based on a new multiscale directional transform (shearlet transform) with a strong ability to localize distributed discontinuities. This is mainly based on distorted natural image that leads to significant variation in the spread discontinuities in all directions. Thus, the statistical property of the distorted image is significantly different from that of natural images in fine scale shearlet coefficients, which are referred to as ‘distorted parts’. However, some ‘natural parts’ are reserved in coarse scale shearlet coefficients. The algorithm relies on utilizing the natural parts to predict the natural behavior of distorted parts. The predicted parts act as ‘reference’ and the difference between the reference and distorted parts is used as an indicator to predict the image quality. In order to achieve this goal, we modify the general sparse autoencoder to serve as a predictor to get the predicted parts from natural parts. By translating the NR-IQA problem into classification problem, the predicted parts and distorted parts are utilized to form features and the differences between them are identified by softmax classifier. The resulting algorithm, which we name SHeArlet based No-reference Image quality Assessment (SHANIA), is tested on several database (LIVE, Multiply Distorted LIVE and TID2008) and shown to be suitable for many common distortions, consistent with subjective assessment and comparable to full-reference IQA methods and state-of-the-art general purpose NR-IQA algorithms.     

Blind image quality assessment in the contourlet domain

No-reference/blind image quality assessment (NR-IQA/BIQA) algorithms play an important role in image evaluation, as they can assess the quality of an image automatically, only using the distorted image whose quality is being assessed. Among the existing NR-IQA/BIQA methods, natural scene statistic (NSS) models which can be expressed in different bandpass domains show good consistency with human subjective judgments of quality.In this paper, we create new ‘quality-aware’ features: the energy differences of the sub-band coefficients across scales via contourlet transform, and propose a new NR-IQA/BIQA model that operates on natural scene statistics in the contourlet domain. Prior to applying the contourlet transform, we apply two preprocessing steps that help to create more information-dense, low-entropy representations. Specifically, we transform the picture into the CIELAB color space and gradient magnitude map. Then, a number of ‘quality-aware’ features are discovered in the contourlet transform domain: the energy of the sub-band coefficients within scales, and the energy differences between scales, as well as measurements of the statistical relationships of pixels across scales. A detailed analysis is conducted to show how different distortions affect the statistical characteristics of these features, and then features are fed to a support vector regression (SVR) model which learns to predict image quality. Experimental results show that the proposed method has high linearity against human subjective perception, and outperforms the state-of-the-art NR-IQA models.     

Unsupervised blind image quality assessment based on joint structure and natural scene statistics features

Compared with the widely used supervised blind image quality assessment (BIQA) models, unsupervised BIQA models require little prior knowledge for calculating the objective quality scores of distorted images. In this paper, we propose an unsupervised BIQA method that aims to achieve both good performance and generalization capability with low computational complexity. Carefully selected and extensive structure and natural scene statistics (NSS) features can better represent image quality. First, we employ phase congruency (PC) and finely selected gradient magnitude map and Laplacian of Gaussian response (GM-LOG) features to represent image structure information. Second, we calculate the local mean-subtracted and contrast-normalized (MSCN) coefficients and the Karhunen–Loéve transform (KLT) coefficients to represent the naturalness of the distorted images. Last, multivariate Gaussian (MVG) model with joint features extracted from both the pristine images and the distorted images is adopted to calculate the objective image quality. Extensive experiments conducted on nine IQA databases demonstrate that the proposed method achieves better performance than the state-of-the-art BIQA methods.     

No-reference image quality assessment based on neighborhood co-occurrence matrix

No-reference image quality assessment (NR-IQA) aims to develop models that can predict the quality of distorted image automatically and accurately in the absent of reference image. Previous NR-IQA methods based on natural scene statistics (NSS) always focus on the luminance contrast of image but attach limited attention to pixel-wise relationship. However, human visual system (HVS) is highly adaptive to extract spatial correlation according to relative position within visual field. In this paper, a new approach is proposed for NR-IQA, in which the neighborhood co-occurrence matrix (NCM) is introduced to describe spatial correlation of pixels for quality assessment. The NCM is constructed based on spatial correlation of every pixel and its neighborhood through a mapping to highlight the one-to-many pixel-wise relationship. Moreover, a series of tailored statistical metrics are designed to quantify the unnaturalness extent of NCM effectively, which is combined with others natural scene statistics to predict image quality. Extensive experiments demonstrate the proposed method has superior performance against compared methods, and achieves significant improvements on distortions associated with color or locality.     

Kurtosis-based no-reference quality assessment of JPEG2000 images

No-reference (NR) image quality assessment (QA) presumes no prior knowledge of reference (distortion-free) images and seeks to quantitatively predict visual quality solely from the distorted images. We develop kurtosis-based NR quality measures for JPEG2000 compressed images in this paper. The proposed measures are based on either 1-D or 2-D kurtosis in the discrete cosine transform (DCT) domain of general image blocks. Comprehensive testing demonstrates their good consistency with subjective quality scores as well as satisfactory performance in comparison with both the representative full-reference (FR) and state-of-the-art NR image quality measures.     

No-reference quality assessment for DCT-based compressed image

A blind/no-reference (NR) method is proposed in this paper for image quality assessment (IQA) of the images compressed in discrete cosine transform (DCT) domain. When an image is measured by structural similarity (SSIM), two variances, i.e. mean intensity and variance of the image, are used as features. However, the parameters of original copies are actually unavailable in NR applications; hence SSIM is not widely applicable. To extend SSIM in general cases, we apply Gaussian model to fit quantization noise in spatial domain, and directly estimate noise distribution from the compressed version. Benefit from this rearrangement, the revised SSIM does not require original image as the reference. Heavy compression always results in some zero-value DCT coefficients, which need to be compensated for more accurate parameter estimate. By studying the quantization process, a machine-learning based algorithm is proposed to estimate quantization noise taking image content into consideration. Compared with state-of-the-art algorithms, the proposed IQA is more heuristic and efficient. With some experimental results, we verify that the proposed algorithm (provided no reference image) achieves comparable efficacy to some full reference (FR) methods (provided the reference image), such as SSIM.     

An optimized CNN-based quality assessment model for screen content image

Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content compared to natural image. As a result, qualities of SCI patches are different, and the subjective differential mean opinion score (DMOS) is not appropriate as qualities of all image patches. Secondly, the average score of image patches does not represent the quality of entire SCI since the human visual system (HVS) is sensitive to image patches containing texture and edge information. In this paper, we propose a novel quadratic optimized model based on the deep convolutional neural network (QODCNN) for full-reference (FR) and no-reference (NR) SCI quality assessment to overcome these two problems. The contribution of our algorithm can be concluded as follows: 1) Considering the characteristics of SCIs, a valid network architecture is designed for both NR and FR visual quality evaluation of SCIs, which makes the networks learn the feature differences for FR-IQA; 2) with the consideration of correlation between local quality and DMOS, a training data selection method is proposed to fine-tune the pre-trained model with valid SCI patches; 3) an adaptive pooling approach is employed to fuse patch quality to obtain image quality, owns strong noise robust and effects on both FR and NR IQA. Experimental results verify that our model outperforms both current no-reference and full-reference image quality assessment methods on the benchmark screen content image quality assessment database (SIQAD). Cross-database evaluation shows high generalization ability and high effectiveness of our model.     

AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment

Stereoscopic imaging is widely used in many fields. In many scenarios, stereo images quality could be affected by various degradations, such as asymmetric distortion. Accordingly, to guarantee the best quality of experience, robust and accurate reference-less metrics are required for quality assessment of stereoscopic content. Most existing stereo no-reference Image Quality Assessment (IQA) models are not consistent with asymmetrical distortions. This paper presents a new no-reference stereoscopic image quality assessment metric using a human visual system (HVS) modeling and an advanced machine-learning algorithm. The proposed approach consists of two stages. In the first stage, cyclopean image is constructed considering the presence of binocular rivalry in order to cover the asymmetrically distorted part. In the second stage, gradient magnitude, relative gradient magnitude, and gradient orientation are extracted. These are used as a predictive source of information for the quality. In order to obtain the best overall performance against different databases, Adaptive Boosting (AdaBoost) idea of machine learning combined with artificial neural network model has been adopted. The benchmark LIVE 3D phase-I, phase-II, and IRCCyN/IVC 3D databases have been used to evaluate the performance of the proposed approach. Experimental results have demonstrated that the proposed metric performance achieves high consistency with subjective assessment and outperforms the blind stereo IQA over various types of distortion.     

Recaptured screen image identification based on vision transformer

Due to the copyright issues often involved in the recapture of LCD screen content, recaptured screen image identification has received lots of concerns in image source forensics. This paper analyzes the characteristics of convolutional neural network (CNN) and vision transformer (ViT) in extracting features and proposes a cascaded network structure that combines local-feature and global-feature extraction modules to detect the recaptured screen image from original images with or without demoiréing operation. We first extract the local features of the input images with five convolutional layers and feed the local features into the ViT to enhance the local perception capability of the ViT module, and further extract the global features of the input images. Through thorough experiments, our method achieves a detection accuracy rate of 0.9691 in our generated dataset and 0.9940 in the existing mixture dataset, both showing the best performance among the compared methods.     

A no-reference perceptual image quality assessment database for learned image codecs

The drastic growth of research in image compression, especially deep learning-based image compression techniques, poses new challenges to objective image quality assessment (IQA). Typical artifacts encountered in the emerging image codecs are significantly different from that produced by traditional block-based codecs, leading to inapplicability of the existing objective IQA algorithms. Towards advancing the development of objective IQA algorithms for recent compression artifacts, we built a learning-based compressed image quality assessment (LCIQA) database involving traditional block-based image codecs, hybrid neural network based image codecs, convolutional neural network based and generative adversarial network (GAN) based end-to-end optimized image coding approaches. Our study confirms the statistical difference and human perception difference between reconstructions of learned compression and traditional block-based compression. We propose a two-step deep learning model for learning-based compressed image quality assessment. Extensive experiments on LCIQA database demonstrate that our proposed model performs better than other counterparts on learning-based compressed images, especially on GAN compressed images, and achieves competitive performance to the state-of-the-art IQA metrics on traditional compressed images.     

An efficient and effective blind camera image quality metric via modeling quaternion wavelet coefficients

As an extension of Discrete and Complex Wavelet Transform, Quaternion Wavelet Transform (QWT) has attracted extensive attention in the past few years, because it can provide better analytic representation for 2D images. The QWT of an image consists of four parts, i.e., one magnitude part and three phase parts. The magnitude is nearly shift-invariant, which characterizes features at any spatial location, and the three phases represent the structure of these features. This indicates that QWT is more powerful in representing image structures, and thus is suitable for image quality evaluation. In this paper, an efficient and effective Camera Image Quality Metric (CIQM) is proposed based on QWT, which is utilized to describe the intrinsic structures of an image. For an image, it is first decomposed by QWT with three scales. Then, for each scale, the magnitude and entropy of the subband coefficients, and natural scene statistics of the third phase are calculated. The magnitude is utilized to describe the generalized spectral behavior, and the entropy is used to encode the generalized information of distortions. Since the third phase of QWT is considered to be texture feature, the natural scene statistics of the third phase of QWT is used to measure structure degradations in the proposed method. All these features reflect the self-similarity and independency of image content, which can effectively reflect image distortions. Finally, random forest is utilized to build the quality model. Experiments conducted on three camera image databases and two multiply distorted image databases have proved that CIQM outperforms the relevant state-of-the-art models for both authentically distorted images and multiply distorted images.     

No-reference image quality assessment based on spatial and spectral entropies

We develop an efficient general-purpose no-reference (NR) image quality assessment (IQA) model that utilizes local spatial and spectral entropy features on distorted images. Using a 2-stage framework of distortion classification followed by quality assessment, we utilize a support vector machine (SVM) to train an image distortion and quality prediction engine. The resulting algorithm, dubbed Spatial–Spectral Entropy-based Quality (SSEQ) index, is capable of assessing the quality of a distorted image across multiple distortion categories. We explain the entropy features used and their relevance to perception and thoroughly evaluate the algorithm on the LIVE IQA database. We find that SSEQ matches well with human subjective opinions of image quality, and is statistically superior to the full-reference (FR) IQA algorithm SSIM and several top-performing NR IQA methods: BIQI, DIIVINE, and BLIINDS-II. SSEQ has a considerably low complexity. We also tested SSEQ on the TID2008 database to ascertain whether it has performance that is database independent.     

图像质量评价的研究现状与发展

目前互联网应用与多媒体通信已成为信息世界的主流,数字图像在获取、压缩编码、存储或传输过程中存在不同程度的退化而影响视觉效果,因此图像质量的评价具有重要的理论和实际意义.梳理了目前国内外图像质量评价的最新研究成果,并对其进行归类、分析、研究与评述.在此基础上,提出图像质量评价的发展方向与研究展望.     

Attention integrated hierarchical networks for no-reference image quality assessment

Quality assessment of natural images is influenced by perceptual mechanisms, e.g., attention and contrast sensitivity, and quality perception can be generated in a hierarchical process. This paper proposes an architecture of Attention Integrated Hierarchical Image Quality networks (AIHIQnet) for no-reference quality assessment. AIHIQnet consists of three components: general backbone network, perceptually guided neck network, and head network. Multi-scale features extracted from the backbone network are fused to simulate image quality perception in a hierarchical manner. The attention and contrast sensitivity mechanisms modelled by an attention module capture essential information for quality perception. Considering that image rescaling potentially affects perceived quality, appropriate pooling methods in the non-convolution layers in AIHIQnet are employed to accept images with arbitrary resolutions. Comprehensive experiments on publicly available databases demonstrate outstanding performance of AIHIQnet compared to state-of-the-art models. Ablation experiments were performed to investigate the variants of the proposed architecture and reveal importance of individual components.     

一种改进的虹膜图像质量评价算法

为确保识别的准确性,提出一种改进的虹膜图像质量评价算法。根据图像总体清晰度和可见度的粗评估,可以快速而有效地剔除质量较差的图像,并利用虹膜纹理清晰度和可见度精评估来量化评价指标。实验结果表明,该方法可准确地判断虹膜图像的质量,提高系统的工作效率,其评价结果和人眼主观评价相吻合,具有一定的应用价值。     

图像拼接质量评价方法

在现有图像质量评价方法相关原理基础上,提出了一种基于图像边缘信息的拼接质量评价新方法。该方法针对图像拼接结果的特点,先对待评价图像进行边缘提取,然后利用拼接前后图像的边缘轮廓信息,综合图像像素误差信息和结构信息,根据其均值和方差等统计信息与影响图像拼接质量的主要因素(拼接错位和亮度突变)之间的关系,对拼接图像进行评价。该评价方法得出的评价结果更加符合人眼视觉对图像拼接质量的主观评价感受,较准确地反映了拼接图像的真实质量和所使用图像拼接算法的性能。