基于多尺度递归网络的图像超分辨率重建

提出了一种基于多尺度递归网络的图像超分辨率网络模型,该模型主要由多个多尺度特征映射单元级联而成,每个单元分别包含一组不同尺度的特征提取层、一个融合层以及一个特征映射层。特征提取直接在原始低分辨率图像上进行,最后采用亚像素卷积重构高分辨率图像。训练阶段使用自适应矩估计优化方法加速网络模型的收敛。实验结果表明,所提算法取得了较好的超分辨率结果,图像纹理清晰、边缘锐利,视觉效果明显得到增强。在Set5、Set14、BSD100以及Urban100等常用测试集上的客观评价指标(PSNR和SSIM)均高于现有的几种主流算法。     

多局部残差连接注意网络的图像去模糊

针对现有的基于卷积神经网络的图像去模糊算法存在图像纹理细节恢复不清晰的问题,提出了一种基于多局部残差连接注意网络的图像去模糊算法。首先,采用一个卷积层进行浅层特征提取;其次,设计了一种新的基于残差连接和并行注意机制的多局部残差连接注意模块,用于消除图像模糊并提取上下文信息;再次,采用一个基于扩张卷积的成对连接模块进行细节恢复;最后,利用一个卷积层重建清晰图像。实验结果表明：在GoPro数据集上的PSNR (peak signal to noise ratio)和SSIM (structure similarity)分别为31.83 dB、0.927 5,在定性和定量两方面都表明所提方法能够有效地恢复模糊图像的纹理细节,网络性能优于对比方法。     

基于颜色预测和语义感知双分支结构的近红外图像彩色化算法北大核心CSCD

针对近红外图像彩色化过程中因近红外与可见光图像间模态差异较大,导致着色后图像在纹理细节处存在颜色晕染、区域误着色的问题,提出了颜色预测和语义感知相结合的生成对抗网络。设计颜色预测和语义感知双分支生成器,颜色预测分支采用带有跳转连接的残差网络,语义感知分支采用带有语义融合的空洞卷积金字塔结构;不同的扩张率,能够获得多个感受野提取多尺度语义特征,将感知到的语义嵌入到颜色预测分支,提高模型的语义理解能力,改善颜色晕染、区域误着色问题;设计循环一致语义损失函数,约束生成器中语义信息的一致性;算法在RGB-NIR场景数据集上进行性能实验比对以及消融实验。实验表明,所提算法相比于现有彩色化算法,PSNR、SSIM和LPIPS评价指标优于现有算法,着色效果更符合视觉感受。     

基于凸优化的傅里叶叠层成像技术研究

在傅里叶叠层成像（FPM）过程中采集的低分辨率图像会对重建图像质量产生直接影响,已有的研究提出用图像超分辨率重建技术和对低分辨率图像进行传统去噪处理的方法来解决该问题,但超分辨率重建的方法需要采集大量的原始图像,会加大采集端的时间损耗,而传统去噪算法会造成原始信息丢失,严重影响重构图像质量。因此论文引入凸优化算法,噪声图像的恢复可以通过求解一个凸优化模型来实现,并用迭代收缩阈值算法来求解该模型,算法中采用Barzilai-Borwein（BB）规则在每次迭代时初始化线搜索步长,加快收敛速度,选用软阈值函数,使图像去噪时原始信息丢失减少,最终重构图像的PSNR为27.634 6 dB,SSIM为0.926 1,所需处理时间为5.850 s,因此基于凸优化的傅里叶叠层成像技术具有时间损耗不大的情况下提高重构图像质量的优点。     

凝视成像降质模型的超分辨率重建

针对吉林一号视频03星凝视成像的特点,为提高其图像空间分辨率,解决光学系统分辨率不足的问题,提出一种视频卫星超分辨率重建的新算法——凸集中间映射。以主动观点分析视频卫星凝视成像特点,建立了基于凝视成像的图像降质模型。为求解该降质模型的逆过程,以凸集理论为基础,建立了基于中间降质过程的约束集和相应的点投影算子,通过点投影算子逐帧修正高分辨率图像灰度值,最终将重建的高分辨率图像约束于凸集的交集上。实验结果表明,该算法使图像分辨率提高近30%,克服了同类算法具有投影误差和重叠伪影的缺点,图像质量评价指标均优于所列其他算法,8帧重建得到收敛解,对不同清晰度的图像重建均具有可行性和稳健性。说明该算法适用于视频卫星图像超分辨率重建。     

保留边界特征的深度图像增强算法研究

针对现有深度图像增强算法存在边界保留特性差的问题,提出梯度掩模导向联合滤波(gradient mask guided joint filter, GMGJF)算法。利用深度图像进行Sobel梯度变换获取边界方向信息,利用深度图像空洞区域生成空洞掩模,再以边界方向和空洞掩模为导向联合彩色图像对深度图像进行迭代高斯滤波和空洞填充。实验结果表明,GMGJF算法的PSNR(peak signal to noise ratio)、SSIM(structural similarity index measure)比IMF(iterative median filter)、GF(guided filter)、JBF(joint bilateral filter)算法的PSNR、SSIM至少提高了3.50%和1.07%,不仅去噪能力、空洞填充能力最强,而且边界特征保持最好,有利于深度图像的特征提取与目标识别。     

基于全变分正则约束的二维冲击波速度场条纹重构技术

在惯性约束聚变过程中,冲击波速度与靶丸内爆压缩对称性密切相关,任意反射面速度干涉仪（VISAR）与压缩超快成像（CUP）系统的结合（CUP-VISAR）为冲击波速度二维时变诊断开辟了新思路。针对系统重建耗时长的问题,提出并实现了一种针对CUP-VISAR系统的全变分正则化快速重建算法。对弯曲条纹的仿真重建分析结果表明,本文提出的TVAL3H算法对比传统TVAL3算法,峰值信噪比（PSNR）提升了6.86 dB (25 frame)～1.20 dB (150 frame),结构相似性（SSIM）提升了26.67%(25frame)～14.10%(150frame),时间消耗降低了92.15%(25frame)～78.30%(150 frame)。对比广义交替投影（GAP）和交替方向乘子（ADMM）算法,时间消耗降低了57.79%(100 frame,GAP)～77.20%(25 frame,ADMM)的同时PSNR和SSIM差异较小。在同一重构时间量级下,所建立重构算法不同frame条件的PSNR相比GAP与ADMM算法分别提高了1.92 dB (25 frame)～0.84 dB (1...     

基于GAN的人脸超分辨率重建算法研究

为解决当前人脸超分辨率算法细节处理不足和过度平滑等问题,基于对抗网络技术提出一种针对单一面部图像的超分辨率重建算法。在生成网络中并联边缘检测网络,提取丰富的人脸轮廓细节以辅助特征提取,通过Ranger优化器优化网络训练过程,最终结合客观评价和主观评价指标,建立数学模型综合评价重建效果。实验结果表明,算法较三次样条法、SRGAN、FSRCNN等方法具有更优的主观和客观评价结果。提升了面部的细节复原能力,具有更好的重建效果。     

纹理细节恢复的图像超分辨率重建算法

针对现有单图像超分辨率方法在重建过程中容易忽略原图像中不同结构-纹理的差异与联系，导致生成的高分辨率图像缺乏纹理细节并存在伪影的问题，提出了纹理细节恢复的图像超分辨率重建算法。该方法由梯度分支、纹理分支和图像超分辨率分支组成。其中，在梯度分支和纹理分支之间使用了类注意力模块处理二者的特征混淆问题，并通过双向特征融合模块实现了对结构特征与纹理特征的相互促进，作为先验信息以达到纹理细节信息增强的目的。此外，在图像超分辨率分支还通过构建特征恢复模块，利用浅层和深层信息帮助网络保留了图像中更丰富的上下文信息和纹理细节。该方法通过在DIV2K数据集上进行了网络训练，并在5个基准测试集Set5、Set14、BSD100、Urban100和MANGA109上进行了实验，峰值信噪比(PSNR, Peak Signal to Noise Ratio)分别：37.88dB、33.28dB、32.0781dB、31.89dB、38.39dB,相比现有方法均有显著提升。实验结果表明，本文方法获得了有效的重建图像并且保留更多的图像细节，生成具有边缘清晰和逼真细节的超分辨率图像。     

基于小递归卷积神经网络的图像超分辨算法

针对现有软件实现超分辨算法通常过于复杂、运算开销大、模型复杂度高的问题,本文从成像过程中图像退化的物理原理出发,提出一套基于小递归卷积神经网络的单帧图像超分辨模型.将物理模型的约束融入到模型中,与现有的基于统计学习的图像超分辨算法相比,本文提出的模型的模型复杂度和计算量几乎可以忽略不计,同时内部的参数也有着更加明确的物理意义,并且引入了外部数据辅助对相应的模型参数进行学习.使用运行速度、峰值信噪比的数值方法对结果进行评价,结果表明:本文提出的算法消耗时间只有传统反向投影算法的75%,而精度比反向投影算法提高了0.2dB,比双线性插值提高了1.2dB.本文提出的算法可以取得比迭代反投影算法更快、重建精度更高的超分辨重建效果.     

稀疏阈值的超分辨率图像重建

为了解决基于字典学习的超分辨重构算法耗时过长的问题,提出了基于稀疏阈值模型的图像超分辨率重建方法。首先,将联合字典理论与图像块稀疏阈值方法相结合,训练得到高、低分辨率过完备图像字典对。接着,通过稀疏阈值OMP算法对图像特征块进行稀疏表示。然后,通过高分辨率字典重构出初始的超分辨图像。最后,通过改进迭代反投影算法对初始的超分辨图像进行全局优化,从而进一步提高图像重构质量。实验结果表明,超分辨图像重构平均峰值信噪比(PSNR)为30.1 d B,平均结构自相似度(SSIM)为0.937 9,平均计算时间为10.2 s。有效提高了超分辨重构的速度,改善了重构高分辨图像的质量。     

基于DCGANs的半片光伏组件电致发光图像增强技术

针对半片光伏组件电致发光（electroluminescence,EL）缺陷自动识别过程中训练用样本不足导致模型过拟合的问题,采用深度卷积生成对抗网络（deep convolutional generative adversarial networks,DCGANs）生成可控制属性的半片光伏组件EL图像,再采用多尺度结构相似性（multiscale structural similarity,MS-SSIM）指标对生成的EL图像与拍摄的EL图像之间的相似程度进行了评估。评估结果得到,使用DCGANs生成的所有类型半片光伏组件的EL图像与拍摄的EL图像的MS-SSIM指标都大于0.55,大部分的MS-SSIM值在0.7附近。在分类模型的训练过程中,测试集准确率随着训练集中生成图像数量的增加而升高,当生成图像数量达到6 000张时,测试集准确率达到97.92%。实验结果表明,采用DCGANs能够生成高质量且可控制属性的半片光伏组件EL图像,较好地解决因缺少训练样本而导致的模型过拟合问题。     

Image quality assessment in multiband DCT domain based on SSIM

The performance of image quality assessment method based on SSIM (structural similarity) is better than the PSNR (peak signal to noise ratio), but the assessment effects of SSIM is poor for seriously blurred image, therefore, the model that combined HVS (human visual sensitivity) and SSIM was established. The basic idea is based on the human eye's sensitivity to different frequency distortion image, the image is two-dimensional discrete cosine transform frequency component into low, mid, high-frequency component, to obtain the frequency component of light, contrast and structural information, using Pearson coefficient for weight and sum processing to the sub-image according to frequency bands of different sensitive degree, finally, get the sharpness of the image. Through nonlinear regression analysis of objective assessment and DMOS, experiments showed that this method was closer to human perception than SSIM and GSSIM for serious blurred distortion image. At the same time, compared to conventional algorithm MAE (mean absolute error), MSE (mean square error) and PSNR, this model was more consistent with human visual characteristics.     

Structural similarity based single image super-resolution with nonlocal regularization

Recently, sparse coding based image super-resolution has attracted increasing interests. This paper proposes an improved image super-resolution method, by incorporating structural similarity (SSIM) index and nonlocal regularization into the framework of image super-resolution via sparse coding. Firstly, an algorithm of combining SSIM based sparse coding and K-SVD is proposed to train the high resolution (HR) and low resolution (LR) dictionary pairs. And then, the sparse representations of observed LR image are sought to reconstruct the HR image with the trained LR and HR dictionary pairs by exploiting nonlocal self-similarities. Experimental results demonstrate the effectiveness of the proposed method, both in its visual effects and in quantitative terms.     

基于注意力残差编解码网络的动态场景图像去模糊

动态场景下的图像去模糊技术是一个具有挑战性的计算机视觉问题。模糊图像不仅影响主观感受,还会影响后续的智能化分析的性能。提出了一种基于注意力残差编解码网络的动态场景图像去模糊方法。首先,编码阶段采用多个残差模块提取特征,加入空间注意力模块感知模糊的空间位置信息;其次,通过在网络中采用全局-局部残差连接策略融合多层卷积特征,减少信息丢失;最后,解码阶段生成具有清晰边缘结构的复原图像。实验结果显示,提出的算法在公开数据集上获得的峰值信噪比值为31.76 dB,结构相似性值为0.912。客观和主观质量评估表明,本文算法能够有效地复原包含丰富边缘轮廓信息的清晰图像,在对比算法中获得最优的性能。     

Fast single image super-resolution using estimated low-frequency k-space data in MRI

PurposeSingle image super-resolution (SR) is highly desired in many fields but obtaining it is often technically limited in practice. The purpose of this study was to propose a simple, rapid and robust single image SR method in magnetic resonance (MR) imaging (MRI).MethodsThe idea is based on the mathematical formulation of the intrinsic link in k-space between a given (modulus) low-resolution (LR) image and the desired SR image. The method consists of two steps: 1) estimating the low-frequency k-space data of the desired SR image from a single LR image; 2) reconstructing the SR image using the estimated low-frequency and zero-filled high-frequency k-space data. The method was evaluated on digital phantom images, physical phantom MR images and real brain MR images, and compared with existing SR methods.ResultsThe proposed SR method exhibited a good robustness by reaching a clearly higher PSNR (25.77dB) and SSIM (0.991) averaged over different noise levels in comparison with existing edge-guided nonlinear interpolation (EGNI) (PSNR=23.78dB, SSIM=0.983), zero-filling (ZF) (PSNR=24.09dB, SSIM=0.985) and total variation (TV) (PSNR=24.54dB, SSIM=0.987) methods while presenting the same order of computation time as the ZF method but being much faster than the EGNI or TV method. The average PSNR or SSIM over different slice images of the proposed method (PSNR=26.33 dB or SSIM=0.955) was also higher than the EGNI (PSNR=25.07dB or SSIM=0.952), ZF (PSNR=24.97dB or SSIM=0.950) and TV (PSNR=25.70dB or SSIM=0.953) methods, demonstrating its good robustness to variation in anatomical structure of the images. Meanwhile, the proposed method always produced less ringing artifacts than the ZF method, gave a clearer image than the EGNI method, and did not exhibit any blocking effect presented in the TV method. In addition, the proposed method yielded the highest spatial consistency in the inter-slice dimension among the four methods.ConclusionsThis study proposed a fast, robust and efficient single image SR method with high spatial consistency in the inter-slice dimension for clinical MR images by estimating the low-frequency k-space data of the desired SR image from a single spatial modulus LR image.     

一种基于信息保留网络的图像去噪算法

由于成像设备等各种因素影响, 图像在成像或传感过程中会受到噪声干扰。图像去噪旨在减少或消除噪声对图像的影响, 这一过程往往会导致高频信息的丢失。为了在去除图像噪声的同时保护图像的边缘信息与纹理细节, 文章提出了一种计算复杂度相对较低的含有信息保留模块的卷积神经网络, 直接对含噪声图像进行降噪。信息保留模块通过残差学习提取局部长路径和局部短路径的混合特征信息。该文采用峰值信噪比(PSNR/dB)和结构相似性(SSIM)两项评价指标对实验结果进行量化, 这两项指标值越大, 说明去噪效果越好。实验结果表明, 在峰值信噪比和结构相似性2项评价指标的均值可达到30.36 dB和0.828 0, 相比其他对比算法, 2项评价指标分别平均提升了2.15 dB和0.072 9。该算法对不同种类、不同水平的噪声都具有良好的去噪效果, 且速度优于所对比的一般算法, 对基于卷积神经网络的去噪工作的进一步发展有一定的作用。     

Perceptual similarity of regional dialects of American English

Previous research on the perception of dialect variation has measured the perceptual similarity of talkers based on regional dialect using only indirect methods. In the present study, a paired comparison similarity ratings task was used to obtain direct measures of perceptual similarity. Naive listeners were asked to make explicit judgments about the similarity of a set of talkers based on regional dialect. The talkers represented four regional varieties of American English and both genders. Results revealed an additive effect of gender and dialect on mean similarity ratings and two primary dimensions of perceptual dialect similarity: geography (northern versus southern varieties) and dialect markedness (many versus few characteristic properties). The present findings are consistent with earlier research on the perception of dialect variation, as well as recent speech perception studies which demonstrate the integral role of talker gender in speech perception.     

Deep artifact suppression for spiral real-time phase contrast cardiac magnetic resonance imaging in congenital heart disease

PurposeReal-time spiral phase contrast MR (PCMR) enables rapid free-breathing assessment of flow. Target spatial and temporal resolutions require high acceleration rates often leading to long reconstruction times. Here we propose a deep artifact suppression framework for fast and accurate flow quantification.MethodsU-Nets were trained for deep artifact suppression using 520 breath-hold gated spiral PCMR aortic datasets collected in congenital heart disease patients. Two spiral trajectories (uniform and perturbed) and two losses (Mean Absolute Error -MAE- and average structural similarity index measurement -SSIM-) were compared in synthetic data in terms of MAE, peak SNR (PSNR) and SSIM. Perturbed spiral PCMR was prospectively acquired in 20 patients. Stroke Volume (SV), peak mean velocity and edge sharpness measurements were compared to Compressed Sensing (CS) and Cartesian reference.ResultsIn synthetic data, perturbed spiral consistently outperformed uniform spiral for the different image metrics. U-Net MAE showed better MAE and PSNR while U-Net SSIM showed higher SSIM based metrics.In-vivo, there were no significant differences in SV between any of the real-time reconstructions and the reference standard Cartesian data. However, U-Net SSIM had better image sharpness and lower biases for peak velocity when compared to U-Net MAE. Reconstruction of 96 frames took ~59 s for CS and 3.9 s for U-Nets.ConclusionDeep artifact suppression of complex valued images using an SSIM based loss was successfully demonstrated in a cohort of congenital heart disease patients for fast and accurate flow quantification.     

Multi-scale SSIM metric based on weighted wavelet decomposition

Image quality assessment aims to use computational models to assess the image quality consistently with subjective evaluations. This paper proposes a new metric composed of weighted wavelet multi-scale structural similarity (WWMS-SSIM). Four-level 2-D wavelet decomposition is performed for the original and disturbed images, respectively. Each image can be partitioned into one low-frequency subband (LL) and a series of octave high-pass subbands (HL, LH and HH). Different subbands are processed with different weighting factors. Based on the results of the above, we can construct a modified WWMS-SSIM. Comparison experiments show that the correlation, prediction accuracy and consistency of the proposed metric are respectively 5.8%, 5.2% and 4.8% higher than the PSNR metric. The correlation, prediction accuracy and consistency of the proposed metric are respectively 0.7%, 1.6% and 2.6% higher than the SSIM metric. In terms of the experiment results, the WWMS-SSIM metric shows good feasibility comparing with PSNR and SSIM methods.