Depth-aware blending of smoothed images for Bokeh effect generation

Bokeh effect is used in photography to capture images where the closer objects look sharp and everything else stays out-of-focus. Bokeh photos are generally captured using Single Lens Reflex cameras using shallow depth-of-field. Most of the modern smartphones can take bokeh images by leveraging dual rear cameras or a good auto-focus hardware. However, for smartphones with single-rear camera without a good auto-focus hardware, we have to rely on software to generate bokeh images. This kind of system is also useful to generate bokeh effect in already captured images. In this paper, an end-to-end deep learning framework is proposed to generate high-quality bokeh effect from images. The original image and different versions of smoothed images are blended to generate Bokeh effect with the help of a monocular depth estimation network. The model is trained through three phases to generate visually pleasing bokeh effect. The proposed approach is compared against a saliency detection based baseline and a number of approaches proposed in AIM 2019 Challenge on Bokeh Effect Synthesis. Extensive experiments are shown in order to understand different parts of the proposed algorithm. The network is lightweight and can process an HD image in 0.03 s. This approach ranked second in AIM 2019 Bokeh effect challenge-Perceptual Track.     

Facial parts swapping with generative adversarial networks

In this paper, we present a novel deep generative facial parts swapping method: parts-swapping generative adversarial network (PSGAN). PSGAN independently handles facial parts, such as eyes (left eye and right eye), nose, mouth and jaw, which achieves facial parts swapping by replacing the target facial parts with source facial parts and reconstructing the entire face image with these parts. By separately modeling the facial parts in the form of region inpainting, the proposed method can successfully achieve highly photorealistic face swapping results, enabling users to freely manipulate facial parts. In addition, the proposed method is able to perform jaw editing based on sketch guidance information. Experimental results on the CelebA dataset suggest that our method achieves superior performance for facial parts swapping and provides higher user control flexibility.     

A regional distance regression network for monocular object distance estimation

Monocular pipelines are convenient and cost-effective solutions for object distance estimation in 3D vision. Current methods for monocular object distance estimation either perform inaccurately or require heavy work on data collection. In this paper, we propose a network with R-CNN based structure to implement object detection and distance estimation simultaneously. We append an efficient branch to integrate the information of camera extrinsic parameters with RGB data in our network. Further, optimized multi-scale feature is utilized to enrich the representation power of deep feature, hence to enhance the estimation accuracy. Finally, several regression methods are explored to improve distance estimation results. We train and validate our network on KITTI object dataset, and compare with other methods to show that our method is accurate and easy to train. To prove the generality of our method under other scenarios, we construct a dataset of surveillance scenes, and conduct similar experiments on this dataset.     

Predicting ASD diagnosis in children with synthetic and image-based eye gaze data

As early intervention is highly effective for young children with autism spectrum disorder (ASD), it is imperative to make accurate diagnosis as early as possible. ASD has often been associated with atypical visual attention and eye gaze data can be collected at a very early age. An automatic screening tool based on eye gaze data that could identify ASD risk offers the opportunity for intervention before the full set of symptoms is present. In this paper, we propose two machine learning methods, synthetic saccade approach and image based approach, to automatically classify ASD given children’s eye gaze data collected from free-viewing tasks of natural images. The first approach uses a generative model of synthetic saccade patterns to represent the baseline scan-path from a typical non-ASD individual and combines it with the real scan-path as well as other auxiliary data as inputs to a deep learning classifier. The second approach adopts a more holistic image-based approach by feeding the input image and a sequence of fixation maps into a convolutional or recurrent neural network. Using a publicly-accessible collection of children’s gaze data, our experiments indicate that the ASD prediction accuracy reaches 67.23% accuracy on the validation dataset and 62.13% accuracy on the test dataset.     

基于迁移学习的机场场面目标检测与跟踪技术研究

本文分析了深度学习技术的基本原理,讨论了其应用与目标检测与跟踪领域的基本方式。利用实地采集的广汉机场场面视频数据,采用深度迁移学习的策略,研究了面向机场场面目标的检测与跟踪技术。     

Deep Neural Architecture for Recovering Dropped Pronouns in Korean

Pronouns are frequently dropped in Korean sentences, especially in text messages in the mobile phone environment. Restoring dropped pronouns can be a beneficial preprocessing task for machine translation, information extraction, spoken dialog systems, and many other applications. In this work, we address the problem of dropped pronoun recovery by resolving two simultaneous subtasks: detecting zero-pronoun sentences and determining the type of dropped pronouns. The problems are statistically modeled by encoding the sentence and classifying types of dropped pronouns using a recurrent neural network (RNN) architecture. Various RNN-based encoding architectures were investigated, and the stacked RNN was shown to be the best model for Korean zero-pronoun recovery. The proposed method does not require any manual features to be implemented; nevertheless, it shows good performance.     

Multi‐band Approach to Deep Learning‐Based Artificial Stereo Extension

In this paper, an artificial stereo extension method that creates stereophonic sound from a mono sound source is proposed. The proposed method first trains deep neural networks (DNNs) that model the nonlinear relationship between the dominant and residual signals of the stereo channel. In the training stage, the band‐wise log spectral magnitude and unwrapped phase of both the dominant and residual signals are utilized to model the nonlinearities of each sub‐band through deep architecture. From that point, stereo extension is conducted by estimating the residual signal that corresponds to the input mono channel signal with the trained DNN model in a sub‐band domain. The performance of the proposed method was evaluated using a log spectral distortion (LSD) measure and multiple stimuli with a hidden reference and anchor (MUSHRA) test. The results showed that the proposed method provided a lower LSD and higher MUSHRA score than conventional methods that use hidden Markov models and DNN with full‐band processing.     

Improved performance of deep-blue polymer light-emitting diodes by one-step coating self-assembly hole injection/transport nanocomposites with both the optical and electrical optimization

In this study we demonstrate an easy solution-processed highly efficient deep-blue polymer light-emitting diode (PLED) via a simple one-step coating of self-assembly hole injection/transport nanocomposites to achieve both a finer hole ohmic contact and an increased light outcoupling, which is the first time report about both the optical and electrical optimization without necessitating changes in the design or structure of the wide bandgap deep-blue PLEDs themselves. The contact angle and surface energy measurement results demonstrate that triazine-based hole injection molecules can vertically migrate towards the bottom PEDOT:PSS layer to obtain a stable minimum of free energy, resulting in an optimal top-to-bottom HOMO energy level arrangement and an improved hole mobility in deep-blue PLEDs. The random surface nanostructure was formed on top of the hole bilayer, leading to the enhancement of light outcoupling verified by transmittance, transmittance haze and light extraction efficiency. Furthermore, in order to explore the reasons of the hole light scattering formation process, a transient drying monitoring technique is applied to track the drying process of the nanostructure films, revealing this approach effectiveness by easily modifying mixing ratios for obtaining different light outcoupling abilities.     

Combining Deep Eutectic Solvents with TEMPO-based Polymer Electrodes: Influence of Molar Ratio on Electrode Performance**

For sustainable energy storage, all-organic batteries based on redox-active polymers promise to become an alternative to lithium ion batteries. Yet, polymers contribute to the goal of an all-organic cell as electrodes or as solid electrolytes. Here, we replace the electrolyte with a deep eutectic solvent (DES) composed of sodium bis(trifluoromethanesulfonyl)imide (NaTFSI) and N-methylacetamide (NMA), while using poly(2,2,6,6-tetramethylpiperidin-1-yl-oxyl methacrylate) (PTMA) as cathode. The successful combination of a DES with a polymer electrode is reported here for the first time. The electrochemical stability of PTMA electrodes in the DES at the eutectic molar ratio of 1 : 6 is comparable to conventional battery electrolytes. More viscous electrolytes with higher salt concentration can hinder cycling at high rates. Lower salt concentration leads to decreasing capacities and faster decomposition. The eutectic mixture of 1 : 6 is best suited uniting high stability and moderate viscosity.     

Comprehensive H2O Molecules Regulation via Deep Eutectic Solvents for Ultra-Stable Zinc Metal Anode

The corrosion, parasitic reactions, and aggravated dendrite growth severely restrict development of aqueous Zn metal batteries. Here, we report a novel strategy to break the hydrogen bond network between water molecules and construct the Zn(TFSI)₂-sulfolane-H₂O deep eutectic solvents. This strategy cuts off the transfer of protons/hydroxides and inhibits the activity of H₂O, as reflected in a much lower freezing point (<−80 °C), a significantly larger electrochemical stable window (>3 V), and suppressed evaporative water from electrolytes. Stable Zn plating/stripping for over 9600 h was obtained. Based on experimental characterizations and theoretical simulations, it has been proved that sulfolane can effectively regulate solvation shell and simultaneously build the multifunctional Zn-electrolyte interface. Moreover, the multi-layer homemade modular cell and 1.32 Ah pouch cell further confirm its prospect for practical application.