Research on Attribute based Generative Adversarial Networks for Image Restoration
1. Research purpose and significance
Photos captured by handheld devices may contain motion blur or artifacts caused by camera shake, object motion, focus or defocus. Image restoration research on how to recover clear latent image from degraded image caused by motion blur. Since the problem is ill-posed, we can restore better visual effect with the clear image prior as the aided knowledge. In recent years, with the increasing popularity of portable and lightweight digital imaging equipment and the rapid development of intelligent security and other Real-life applications, image restoration (Blind image deblurring) has attracted extensive attention from researchers in computer vision and image processing communities. However, most of the current image restoration algorithms have poor performance when dealing with blurred images of dynamic scenes, and the operation speed is slow, which cannot meet the real-time requirements. Therefore, image restoration can be transformed into an image generation problem. The recent prosperity of deep learning has led to significant progress in the image processing field. Specifically, Generative Adversarial Networks (GANs) [1] often yield sharper and more plausible textures than classical feed-forward encoder-decoder architectures, such as the success in image super-resolution [2]. Last year, [3] introduced GAN to deblurring by treating it as a special image-to-image translation task. The proposed model, called DeblurGAN, was demonstrated to restore perceptually pleasing and sharp images, from both synthetic and real-world blurry datasets. Recently, Kupyn et al.[5] introduced DeblurGAN-v2, Feature Pyramid Network to improve previous DeblurGAN. Extensive experiments show that, in terms of the efficiency, DeblurGAN-v2 with MobileNet-DSC is 11 times faster than the previous version. During my research, I will focus on the improvement of DeblurGAN-v2, named DeblurGAN-v2+, and specifically the main goal will be the implementation of a new backbone and framework in order to increase the accuracy and efficiency of DeblurGAN-v2+.
2. Research method and idea
This research proposes an method for single image motion deblurring, named DeblurGAN-v2+, for image restoration with a novel baseline network called EfficientNet[11] and Bidirectional Feature Pyramid Network(BiFPN). First, to take advantage of differnent levels blurry features for blind image deblurring, combination of the Feature Pyramid Network(FPN) with Inception-ResNet-v2 backbone into deblurring is adopted, as a core building block in the generator of our GAN model. Next, eplacing ResNet with EfficientNet backbone gives an instant improvement in mAP(mean Average Precission) and adding BiFPN gives a huge bump in mAP when compared to the baseline[12]. There are several reason accounting for it
EfficientNets achieves much better accuracy and efficiency than previous ConvNets. In particular, as far as I know, EfficientNet-B7 achieves state- of- the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet Compared with the widely used ResNet-50, EfficientNet-B4 uses similar FLOPS, while improving the top-1 accuracy from 76.3% of ResNet-50 to 82.6% (+6.3%); In addition, EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters, suggesting that EfficientNets also transfer well. Based on the above analysis, I expect DeblurGAN-v2 with EfficientNet will achieve better accuracy and visual improvements against state-
of-the-art methods.
, the main reason is that the EfficientNet+BiFPN has fewer parameters and a
very small number of FLOPS comparatively. Besides,