ATTENTION-GUIDED DUAL-DOMAIN EMBEDDING FOR INTELLECTUAL PROPERTY PROTECTION OF IMAGE PROCESSING MODEL

Abstract

Deep Learning based image processing models such as denoising, super-resolution and deraining consume significant computational resources, curated data and engineering effort. However when models such as those, are available through publicly accessible APIs, it renders them vulnerable to Surrogacy Model Extraction attacks, in which an adversary sends queries to the model, collects the output and training a functionally equivalent replica of the model without acquiring its parameters. Watermarking schemes typically fail to protect the model as, its reconstruction objective, is trained on the premise that the pixel level watermark embedded within it is random noise, and averaging it will result in elimination of such information. This thesis proposes an Attention Guided Dual Domain Embedding (AGDDE) network framework for the IP protection of image processing models, in the form of Frequency Aware Structure Consistent Watermarking (FA-SCW) system. A dual branch attention mechanism which is much simpler and replacing the fixed Sobel detector used in previous work, learning a differentiable mask identifying both edges and flat regions for embedding. A dual domain embedding network which combines a correct U-shape Network in spatial domain and CNN in middle frequency DCT coefficients domain. A self-supervised frequency consistency loss can be achieved to maintain spectrum invariance under geometric perturbation providing robustness to crop, rotation, JEPG compression without knowing the surrogate model. Evaluation has been performed on Color Berkeley Segmentation Dataset of 68 images (CBSD68) and DIVerse 2K resolution dataset (DIV2K), to show the imperceptibility is more than 35 dB PSNR, surrogate extraction Success Rate (SR) is higher than 85% and Bit Error Rate (BER) is less than 0.10 which is 22.9% higher than nearest competitor at higher quality. Complete system takes 30 minutes to converge on single GPU.