Diffusion models are effective purification methods,where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks.However,the efficienc...Diffusion models are effective purification methods,where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks.However,the efficiency of diffusion models is still a concern,and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps.Hence,we propose TendiffPure as a tensorized and compressed diffusion model for purification.Unlike the knowledge distillation methods,we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition,which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images.The space complexity is reduced from O(N^(2))to O(NR^(2))with R≤4 as the tensor-train rank and N as the number of channels.Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10,Fashion-MNIST,and MNIST datasets for two noises and one adversarial attack.展开更多
文摘Diffusion models are effective purification methods,where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks.However,the efficiency of diffusion models is still a concern,and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps.Hence,we propose TendiffPure as a tensorized and compressed diffusion model for purification.Unlike the knowledge distillation methods,we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition,which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images.The space complexity is reduced from O(N^(2))to O(NR^(2))with R≤4 as the tensor-train rank and N as the number of channels.Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10,Fashion-MNIST,and MNIST datasets for two noises and one adversarial attack.
