Crowd counting has important applications in public safety and pandemic control.A robust and practical crowd counting system has to be capable of continuously learning with the newly incoming domain data in real-world...Crowd counting has important applications in public safety and pandemic control.A robust and practical crowd counting system has to be capable of continuously learning with the newly incoming domain data in real-world scenarios instead of fitting one domain only.Off-the-shelf methods have some drawbacks when handling multiple domains:(1)the models will achieve limited performance(even drop dramatically)among old domains after training images from new domains due to the discrepancies in intrinsic data distributions from various domains,which is called catastrophic forgetting;(2)the well-trained model in a specific domain achieves imperfect performance among other unseen domains because of domain shift;(3)it leads to linearly increasing storage overhead,either mixing all the data for training or simply training dozens of separate models for different domains when new ones are available.To overcome these issues,we investigate a new crowd counting task in incremental domain training setting called lifelong crowd counting.Its goal is to alleviate catastrophic forgetting and improve the generalization ability using a single model updated by the incremental domains.Specifically,we propose a self-distillation learning framework as a benchmark(forget less,count better,or FLCB)for lifelong crowd counting,which helps the model leverage previous meaningful knowledge in a sustainable manner for better crowd counting to mitigate the forgetting when new data arrive.A new quantitative metric,normalized Backward Transfer(nBwT),is developed to evaluate the forgetting degree of the model in the lifelong learning process.Extensive experimental results demonstrate the superiority of our proposed benchmark in achieving a low catastrophic forgetting degree and strong generalization ability.展开更多
基金Project supported by the National Natural Science Foundation of China(Nos.62176059,62101136,and U1811463)the Shanghai Municipal Science and Technology Major Project(No.2018SHZDZX01)+3 种基金Zhangjiang Lab,the Shanghai Municipal of Science and Technology Project(No.20JC1419500)the Shanghai Sailing Program(No.21YF1402800)the Natural Science Foundation of Shanghai(No.21ZR1403600)the Shanghai Center for Brain Science and Brain-inspired Technology。
文摘Crowd counting has important applications in public safety and pandemic control.A robust and practical crowd counting system has to be capable of continuously learning with the newly incoming domain data in real-world scenarios instead of fitting one domain only.Off-the-shelf methods have some drawbacks when handling multiple domains:(1)the models will achieve limited performance(even drop dramatically)among old domains after training images from new domains due to the discrepancies in intrinsic data distributions from various domains,which is called catastrophic forgetting;(2)the well-trained model in a specific domain achieves imperfect performance among other unseen domains because of domain shift;(3)it leads to linearly increasing storage overhead,either mixing all the data for training or simply training dozens of separate models for different domains when new ones are available.To overcome these issues,we investigate a new crowd counting task in incremental domain training setting called lifelong crowd counting.Its goal is to alleviate catastrophic forgetting and improve the generalization ability using a single model updated by the incremental domains.Specifically,we propose a self-distillation learning framework as a benchmark(forget less,count better,or FLCB)for lifelong crowd counting,which helps the model leverage previous meaningful knowledge in a sustainable manner for better crowd counting to mitigate the forgetting when new data arrive.A new quantitative metric,normalized Backward Transfer(nBwT),is developed to evaluate the forgetting degree of the model in the lifelong learning process.Extensive experimental results demonstrate the superiority of our proposed benchmark in achieving a low catastrophic forgetting degree and strong generalization ability.