LayeredDoc: Domain Adaptive Document Restoration with a Layer Separation Approach
HyperNeRF

Domain Adaptive Document Restoration with a Layer Separation Approach

ECCV WICV workshop 2024 | ICDAR ADAPDA workshop 2024

Maria Pilligua1,2,⭐
Nil Biescas1,2,⭐
Javier Vazquez-Corral1,2
Josep Lladós1,2
Ernest Valveny1,2
Sanket Biswas1,2
1Computer Vision Center, 2Computer Science Department Universitat Autònoma de Barcelona, Catalonia, Spain
⭐Equal contribution
Paper arXiv Code Data
.

LayeredDoc is able to decompose a document image into distinct layers, each representing different information types such as textual or graphical elements.

Abstract

The rapid evolution of intelligent document processing systems demands robust solutions that adapt to diverse domains without extensive retraining. Traditional methods often falter with variable document types, leading to poor performance. To overcome these limitations, this paper introduces a text-graphic layer separation approach that enhances domain adaptability in document image restoration (DIR) systems. We propose LayeredDoc, which utilizes two layers of information: the first targets coarse-grained graphic components, while the second refines machine-printed textual content. This hierarchical DIR framework dynamically adjusts to the characteristics of the input document, facilitating effective domain adaptation. We evaluated our approach both qualitatively and quantitatively using a new real-world dataset, LayeredDocDB, developed for this study. Initially trained on a synthetically generated dataset, our model demonstrates strong generalization capabilities for the DIR task, offering a promising solution for handling variability in real-world data.

Results on LayeredDocDB

BibTeX

@misc{pilligua2024layereddocdomainadaptivedocument,
      title={LayeredDoc: Domain Adaptive Document Restoration with a Layer Separation Approach}, 
      author={Maria Pilligua and Nil Biescas and Javier Vazquez-Corral and Josep Lladós and Ernest Valveny and Sanket Biswas},
      year={2024},
      eprint={2406.08610},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2406.08610}, 
}

Acknowledgements

This work was partially supported by "The European Lighthouse on Safe and Secure AI - ELSA" funded by the European Union’s Horizon Europe programme under grant agreement No 101070617; the Spanish projects PID2021-128178OBI00 and PID2021-126808OB-I00 funded by MCIN/AEI/10.13039/501100011033, ERDF "A way of making Europe"; and the Catalan projects 2021-SGR-01499, and 2021-SGR-01559, funded by the Generalitat de Catalunya. S. Biswas is supported by the PhD Scholarship from AGAUR (2023 FI-3-00223), and N. Biescas and M. Pilligua are supported by the CVC Rosa Sensat Student Fellowship. The Computer Vision Center is part of the CERCA Program/Generalitat de Catalunya.