Deep Learning-Based Inpainting for Reconstructing Severely Damaged Handwritten Javanese Characters
https://doi.org/10.26594/register.v10i2.4929
Keywords:
Deep Learning, Inpainting, Reconstruction, Extensive Damage Areas, Handwritten Javanese CharacterAbstract
Long-term storage at museums can damage ancient Javanese manuscripts; for instance, temperature changes and other factors may cause parts of the script to disappear. The Javanese script exhibits similarities among its letters, making the reconstruction process challenging, particularly when dealing with severe damage to the script's characteristic areas. To address this issue, we conducted a character painting technique that utilizes deep learning architecture, specifically the convolutional autoencoder, partial convolutional neural network, UNet, and ResUNet. The dataset contains 12,000 handwritten Javanese characters. We evaluated the restoration of missing characters using SSIM and PSNR metrics. The ResUNet achieves the best performance compared to other methods, with an SSIM value of 0.9319 and a PSNR value of 18.9507 dB. According to this study, the ResUNet models can reconstruct Javanese manuscripts with strong performance, offering an alternative solution to ensure the preservation and accessibility of these valuable historical documents for future generations.
Downloads
References
D. Iskandar, S. Hidayat, U. Jamaludin, and S. Mukti Leksono, "Javanese script digitalization and its utilization as learning media: an etnopedagogical approach," Int. J. Math. Sci. Educ., vol. 1, no. 1, pp. 21-30, 2023, doi: 10.59965/ijmsed.v1i1.24.
A. Susanto, I. U. W. Mulyono, C. A. Sari, E. H. Rachmawanto, D. R. I. M. Setiadi, and M. K. Sarker, "Improved Javanese script recognition using custom model of convolution neural network," Int. J. Electr. Comput. Eng., vol. 13, no. 6, pp. 6629-6636, 2023, doi: 10.11591/ijece.v13i6.pp6629-6636.
A. Susanto, I. U. W. Mulyono, C. A. Sari, E. H. Rachmawanto, D. R. I. M. Setiadi, and M. K. Sarker, "Handwritten Javanese script recognition method based 12-layers deep convolutional neural network and data augmentation," IAES Int. J. Artif. Intell., vol. 12, no. 3, pp. 1448-1458, 2023, doi: 10.11591/ijai.v12.i3.pp1448-1458.
A. Setiawan, A. S. Prabowo, and E. Y. Puspaningrum, "Handwriting Character Recognition Javanese Letters Based On Artificial Neural Network Text Mining and Text Information Retrieval View project Artificial intelligence View project Handwriting Character Recognition Javanese Letters Based on Artificial Neura," Netw. Secur. Inf. Syst., vol. 1, no. 1, pp. 39-42, 2019, [Online]. Available: https://www.researchgate.net/publication/343228550.
A. Susanto, I. U. Wahyu Mulyono, C. Atika Sari, E. H. Rachmawanto, and D. R. I. Moses Setiadi, "An Improved Handwritten Javanese Script Recognition using Adaptive Threshold and Multi-Feature Extraction," 2022 Int. Semin. Appl. Technol. Inf. Commun. Technol. 4.0 Smart Ecosyst. A New W. Doing Digit. Business, iSemantic 2022, no. September, pp. 66-70, 2022, doi: 10.1109/iSemantic55962.2022.9920462.
F. Damayanti, Y. K. Suprapto, and E. M. Yuniarno, "Segmentation of Javanese Character in Ancient Manuscript using Connected Component Labeling," CENIM 2020 - Proceeding Int. Conf. Comput. Eng. Network, Intell. Multimed. 2020, no. Cenim, pp. 412-417, 2020, doi: 10.1109/CENIM51130.2020.9297954.
Z. Qin, Q. Zeng, Y. Zong, and F. Xu, "Image inpainting based on deep learning: A review," Displays, vol. 69, no. March, p. 102028, 2021, doi: 10.1016/j.displa.2021.102028.
H. Liu, B. Jiang, Y. Song, W. Huang, and C. Yang, "Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations," Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 12347 LNCS, pp. 725-741, 2020, doi: 10.1007/978-3-030-58536-5_43.
M. C. Sagong, Y. G. Shin, S. W. Kim, S. Park, and S. J. Ko, "PEPSI: Fast image inpainting with parallel decoding network," Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2019-June, pp. 11352-11360, 2019, doi: 10.1109/CVPR.2019.01162.
S. Ozturk, "Image Inpainting based Compact Hash Code Learning using Modified U-Net," 4th Int. Symp. Multidiscip. Stud. Innov. Technol. ISMSIT 2020 - Proc., no. 120, pp. 1-5, 2020, doi: 10.1109/ISMSIT50672.2020.9255239.
M. I. Hosen and M. B. Islam, "Masked Face Inpainting Through Residual Attention UNet," Proc. - 2022 Innov. Intell. Syst. Appl. Conf. ASYU 2022, 2022, doi: 10.1109/ASYU56188.2022.9925541.
G. Song, J. Li, and Z. Wang, "Occluded offline handwritten Chinese character inpainting via generative adversarial network and self-attention mechanism," Neurocomputing, vol. 415, pp. 146-156, 2020, doi: 10.1016/j.neucom.2020.07.046.
I. S. Jo, D. Bin Choi, and Y. B. Park, "Chinese character image completion using a generative latent variable model," Appl. Sci., vol. 11, no. 2, pp. 1-19, 2021, doi: 10.3390/app11020624.
H. Li et al., "Generative character inpainting guided by structural information," Vis. Comput., vol. 37, no. 9-11, pp. 2895-2906, 2021, doi: 10.1007/s00371-021-02218-y.
Y. Weng, H. Zhou, and J. Wang, "Image Inpainting Technique Based on Smart Terminal: A Case Study in CPS Ancient Image Data," IEEE Access, vol. 7, pp. 69837-69847, 2019, doi: 10.1109/ACCESS.2019.2919326.
B. Su, X. Liu, W. Gao, Y. Yang, and S. Chen, "A restoration method using dual generate adversarial networks for Chinese ancient characters," Vis. Informatics, vol. 6, no. 1, pp. 26-34, 2022, doi: 10.1016/j.visinf.2022.02.001.
R. Liu et al., "SCCGAN: Style and Characters Inpainting Based on CGAN," Mob. Networks Appl., vol. 26, no. 1, pp. 3-12, 2021, doi: 10.1007/s11036-020-01717-x.
J. Wang, G. Pan, D. Sun, and J. Zhang, "Chinese Character Inpainting with Contextual Semantic Constraints," MM 2021 - Proc. 29th ACM Int. Conf. Multimed., pp. 1829-1837, 2021, doi: 10.1145/3474085.3475333.
C. Xing and Z. Ren, "Binary Inscription Character Inpainting Based on Improved Context Encoders," IEEE Access, vol. 11, no. June, pp. 55834-55843, 2023, doi: 10.1109/ACCESS.2023.3282442.
L. Zhao and Y. Lou, "An Auto-Encoding Network for Binary Inscription Character Inpainting based on U-Net and Self- Attention An Auto-Encoding Network for Binary Inscription Character Inpainting based on U-Net," pp. 0-21, 2023.
L. Zhao and Y. Lou, "An Auto-Encoder of Inscription Character Inpainting based on Branch Convolutional Channel Attention Module An Auto-Encoder of Inscription Character Inpainting based on Branch Convolutional Channel Attention Module," pp. 0-20, 2023.
T. Chavez, E. J. Roberts, P. H. Zwart, and A. Hexemer, "A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering," J. Appl. Crystallogr., vol. 55, no. 5, pp. 1277-1288, 2022, doi: 10.1107/S1600576722007105.
A. Kornilov, I. Safonov, and I. Yakimchuk, "Inpainting of Ring Artifacts on Microtomographic Images by 3D CNN," Conf. Open Innov. Assoc. Fruct, vol. 2020-April, pp. 200-206, 2020, doi: 10.23919/FRUCT48808.2020.9087422.
H. Patel, A. Kulkarni, S. Sahni, and U. Vyas, "Image Inpainting using Partial Convolution," pp. 1-9, 2021, [Online]. Available: http://arxiv.org/abs/2108.08791.
M. W. Sabir et al., "Segmentation of Liver Tumor in CT Scan Using ResU-Net," Appl. Sci., vol. 12, no. 17, pp. 1-15, 2022, doi: 10.3390/app12178650.
F. I. Diakogiannis, F. Waldner, P. Caccetta, and C. Wu, "ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data," ISPRS J. Photogramm. Remote Sens., vol. 162, no. March 2019, pp. 94-114, 2020, doi: 10.1016/j.isprsjprs.2020.01.013.
E. Mique and A. Malicdem, "Deep Residual U-Net Based Lung Image Segmentation for Lung Disease Detection," IOP Conf. Ser. Mater. Sci. Eng., vol. 803, no. 1, 2020, doi: 10.1088/1757-899X/803/1/012004.
V. Pollatos, L. Kouvaras, and E. Charou, "Land cover semantic segmentation using ResUNet," CEUR Workshop Proc., vol. 2844, pp. 55-65, 2020.
O. Elharrouss, N. Almaadeed, S. Al-Maadeed, and Y. Akbari, "Image Inpainting: A Review," Neural Process. Lett., vol. 51, no. 2, pp. 2007-2028, 2020, doi: 10.1007/s11063-019-10163-0.
S. He and L. Schomaker, "DeepOtsu: Document enhancement and binarization using iterative deep learning," Pattern Recognit., vol. 91, pp. 379-390, 2019, doi: 10.1016/j.patcog.2019.01.025.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Fitri Damayanti, Eko Mulyanto Yuniarno , Yoyon Kusnendar Suprapto
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Please find the rights and licenses in Register: Jurnal Ilmiah Teknologi Sistem Informasi. By submitting the article/manuscript of the article, the author(s) agree with this policy. No specific document sign-off is required.
1. License
The non-commercial use of the article will be governed by the Creative Commons Attribution license as currently displayed on Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
2. Author(s)' Warranties
The author warrants that the article is original, written by stated author(s), has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author(s).
3. User/Public Rights
Register's spirit is to disseminate articles published are as free as possible. Under the Creative Commons license, Register permits users to copy, distribute, display, and perform the work for non-commercial purposes only. Users will also need to attribute authors and Register on distributing works in the journal and other media of publications. Unless otherwise stated, the authors are public entities as soon as their articles got published.
4. Rights of Authors
Authors retain all their rights to the published works, such as (but not limited to) the following rights;
Copyright and other proprietary rights relating to the article, such as patent rights,
The right to use the substance of the article in own future works, including lectures and books,
The right to reproduce the article for own purposes,
The right to self-archive the article (please read out deposit policy),
The right to enter into separate, additional contractual arrangements for the non-exclusive distribution of the article's published version (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal (Register: Jurnal Ilmiah Teknologi Sistem Informasi).
5. Co-Authorship
If the article was jointly prepared by more than one author, any authors submitting the manuscript warrants that he/she has been authorized by all co-authors to be agreed on this copyright and license notice (agreement) on their behalf, and agrees to inform his/her co-authors of the terms of this policy. Register will not be held liable for anything that may arise due to the author(s) internal dispute. Register will only communicate with the corresponding author.
6. Royalties
Being an open accessed journal and disseminating articles for free under the Creative Commons license term mentioned, author(s) aware that Register entitles the author(s) to no royalties or other fees.
7. Miscellaneous
Register will publish the article (or have it published) in the journal if the article’s editorial process is successfully completed. Register's editors may modify the article to a style of punctuation, spelling, capitalization, referencing and usage that deems appropriate. The author acknowledges that the article may be published so that it will be publicly accessible and such access will be free of charge for the readers as mentioned in point 3.
