Defines the technical connotation of digitization, elaborates on the core scope and dual objectives of Tangut digitization—document preservation and disciplinary paradigm transformation—and identifies key technical challenges including character encoding, automatic recognition, word segmentation, machine translation, knowledge graph construction, and database building.
Reviews the complete journey of Tangut script from discovery to international standard encoding. Tangut script, a square logographic script created by the Tangut people, was used until the mid-Ming Dynasty. The large number of documents discovered by Kozlov in Khara-Khoto in 1909 propelled modern Tangut studies. Early scholars such as Luo Fuchang and Nishida Tatsuo systematically analyzed Tangut radicals and structures following Chinese character research paradigms, while multiple unearthed dictionaries provided internal materials for research. In encoding practice, Grinstead first proposed a scheme in 1972, but it only became a "telegraph code"; in the late 1980s, van Driem and Kepping attempted computer input but discontinued due to technical limitations; in 1996, Nakajima Motoki completed the first complete font library; the system developed by Ningxia University in 1999 had numerous flaws due to lack of professional verification, while the Academia Sinica font library gained international recognition through expert involvement. In 2007, Richard Cook submitted a Unicode proposal, which after revision by Chinese experts and multinational collaboration, led to Tangut script's inclusion in Unicode 9.0 in 2016 with 6,125 characters, marking a critical breakthrough in Tangut information infrastructure construction.
Introduces the development status and research progress of Tangut optical character recognition technology. Currently, Tangut OCR processing platforms are relatively scarce, mainly including the "Longshui Tangut Recognition Software" developed by the Chinese Academy of Social Sciences and printed text recognition platforms. The academic community has basically entered the stage of deep learning-based text recognition, adopting supervised learning patterns and relying on large-scale annotated datasets for neural network training. Research focus has shifted from single-character recognition to complex layout processing, including text line and region detection, dense text segmentation, and other key links. Addressing the characteristics of Tangut characters' numerous strokes and similar glyphs, scholars have proposed various solutions: Meng Yifei established a dataset based on four-corner code encoding and utilized generative adversarial networks to expand samples; Zhang Guangwei borrowed mature neural network architectures to achieve over 98% recognition accuracy; Liu Tong, Jing Shiyun, and others respectively employed CRAFT, PSENet, DS-CBAM-DenseNet, Res2Net and other models to improve detection and recognition performance. In recent years, research is shifting from supervised paradigms relying on massive annotation to efficient self-learning paradigms to address the complexity and scarcity of documentary resources. These technological advances have significantly promoted the digital research of Tangut documents.
Discusses the necessity and feasible approaches of Tangut automatic word segmentation research. As a fundamental step in natural language processing, word segmentation holds important value for downstream tasks such as part-of-speech tagging and named entity recognition, while this field remains blank for Tangut script and requires drawing on Chinese and Tibetan experiences. Chinese automatic word segmentation has evolved through statistical learning models (HMM, CRF), deep learning models (BiLSTM-CRF) to pre-trained language models (BERT, LEBERT), with large language models currently showing potential in cross-lingual classical text segmentation. Tibetan automatic word segmentation faces unique challenges due to its special structure of "syllable separation, word connection," evolving from rule-based, statistical methods, rule-statistical combination to deep learning and pre-trained models, with the establishment of "Tibetan Word Segmentation Standards for Information Processing" providing crucial support for its standardization. Tangut automatic word segmentation research is currently in its initial stage, having added a fifth line for manual word segmentation annotation based on "four-line parallel translation," completing annotation of approximately 25,000 characters of secular and religious documents, with plans to adopt three approaches—statistical models, deep learning models, and pre-trained models—to construct segmentation systems in the future.
Explores the development path of Tangut digitization from character image recognition to computational cognition. The current primary task is constructing a hierarchical digital corpus system, including basic data (high-definition images and metadata standards) and annotated materials (four-line parallel translation or interlinear comparison formats), but most institutions have been hindered by lack of large-scale interpreted document data. Future priority tasks include automatic word segmentation, part-of-speech tagging, and named entity recognition to build computational corpora supporting linguistic research; combining GIS and knowledge graphs to construct Tangut cultural digital maps and spatiotemporal knowledge ontologies, achieving structured representation of phonological, semantic, component, and grammatical information. At the technical level, structured visual modeling needs to be developed, fusing vision Transformers with graph convolutional networks, and advancing multimodal semantic fusion to integrate image, sound, cross-lingual text and other data, ultimately achieving an intelligent leap from "visual recognition" to "semantic understanding," providing solutions driven by both traditional philology and digital humanities technology for the revival of冷门学科.
* 以上内容由AI自动生成,内容仅供参考。对于因使用本网站以上内容产生的相关后果,本网站不承担任何商业和法律责任。