New corpora, tools and methods | 更新时间:2024-09-12
Development of the People’s Education Press primary and secondary school textbook Corpus
黄攀伟 * ,  胡源 ,  杨亚欣 ,  张博翎    作者信息&出版信息
Corpus Linguistics   ·   2024年9月12日   ·   2024年 11卷 第1期  
2 0(CNKI)
PDF
该文暂无导航

AI 摘要

1 Introduction

Corpus is an important component of language data resources and is crucial for language services and information processing. As a special type, textbook corpus is built based on textbook content and has retrieval function. The construction of Chinese textbook corpus started early but developed slowly, with many problems such as outdated corpus, limited coverage of disciplines, and unclear annotations. The PEPTC corpus developed by the People's Education Press aims to address these issues, utilize its rich textbook resources, improve book quality, and support textbook writing and research.

System Architecture Design

The PEPTC system is based on the Human Resources Education and Social Security Resource Management and Intelligent Service Platform, adopting a three-tier architecture to achieve unified storage, hierarchical management, and shared use of resources. The system resources mainly come from XML files of primary and secondary school textbooks and non textbook PDF files. PEPTC adopts a microservice architecture, including six levels: data source, data mining, data lake, application service, API service, and portal. The data sources are textbooks and non textbook files, data mining is used to discover indexed data, data lakes store analyzed corpora, application services provide intelligent analysis and processing capabilities, API layers provide interface capabilities, and portals are user interaction terminals. The system architecture design aims to achieve digitalization, knowledge-based, and manageable reuse of publishing resources.

3 Data processing principles

The data processing flow of PEPTC includes reading textbook XML files through the DAM interface, extracting structured content and saving it to the Elastic Search database, while storing structured data and metadata in the PostgreSQL database. The construction of data resources follows the principles of metadata standards, massive storage, introduction of intelligent data analysis software, and service-oriented approach, ensuring resource fragmentation, precise positioning, and support for digital transformation. The platform uses big data analysis software and document management system to achieve native integration, providing intelligent data analysis services, building algorithm models suitable for primary and secondary school textbooks, and providing customized business interfaces and online services through standard interfaces.

4 Corpus situation

The People's Education Press Primary and Secondary School Textbook Corpus (PEPTC) includes three sub databases: textbook corpus, non textbook ancient Chinese, and non textbook English. Textbook corpus is the core, and nearly 300 textbooks have been stored, with a corpus volume of 25 million bytes covering 18 subjects. The non textbook ancient Chinese corpus mainly consists of classic literature, while the non textbook English corpus includes middle and high school entrance exam questions, newspaper materials, and English and American literature. Users can select sub libraries through the homepage sidebar to search for raw or idiomatic materials, meeting different research needs.

5 Function Implementation

PEPTC adopts a browser server architecture, integrating a corpus and analysis tools, providing functions such as information overview, corpus analysis, resource management, and intelligent data analysis. The system homepage has a basic search bar and a corpus overview display. The retrieval and analysis tool supports multiple conditional filtering, multimodal retrieval, multidimensional statistics and comparison, synonym discrimination, etc., and the results can be exported with one click. System management includes corpus review and resource management. Users can edit or delete tags, entity objects, entity relationships, etc. The administrator reviews and modifies them, and also opens the corpus import function. Permission management has designed permission groups to enable different user permission configurations.

6 Construction Characteristics

The PEPTC corpus is of moderate scale, limited by the scope of primary and secondary school textbooks, incomplete inclusion of some corpora, and copyright issues. It mainly uses materials published by People's Education Press. The high quality of the corpus is attributed to the high-quality textbook content and standardized data processing procedures, including transcoding, OCR technology, and manual error correction. The corpus has undergone detailed preprocessing and annotation of teaching materials and language information to meet the needs of retrieval and analysis. Fully functional and easy to use, thanks to the application of forward-looking design, advanced technology, and artificial intelligence technology, such as neural network model training. R&D and application are carried out synchronously, with in-house editors and engineers working in parallel to timely realize the value of corpus resources and adjust and improve them.

7 Conclusion

PEPTC plays an important role in textbook writing, dictionary compilation, language teaching, and other fields, providing resources for subsequent development and utilization. PEPTC directly serves the compilation, research, and publication of textbooks, dictionaries, and educational books by the People's Education Press, improving the quality of student reference book compilation and providing reliable basis for language research in textbooks. Publishing houses achieve resource standardization and standardized management through corpus construction, acquire digital assets, and promote data interconnectivity. The future development direction of PEPTC includes expanding corpus, improving functions, building interdisciplinary knowledge bases, optimizing knowledge retrieval using pre trained model technology, and presenting knowledge element associations.

* 以上内容由AI自动生成,内容仅供参考。对于因使用本网站以上内容产生的相关后果,本网站不承担任何商业和法律责任。

展开

当前期刊

当前期刊
    目录