1. Introduction

This chapter discussed how the 2022 release of ChatGPT accelerated the adoption of large language models in language learning, emphasizing their superior interactivity, contextual understanding, and personalized explanations over traditional dictionaries. It noted the emergence of a question-answer knowledge production paradigm that replaces keyword queries with natural human-machine dialogue, thereby eroding distinctions among dictionaries, corpora, and translation tools. A survey of 250 university students revealed that users now loosely group any online bilingual resource under the label “dictionary,” underscoring the need to reassess tool boundaries. LLMs can automatically generate entries that mirror standard dictionary components—definitions, examples, pragmatic notes—yet differ fundamentally in authority and reliability, prompting scholars to declare a new lexicographical era. The chapter argued that systematically evaluating LLM-generated entries clarifies their viability as substitutes and guides optimization of traditional dictionaries amid digital transformation, while also illuminating evolving learner behaviors. Finally, it highlighted the scarcity of research on LLM applications for outward-oriented Chinese learner’s dictionaries and framed the study’s comparative analysis between ChatGPT and the manually compiled Juzi Chinese dictionary to assess definition quality, pinpoint word-type discrepancies, and gauge both feasibility and limitations of AI-assisted CSL lexicography.

2. Literature Review

This chapter reviewed empirical comparisons between large language models and traditional dictionaries in language learning and examined the emerging role of LLMs in dictionary compilation. Controlled experiments with 223 upper-intermediate to advanced university students showed ChatGPT outperforming the Longman Dictionary of Contemporary English on both vocabulary comprehension and production tasks, with significantly faster query times in production. A three-group replication with 166 participants confirmed ChatGPT’s advantage over monolingual and bilingual dictionaries in production tasks and its parity with bilingual dictionaries in comprehension. A reading task study revealed that Macmillan English Dictionary definitions improved performance relative to no definitions, while ChatGPT-generated definitions produced similar gains, indicating practical value for AI definitions. Automated text similarity analysis further demonstrated that ChatGPT outputs approximated Oxford Advanced Learner’s Dictionary microstructures, supporting its lexicographic potential.

Researchers attribute ChatGPT’s effectiveness to interactivity, immediate feedback, and context sensitivity, yet stress that optimal outcomes arise from hybrid integration of AI and traditional tools. The literature also documents LLMs’ expanding role in core lexicographic processes: definition generation, example sentence creation, and sense division. ChatGPT definitions rival human-authored ones, though example sentence quality remains slightly lower. Fully automated generation offers natural language, low cost, and flexibility but suffers from terminology drift, ambiguous boundaries, and hallucinations. Systematic evaluations confirm that successive model updates (GPT-3 to GPT-4) improve compilation quality, and specialized systems such as an automatic example generator for polysemous words and OxfordEval for large-scale sentence assessment have emerged.

The field has progressed toward “post-editing lexicography,” where human editors refine automatically generated content. Human-machine collaboration must foreground human leadership, interdisciplinary oversight, and critical interaction. Despite gains, LLM output still fails to meet editorial standards, necessitating substantial post-editing, especially for culturally sensitive expressions. Future lexicography must therefore emphasize production efficiency, data reuse, and technological integration while redefining human roles in guidance and refinement.

In Chinese lexicography, preliminary studies assessed ChatGPT’s ability to compile entries in Modern Chinese Dictionary style; 37.5 % of generated entries gained majority approval, indicating potential yet clear quality gaps. Other work successfully generated standardized definitions for Chinese neologisms via prompt learning. However, research remains scarce compared with English, and no study has addressed outward-oriented learner dictionaries for Chinese as a second language, highlighting the research gap this paper addresses.

3. Methodology

This chapter described a mixed-methods evaluation contrasting ChatGPT-4o and the human-edited Juzi Chinese learner’s dictionary across 60 stratified target words drawn from the official proficiency standards. Quantitative and qualitative criteria assessed definition accuracy, clarity, and example quality, aiming to map LLM capabilities and limitations in lexicography. ChatGPT-4o, OpenAI’s latest release, was chosen for its improved polysemy handling and contextual reasoning, while Juzi, curated by Beijing Language and Culture University, exemplified expert human practice optimized for digital learners.

The 60-item sample balanced elementary, intermediate, and advanced levels and included monosemous, polysemous, and idiomatic items across major parts of speech. A zero-shot prompt instructed ChatGPT to act as a lexicographer, restricting metalanguage and aligning output with learner vocabulary; it requested part of speech, concise sense definitions, and three varied examples for each sense. Generation proceeded word-by-word under identical prompt conditions. Juzi entries were extracted verbatim, limiting examples to the first three when excess sentences existed, ensuring dataset uniformity for subsequent comparative analysis.

4. Results

This chapter assessed definition quality for 60 sample Chinese words by comparing ChatGPT output against the human-curated Juzi dictionary, focusing on accuracy and comprehensibility. Accuracy was judged against the Modern Chinese Dictionary (7th Edition) and revealed five recurring error types in ChatGPT: missing senses, incorrect or imprecise semantics, wrong part-of-speech labels, redundant senses, and circular synonym-based explanations. The most frequent failure was omission of senses for polysemous items; for “import” the trade sense was absent, and for “forever” the sense “permanent future” disappeared. POS mislabeling appeared for abstract or idiomatic units—ChatGPT tagged “once” as a conjunction, “as for” as a conjunction, and “that is to say” as an adverb. Redundant senses created verbosity, while synonym strings like “anxiety, unease, worry” obscured fine distinctions. Juzi definitions, by contrast, kept senses discrete, POS labels consistent, and wording concise.

Error frequency was mapped across monosemous, polysemous, and idiomatic items. Polysemous words carried 32 of the 60 recorded issues, with missing senses alone accounting for 14. Monosemous words accumulated only 10 issues, mostly semantic inaccuracies. Idiomatic phrases showed no fully accurate entries because POS tagging failed in every case. Overall, 21 definitions were fully accurate—16 monosemous (59.3 %), 5 polysemous (17.2 %), and 0 idiomatic. When the same errors were grouped by grammatical category, content words suffered chiefly from missing senses (40.6 %), semantic imprecision (28.1 %), and redundancy (18.8 %). Function-word errors clustered around semantic imprecision (66.7 %), while hybrid and idiomatic items failed on POS assignment.

Comprehensibility was gauged with a graded readability system calibrated to the Chinese Proficiency Grading Standards. ChatGPT definitions scored higher difficulty (Level 5 to Level 9) than Juzi definitions, which stayed at Level 4 regardless of headword complexity. Juzi restricted defining vocabulary to 3,181 unique items, 96.6 % of which are elementary or intermediate, whereas ChatGPT introduced higher-level lexemes such as “anxiety” to explain “worry” and “occasionally” to explain “occasionally.” Concrete examples illustrated the contrast: for “worry,” ChatGPT offered “feel uneasy or anxious,” embedding a Level 5 word; Juzi used “feel uncertain and uneasy,” couched in Level 1–3 vocabulary.

2. “adj. Not often happening.”

This chapter discussed the comparative quality of ChatGPT-generated content versus human-curated entries in the Juzi learner’s dictionary, focusing on definitions and example sentences. ChatGPT’s definition for “adj. Not often happening” was found to employ lexis and phrasing that exceeded the cognitive load typical for intermediate learners, rendering the entry less pedagogically useful. Quantitative comparison via the Jaccard coefficient revealed a mere 28 % lexical overlap between ChatGPT and Juzi definitions, underscoring divergent choices in wording and conceptual framing.

Example sentences were evaluated along four axes—accuracy, comprehensibility, practicality, and diversity—and systematically contrasted with Juzi counterparts.

Accuracy: GPT-produced sentences exhibited multiple recurrent flaws. The most common was outright misuse of the target word (e.g., “光荣” interpreted as “嘉奖”). Additional problems included transitive misapplication of intransitive verbs, unnatural or ambiguous phrasing, omission of the headword itself, and occasional collocation errors or sentence fragments. These inaccuracies appeared uncorrelated with the polysemy or grammatical class of the word, pointing to stochastic generation rather than principled error patterns.

Comprehensibility: Using an automated readability metric, GPT sentences scored higher difficulty levels than Juzi sentences across all proficiency bands, even though GPT sentences were shorter and less lexically varied. The gap between GPT definitions and GPT examples was also wider, indicating inconsistent control of linguistic complexity. Juzi mitigates difficulty by explicitly encoding grammatical frames (“forms”) before contextual examples, converting implicit grammatical knowledge into overt schemata that scaffold learner interpretation.

Practicality and Diversity: While GPT examples often portray everyday situations, they tend to be bland and repetitive. Juzi, leveraging corpus-driven Pattern Grammar, supplies concise yet vivid examples that span sports, travel, life planning, and other authentic domains. Sentence-type distribution in GPT skews 97.5 % toward declaratives, whereas Juzi incorporates interrogatives, imperatives, and exclamatives, achieving richer syntactic variety. Collocational breadth in Juzi is systematically expanded through high-frequency constructions extracted from corpora, covering both formal and informal registers.

5. Discussion

This chapter discussed the comparative quality between ChatGPT-generated entries and the manually compiled Juzi dictionary, examined how the model’s performance varied across different word types, and evaluated its feasibility and limitations for outward-oriented Chinese learner’s dictionary production. ChatGPT fell short of human standards in four quality dimensions: accuracy, comprehensibility, practicality, and diversity. Definitions suffered from missing senses, incorrect or circular explanations, POS mislabelling, redundancy, and only 35 % were fully accurate, whereas Juzi maintained near-perfect precision. Readability analysis showed ChatGPT employed more complex diction, producing lower comprehensibility scores; the Jaccard coefficient of 0.28 indicated minimal lexical overlap with Juzi’s deliberately simple metalanguage. Example sentences displayed incorrect usage, unnatural phrasing, absent target words, poor collocations, and truncated forms. They also leaned heavily on declarative structures, offered limited register coverage, and lacked Pattern-Grammar scaffolding, whereas Juzi balanced sentence types and explicitly highlighted productive patterns.

The investigation further revealed that ChatGPT’s weaknesses intensified with linguistic complexity. Polysemous words triggered 69.6 % of all definitional errors, with “missing senses” dominating; high-frequency polysemes such as 打, 上, and 把 retained only a fraction of their documented meanings. Monosemous items produced far fewer problems, mainly minor inaccuracies or redundancy. Content words generated more issues overall, but function words provoked proportionally severe definition errors and POS tagging failures. Difficulty level did not affect definition accuracy, yet both definitions and examples grew linguistically harder as the lemma complexity increased, violating learner-dictionary principles that require controlled metalanguage.

Despite these shortcomings, ChatGPT offers tangible benefits. It can rapidly generate complete draft entries and extensive example banks, cutting time and labour costs dramatically. Its contextual breadth supplies varied usage scenarios, registers, and even short dialogues, surpassing the coverage of many traditional dictionaries. Multilingual competence supports bilingual or multilingual compilation, and continual fine-tuning on updated corpora promises incremental quality gains. Nevertheless, current limitations—semantic omissions, grammatical imprecision, opaque phrasing, and failure to align linguistic complexity with learner needs—make full automation premature. Human lexicographers remain indispensable for verification, simplification, and pedagogical adaptation, positioning ChatGPT as a high-speed assistant rather than a replacement.

6. Conclusion

This chapter discussed the comparative evaluation of ChatGPT against the manually compiled Juzi for Chinese learner lexicography. It revealed that while the model excels in generation speed, linguistic flexibility, and coverage of new words, it falls short on definition precision and exemplification, especially for polysemy, abstract function words, and learner-oriented metalanguage. Quantitative results showed low accuracy concentrated in sense differentiation. Nevertheless, ChatGPT’s strengths in multimodal, multilingual, and interactive features surpass traditional resources. Optimal performance occurs with concrete, monosemous lexical items. Future practice will shift toward human–AI collaboration, with lexicographers acting as prompt engineers and quality supervisors, merging AI literacy with linguistic expertise to maintain rigorous standards amid rapid language change.