1 Introduction

We discussed the advancements in artificial intelligence and natural language processing technologies, particularly the achievements of Large Language Models (LLMs) in understanding and generating natural language. Liu et al. (2023) mentioned the evaluation of ChatGPT in different fields, pointing out that current research focuses more on the application effectiveness of LLM and lacks evaluation of its disciplinary expertise. This study aims to comprehensively evaluate the professional competence of LLM in the core framework of cognitive linguistics, "Conceptual Metaphor Theory," and will conduct a systematic assessment from four levels: knowledge, understanding, application, and critical thinking.

Research on the Evaluation of ChatGPT Application Capability

Discussed the performance and limitations of ChatGPT in natural language processing tasks. Research has shown that ChatGPT performs well in text generation and comprehension, dialogue question answering, and other areas, but there are shortcomings in logical reasoning, non textual reasoning, and common sense reasoning. In terms of data annotation, ChatGPT performs close to human level and can effectively assist in corpus annotation work. However, as a closed source product, ChatGPT has deficiencies in transparency, reproducibility, and other aspects. In addition, ChatGPT's understanding of natural language is still insufficient, especially in metaphor analysis. At present, research on ChatGPT mainly focuses on its basic abilities, with limited research on metaphor analysis. This article will systematically evaluate ChatGPT's professional competence in the core theory of cognitive linguistics, "Conceptual Metaphor Theory".

Basic Principles of Conceptual Metaphor Theory

Conceptual Metaphor Theory (CMT) is a core theory in cognitive linguistics, proposed by Lakoff and Johnson, which holds that metaphor is a fundamental cognitive system structure of thought patterns and concepts. This theory describes the conceptual mapping from the source domain to the target domain, projecting the structured features of the source domain onto the target domain to form abstract concepts. Concept mapping is based on experiential experience and rooted in the interaction between humans and the environment. CMT reveals the core role of metaphor in cognition, demonstrating how language organizes and understands abstract concepts through concrete experiences. This theory has significant implications for fields such as linguistics, psychology, cognitive science, and artificial intelligence. This study takes CMT as an example to evaluate the professional competence of GPT-4 in the field of cognitive linguistics, particularly its ability to recognize implicit metaphorical language and abstract conceptual metaphors.

The creation mechanism and inherent limitations of ChatGPT

Introduced the creation mechanism of ChatGPT, including its LLM feature based on GPT, generative model, Transformer architecture, self attention mechanism, as well as pre training methods and fine-tuning process. Emphasis was placed on GPT's ability to understand and generate natural language, as well as its "emergent" abilities such as dialogue, reasoning, and cross task generalization. At the same time, the inherent limitations of ChatGPT were pointed out, such as non interpretability, data opacity, real-time data issues, controversies over "emergence" capabilities, security issues, data reliability and legality issues, as well as the need for instruction design optimization. We discussed the data update limitations of GPT-4-turb, the need for real-time information access, challenges in understanding the "emergence" capability, possible deception behavior of LLM, data reliability and legality issues, and the impact of instruction design on output results. Finally, the limitations that researchers need to recognize when using GPT-4 for cognitive linguistics research were emphasized, and appropriate measures were taken to ensure the reliability and effectiveness of the research results.

5 GPT-4 Cognitive Linguistics Professional Ability Assessment

This chapter provides a detailed evaluation of GPT-4's performance in the field of cognitive linguistics, particularly focusing on the evaluation of conceptual metaphor theory. The evaluation framework is based on Bloom's cognitive goal classification, which divides cognitive goals into four levels: knowledge, understanding, application, and critical thinking. The evaluation task design covers different levels of difficulty from basic understanding to complex applications.

6 Summary

A comprehensive evaluation was conducted on the cognitive linguistics professional competence of GPT-4 within the framework of conceptual metaphor theory, including four levels of knowledge, understanding, application, and critical thinking. Research has found that GPT-4 has inadequate understanding of core terminology at the knowledge level and an outdated knowledge system; The ability to understand theoretical frameworks at the level of comprehension is also inadequate; At the application level, it performs well in recognizing metaphorical language, but has poor ability to extract conceptual metaphors; At the level of critical thinking, the output results are general and do not touch on core issues. Overall, GPT-4 performs poorly in handling abstract and complex tasks, with limitations such as reliance on pre trained data, performance fluctuations, and lack of autonomous reasoning ability. It does not reach the level of primary cognitive research linguists and requires manual verification when used.