1 Introduction

In November 2022, OpenAI released ChatGPT, which performed well in translation tasks. In March 2023, OpenAI will launch a more advanced GPT-4. The translation of Chinese political terms is crucial for the dissemination of national image. Based on the Chinese English parallel corpus of the white paper "Jointly Building the the Belt and Road: A Major Practice of Building a Community with a Shared Future for Humanity", this paper makes a comparative analysis of the quality of ChatGPT-3.5, ChatGPT-4, Google translation and human translation. The research focuses on evaluating the performance of ChatG model PT in the translation of Chinese political terms, and exploring its shortcomings and improvement suggestions in the translation of external propaganda texts, in order to unleash the potential of the big language model in the field of external propaganda.

A review of research on ChatGPT translation quality

Discussed the performance of ChatGPT in translation quality and its differences from traditional machine translation systems. ChatGPT relies on the quality of training data and the accuracy of instructions, and its translation ability is affected by language data resources and instruction engineering. Research has shown that ChatGPT performs well in high resource language translation and can generate diverse answers based on instructions. Detailed instructions can significantly improve its translation performance, even surpassing mainstream translation engines. In the general field, ChatGPT's translation quality is comparable to Google Translate and Huawei Translate, but its accuracy and formality are insufficient in the professional field. ChatGPT has advantages in lexical diversity and syntactic complexity, especially in constructing complex sentence structures. In the translation of Chinese characteristic discourse, ChatGPT outperforms other tools in terms of matching degree and implicit metaphor recognition, but has limitations in handling ideological and cultural loaded words. Although automatic evaluation indicators are easy to operate, they are not sensitive enough to deep details and semantic accuracy. Manual evaluation is more comprehensive but costly and subjective. The corpus based translation quality assessment model improves the credibility of conclusions through multidimensional analysis. This study will quantitatively and qualitatively analyze the performance of ChatGPT in the translation of external propaganda texts from the perspectives of vocabulary, sentences, and discourse.

3 Research Design

This paper introduces a corpus based research design aimed at quantitatively and qualitatively analyzing the quality of English translations of the White Paper by ChatGPT-3.5, ChatGPT-4, Google Translate, and professional human translators. The research questions include the comparison of translation quality between ChatGPT and manual and Google translation, the way ChatGPT handles Chinese political terminology and typical cases, and the problems and improvement methods of ChatGPT in the English translation of the White Paper. The research text is the English version of "Jointly Building the the Belt and Road: Major Practice of Building a Community with a Shared Future for Human Beings", which was chosen because its release time is later than the completion time of model training, and it is representative as an official document. The research methods include Q&A interaction with ChatGPT, input of long text using Superpower ChatGPT extension, alignment of sentence segments using Tmxmall platform, and analysis of vocabulary and discourse level data using various tools.

4 Results and Discussion

In the "Results and Discussion" section of the paper, a detailed analysis was conducted on the vocabulary level of the Chinese original text and four different translations (official manual translation, ChatGPT-3.5 translation, ChatGPT-4 translation, Google Translate translation) of the White Paper. Statistics show that manual translation has the highest vocabulary richness, followed by ChatGPT-4, while Google Translate has the lowest vocabulary richness. In terms of average word length and word complexity, the translations of ChatGPT-3.5 and ChatGPT-4 are slightly higher than manual translations, indicating that machine translation is more complex in word usage. In terms of vocabulary density, manual translation has the richest content expression, with ChatGPT-4 having a higher vocabulary density than ChatGPT-3.5, while Google Translate has the lowest vocabulary density.

5 Conclusion and Implications

This study conducted a qualitative and quantitative analysis of the translation quality of the White Paper by ChatGPT-3.5, ChatGPT-4, and Google Translate using corpus methods. Research has found that ChatGPT-3.5 and ChatGPT-4 outperform Google Translate in vocabulary usage, with ChatGPT-4 having richer and more complex vocabulary expressions. Artificial translation has slightly richer vocabulary usage than ChatGPT translation and reduces the use of difficult words. Despite the complexity of the original text, the translation generated by ChatGPT significantly reduces readability compared to manual translation. ChatGPT can correctly understand text content and objectively convey emotional information, but when translating Chinese political terminology, some terms are translated literally and fail to achieve semantic equivalence. The study also found that the phenomenon of "illusion" in generative artificial intelligence is reflected in ChatGPT-3.5's omission of a very small number of sentence segments.