7f9ef370b0f16084239c86e1449c706efadbb94e84fdd23e5082b7d3a98bcec46dcfdbf93c8a8856bfa367e63576e1d1aae019db

74e4f576b5f16bf4239ef4e2409c706d5734ff2caeb59a54b03bd95c9075cabdd64756f5e9d33f9b4c4dc4f90899cba96e94548a

Is GPT-3 a Psychopath? Evaluating Large Language Models from a Psychological PerspectiveAre large language models (LLMs) like GPT-3 psychologically safe? In thiswork, we design unbiased prompts to evaluate LLMs systematically from apsychological perspective. Firstly, we test the personality traits of threedifferent LLMs with Short Dark Triad (SD-3) and Big Five Inventory (BFI). Wefind all of them show higher scores on SD-3 than the human average, indicatinga relatively darker personality. Furthermore, LLMs like InstructGPT andFLAN-T5, which are fine-tuned with safety metrics, do not necessarily have morepositive personalities. They score higher on Machiavellianism and Narcissismthan GPT-3. Secondly, we test the LLMs in GPT-3 series on well-being tests tostudy the impact of fine-tuning with more training data. Interestingly, weobserve a continuous increase in well-being scores from GPT-3 to InstructGPT.Following the observations, we show that instruction-finetune FLAN-T5 withpositive answers in BFI can effectively improve the model from a psychologicalperspective. Finally, we call on the community to evaluate and improve LLMs'safety systematically instead of at the sentence level only.arxiv.org