Abstract :

Language models have revolutionized natural language processing by greatly improving text generation and comprehension. Optimizing their functioning is related to how one designs prompts because the kind and quality of response produced affects the nature of response that is generated. This article explores the impact of prompt length and specificity on AI chatbots’ capabilities concerning accuracy, fluency, and relevance of generated responses. We present evidence that careful prompt engineering is severely lacking to improve conversational performance, and illustrate this using studies and experiments on the Cornell Movie Dialogs Corpus; thus, providing interesting guidelines to the developers and researchers interested in improving chatbot responses

---

Keywords :

AI performance, Chatbot accuracy, language models, Prompt length, Prompt specificity.

---

References :

1. Hu, S., Ding, N., Wang, H., Liu, Z., Li, J., and Sun, M. Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification. CoRR, abs/2108.02035, (2021).
2. Jiang, Z., Xu, F. F., Araki, J., and Neubig, G. How Can We Know What Language Models Know? Transactions of the Association for Computational Linguistics, 8, 423–438, (2020).
3. Kojima, T., Gu, S. (Shane), Reid, M., Matsuo, Y., and Iwasawa, Y. Large language models are zero-shot reasoners. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Advances in Neural Information Processing Systems, 35, 22199–22213. Curran Associates, Inc., (2022).
4. Lester, B., Al-Rfou, , and Constant, N. The power of scale for parameter-efficient prompt tuning. CoRR, abs/2104.08691, (2021).
5. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Comput. Surv., 55(9), (2023).
6. Liu, X., Ji, K., Fu, Y., Tam, W. L., Du, Z., Yang, Z., and Tang, J. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, (2021).
7. Mungoli, N. Exploring the synergy of prompt engineering and reinforcement learning for enhanced control and responsiveness in chat gpt. Journal of Electrical Electronics Engineering, 2(3):201–205, (2023).
8. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, (2022).
9. Reynolds, L., and McDonell, K. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA ’21, New York, NY, USA, (2021). Association for Computing Machinery.
10. Shin, T., Razeghi, Y., Logan, R. L. IV, Wallace, E., and Singh, S. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, (2020).
11. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Advances in Neural Information Processing Systems, 35, 24824–24837. Curran Associates, Inc., (2022).
12. White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D. C. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382, (2023).
13. Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16816–16825, (2022).
14. Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348, (2022).
15. Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910, (2022).