Large Language Models (LLMs) have transformed natural language processing and generation activities in recent years. However, as the scale and complexity of these models grow, their ability to write correct and secure code has come under scrutiny. In our research, we delve into the critical examination of Large Language Models (LLMs) like ChatGPT-3.5 (text-davinci-002-render-sha), legacy Bard (PaLM 2), and Gemini (gemini-1.0-pro-001), and their proficiency in generating accurate and secure code, particularly focusing on the occurrence of identifier swaps within the code they produce. Our methodology encompasses the creation of a diverse dataset comprising a range of coding tasks designed to challenge the code generation capabilities of these models. Further, we employ Pylint for an extensive code quality assessment and undertake multi-shot identifier swap testing to evaluate the models’ ability to maintain context and coherence over sequential coding prompts. Our findings contribute to the understanding of LLMs’ performance\r\nin code generation, highlighting potential areas for improvement in ensuring the reliability and security of LLM-generated code in software development applications.