English
Received 29.08.2024, Revised 20.01.2025, Accepted 26.02.2025
This study explored innovative approaches to enhancing memory systems in large language models to improve efficiency and automate software development. The primary focus was on optimising memory systems that enable long-term context storage and facilitate model adaptation to evolving interaction conditions. The research analysed contemporary methods of data storage and processing that enhance the ability of models to handle large volumes of information efficiently. This included the utilisation of specialised algorithms and memory mechanisms that improve the accuracy and adaptability of large language models in executing complex tasks. A secondary focus of the study examined the capabilities of large language models in automating software development. It assessed how these models can generate code, optimise it, and perform error detection. Particular attention was given to analysing the impact of automation on software quality and development time. In this context, the study investigated the use of large language models for automating repetitive tasks, generating tests, and implementing best programming practices. The findings indicated that enhancing the memory systems of large language models significantly improves their efficiency in tasks requiring longterm interaction. Integrating such models into software development processes has been shown to reduce both time and resource expenditures while enhancing product quality. The practical significance of this study lies in the formulation of recommendations for the optimal utilisation of large language models in the field of information technology
contextual processing; algorithm optimisation; code generation; intelligent agents; big data processing; automation; interaction modelling
[1] Alenezi, M., & Akour, M. (2025). AI-Driven innovations in software engineering: A review of current practices and future directions. Applied Sciences, 15(3), article number 1344. doi: 10.3390/app15031344.
[2] Antero, U., Blanco, F., Onativia, J., & Salle, D. (2024). Harnessing the power of large language models for automated code generation and verification. Robotics, 13(9), article number 137. doi: 10.3390/robotics13090137.
[3] Bajaj, Y., & Samal, M.K. (2023). Accelerating software quality: Unleashing the power of generative AI for automated test-case generation and bug identification. International Journal for Research in Applied Science and Engineering Technology, 11(7), 345-350. doi: 10.22214/ijraset.2023.54628.
[4] Chalumeau, F., Shabe, R., de Nicola, N., Pretorius, A., Barrett, T.D., & Grinsztajn, N. (2024). Memory-enhanced neural solvers for efficient adaptation in combinatorial optimization. doi: 10.48550/arXiv.2406.16424.
[5] Cheng, J., Fleming, S., Chen, Y.T., Anderson, J., Wickerson, J., & Constantinides, G.A. (2021). Efficient memory arbitration in high-level synthesis from multi-threaded code. IEEE Transactions on Computers, 71(4), 933-946. doi: 10.1109/tc.2021.3066466.
[6] Cruz-Benito, J., Sánchez-Prieto, J.C., Therón, R., & García-Peñalvo, F.J. (2019). Measuring students’ acceptance to AI-Driven assessment in eLearning: Proposing a first TAM-Based research model. In P. Zaphiris & A. Ioannou (Eds.), 6th international conference: Learning and collaboration technologies. Designing learning experiences (pp. 15-25). Cham: Springer. doi: 10.1007/978-3-030-21814-0_2.
[7] Dieu, A.N.T., Nguyen, H.T., & Cong, C.T.D. (2024). The enhanced context for AI-generated learning advisors with Advanced RAG. In L.-S. Lê, M. Kappes & J. Küng (Eds.), 18th international conference on advanced computing and analytics (pp. 94-101). Ben Cat: Institute of Electrical and Electronics Engineers. doi: 10.1109/ACOMPA64883.2024.00021.
[8] Dolcetti, G., Arceri, V., Iotti, E., Maffeis, S., Cortesi, A., & Zaffanella, E. (2024). Helping LLMs improve code generation using feedback from testing and static analysis. doi: 10.48550/arxiv.2412.14841.
[9] Görmez, M.K., Yılmaz, M., & Clarke, P.M. (2024). Large language models for software engineering: A systematic mapping study. In M. Yilmaz, P. Clarke, A. Riel, R. Messnarz, C. Greiner & T. Peisl (Eds.), 31st European conference: Systems, software and services process improvement (pp. 64-79). Cham: Springer. doi: 10.1007/978-3-031-71139-8_5.
[10] He, R., Ying, A., & Hu, X. (2024). Improving OpenDevin: Boosting code generation LLM through advanced memory management. Applied and Computational Engineering, 68(1), 320-327. doi: 10.54254/2755-2721/68/20241506.
[11] Hoda, R., Dam, H., Tantithamthavorn, C., Thongtanunam, P., & Storey, M. (2023). Augmented agile: Human-centered AI-assisted software management. IEEE Software, 40(4), 106-109. doi: 10.1109/MS.2023.3268725.
[12] Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024). Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology, 33(8), article number 220. doi: 10.1145/3695988.
[13] Jalil, S. (2023). The transformative influence of large language models on software development. doi: 10.48550/ arXiv.2311.16429.
[14] Jiang, X., Dong, Y., Wang, L., Fang, Z., Shang, Q., Li, G., Jin, Z., & Jiao, W. (2024). Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology, 33(7), article number 182. doi: 10.1145/3672456.
[15] Julien, L.A.S., Tang, X., & Gaillardon, P. (2023). Innovative memory architectures using functionality enhanced devices. In M.M.S. Aly & A. Chattopadhyay (Eds.), Emerging computing: From devices to systems: Looking beyond moore and von neumann (pp. 47-83). Singapore: Springer. doi: 10.1007/978-981-16-7487-7_.
[16] Kim, B., et al. (2024). The breakthrough memory solutions for improved performance on LLM inference. IEEE Micro, 44(3), 40-48. doi: 10.1109/MM.2024.3375352.
[17] Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L. Yu, C.H., Gonzales, J., Zhang, H., & Stoica, I. (2023). Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles (pp. 611-626). New York: Association for Computing Machinery. doi: 10.1145/3600006.3613165.
[18] Liu, B., Jiang, Y., Zhang, Y., Niu, N., Li, G., & Liu, H. (2024). An empirical study on the potential of LLMs in automated software refactoring. doi: 10.48550/arXiv.2411.04444.
[19] Lyu, Z., Li, X., Xie, Z., & Li, M. (2025). Top Pass: Improve code generation by pass@k-maximized code ranking. Frontiers of Computer Science, 19(8), article number 198341. doi: 10.1007/s11704-024-40415-9.
[20] Marvin, G., Hellen, N., Jjingo, D., & Nakatumba-Nabende, J. (2024). Prompt engineering in large language models. In I.J. Jacob, S. Piramuthu & P. Falkowski-Gilski (Eds.), Proceedings of ICDICI 2023: Data intelligence and cognitive informatics (pp. 387-402). Singapore: Springer. doi: 10.1007/978-981-99-7962-2_30.
[21] Nguyen, D., Yang, W., Anand, R., Yang, Y., & Mirzasoleiman, B. (2024). Mini-batch coresets for memory-efficient training of large language models. doi: 10.48550/arXiv.2407.19580.
[22] Ozkaya, I. (2023). Application of large language models to software engineering tasks: Opportunities, risks, and implications. IEEE Software, 40(3), 4-8. doi: 10.1109/MS.2023.3248401.
[23] Park, J., & Sung, H. (2023). XLA-NDP: Efficient scheduling and code generation for Deep Learning model training on Near-Data Processing Memory. IEEE Computer Architecture Letters, 22(1), 61-64. doi: 10.1109/LCA.2023.3261136.
[24] Rostam, Z.R.K., Szénási, S., & Kertész, G. (2024). Achieving peak performance for large language models: A systematic review. IEEE Access, 12, 96017-96050. doi: 10.1109/access.2024.3424945.
[25] Sagi, S. (2024). Advancing AI: Enhancing large language model performance through GPU optimization techniques. International Journal of Science and Research, 13(3), 630-633. doi: 10.21275/sr24309100709.
[26] Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2023). An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50(1), 85-105. doi: 10.1109/tse.2023.3334955.
[27] Shantyr, A. (2024). Specifics of quality assessment models application at development and use stages of software systems. Information Technologies and Computer Engineering, 21(1), 127-138. doi: 10.31649/1999-9941-2024-591-127-138.
[28] van Viet, N., & Vinh, N. (2024). Large language models in software engineering. Journal of Education for Sustainable Innovation, 2(2), 146-156. doi: 10.56916/jesi.v2i2.968.
[29] Wu, K. (2024). Code generation and runtime techniques for enabling data efficient deep learning training on GPUs. Urbana: University of Illinois Urbana-Champaign. doi: 10.13140/RG.2.2.15485.47840.
[30] Wu, Q., Lan, Z., Gu, J., Geramifard, A., & Yu, Z. (2020). Memformer: The memory-augmented transformer. doi: 10.48550/ arXiv.2010.06891.
[31] Xu, F.F., Alon, U., Neubig, G., & Hellendoorn, V.J. (2022). A systematic evaluation of large language models of code. In Proceedings of the 6th ACM sigplan international symposium on machine programming (pp. 1-10). New York: Association for Computing Machinery. doi: 10.1145/3520312.3534862.
[32] Zheng, Z., Ning, K., Zhong, Q., Chen, J., Chen, W., Guo, L., Wang, W., & Wang, Y. (2025). Towards an understanding of large language models in software engineering tasks. Empirical Software Engineering, 30, article number 50. doi: 10.1007/s10664-024-10602-0.
[33] Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2024). MemoryBank: Enhancing large language models with long-term memory. In Proceedings of the AAAI conference on artificial intelligence (pp. 19724-19731). Washington: Association for the Advancement of Artificial Intelligence doi: 10.1609/aaai.v38i17.29946.
[34] Zhu, M., Pan, P., Chen, W., & Yang, Y. (2019). DM-GAN: Dynamic memory generative adversarial networks for text-toimage synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 5802-5810). Long Beach: Institute of Electrical and Electronics Engineers.
Українська