Skip to content
KPI Science News
ISSN 2617-5509 · e‑ISSN 2663-7472 Open Access · CC BY 4.0
Vol. 137 · Issue 1-4 · 2024 Dec 31, 2024 article

System Approach to the Combined Use of Large Language Models and Classical Models in Foresight Tasks

VS
Volodymyr Savastiyanov Corresponding National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, IASA, MMSA, Ukraine Ukraine
MS
Mykhailo Stoliar National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, IASA, MMSA, Ukraine Ukraine
PublishedDec 31, 2024
PDF (Ukrainian)
KPISN 1-4 VOL 137 · 1-4
VOL 137 · ISS 1-4 · 2024 View issue

Abstract

Background. Large Language Models (LLMs) and their associated agents have spread wide technology and represent a significant advancement in recent times. These state-of-the-art models expose valuable potential, but they are not devoid of restrictions, inefficiencies, and limits. This article investigates the exploration of these constraints within specific domain areas and prediction problems as examples. Objective. The article highlights features offered by GPT-based models and compares the conclusions with classical methods of textual data analysis in classification tasks using the prediction methodology as an example. The purpose of the study is to develop a system approach to the combined use of traditional machine learning approaches as a practical alternative to LLMs in foresight tasks using the example of STEEP analysis, which provides an opportunity to obtain valuable information from textual data. Methods. The study is structured into four segments, each addressing distinct parts: Data Mining, text pre-processing using LLMs, text pre-processing utilizing Natural Language Processing (NLP) methods, and comparative analysis of results. Data Mining includes data collection and data pre-processing stages for train and test observations. For the utilization of LLMs, chains of thought techniques and prompt engineering were used. Results. Throughout this study, it was acknowledged that the LLMs can be used in combination with classical machine learning methodologies for domain-specific areas in STEEP analysis under Foresight tasks. The outcome revealed a model that was developed significantly faster and with less complexity compared to LLMs such as GPT and Mistral. Increasing the number of models employed leads to more stable results. Conclusions. The main result of the proceeding is that the patterns that reveal LLMs under certain settings can also be identified by classical models. Moreover, augmenting the deployment of LLMs during the data preparation stages contributes to heightened stability in outcomes. Using classical models combined with LLMs speeds up response times during inference and reduces operating costs for running models.

Keywords

#system analysis #foresight #textual analytics #classification #LLM #NLP

References

  1. Li et al. “Knowledge structure of technology licensing based on co-keywords network: A review and future directions” International Review of Economics & Finance, vol. 66, pp. 154–165, 2020. doi: 10.1016/j.iref.2019.11.007.
  2. UNIDO, UNIDO Technology Foresight Manual, vol. 1. United Nations Industrial Development Organization, Vienna (2005).
  3. European Commission. 2020. “Strategic Foresight Report – Charting the course towards a more resilient Europe”. Available online at: https://ec.europa.eu/info/strategy/priorities-2019-2024/new-push-european-democracy/strategic-foresight/2020-strategic-foresight-report_en .
  4. Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu and Michael Zeng, “Want To Reduce Labeling Cost? GPT-3 Can Help”, arXiv preprint arXiv.2108.13487, 2023, doi: 10.48550/arXiv.2108.13487.
  5. P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pretrain, prompt, and predict: A systematic survey of prompting methods in natural language processing”, ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023.
  6. A. B. Rosa, N. Gudowsky, P. Repo “Repo Sensemaking and lens-shaping: Identifying citizen contributions to foresight through comparative topic modelling”, Futures 129, pp. 1–15, 2021, doi: 10.1016/j.futures.2021.102733.
  7. M. Vignoli, J. Rörden, D. Wasserbacher and S Kimpeler, “An Exploration of the Potential of Machine Learning Tools for Media Analysis to Support Sense-Making Processes in Foresight”, Frontiers in Communication 7:750614, 2022. doi :10.3389/fcomm.2022.750614.
  8. N. Pankratova, V. Savastiyanov, “Assessment of situations in the field of social disasters based on the methodology of foresight and textual analytics”. Proceedings of the 2019 IEEE. Second International Conference IEEE UKRCON-2019. pp. 1207–1210, ISBN 9781728138831.
  9. Callie Y. Kim, Christine P. Lee and Bilge Mutlu, “Understanding Large-Language Model (LLM)-powered Human-Robot Interaction”, In Proceedings of the 2024 ACM/IEEE, International Conference on Human-Robot Interaction (HRI ’24). Association for Computing Machinery, New York, NY, USA, pp. 371–380, 2024, doi: 10.1145/3610977.3634966.
  10. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  11. B. Zhang and H. Soh, “Large Language Models as Zero-Shot Human Models for Human-Robot Interaction”, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA, pp. 7961–7968, 2023, doi: 10.1109/IROS55552.2023.10341488.
  12. F. Pourpanah et al., “A Review of Generalized Zero-Shot Learning Methods”, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4051–4070, 1 April 2023, doi: 10.1109/TPAMI.2022.3191696.
  13. C. Mühlroth and M. Grottke, “Artificial Intelligence in Innovation: How to Spot Emerging Trends and Technologies”, in IEEE Transactions on Engineering Management, vol. 69, no. 2, pp. 493–510, April 2022, doi: 10.1109/TEM.2020.2989214.
  14. M. Stoliar and V. Savastiyanov, “Using LLM classification in foresight studies”: Scientific Collection “InterConf”, vol. 157, pp. 367–375, 2023.
  15. En Yu, Liang Zhao, Yana Wei, Jinrong Yang, Dongming Wu, Lingyu Kong, Haoran Wei, Tiancai Wang, Zheng Ge, Xiangyu Zhang and Wenbing Tao, “Merlin: Empowering Multimodal LLMs with Foresight Minds”, arXiv preprint arXiv:2312.00589, 2023, doi: 10.48550/arXiv.2312.00589.
  16. Y. Luo, S. Xu and C. Xie, “E-commerce Big Data Classification and Mining Algorithm based on Artificial Intelligence”, 2022 IEEE 2nd International Conference on Electronic Technology, Communication and Information (ICETCI), Changchun, China, pp. 1153–1155, 2022, doi: 10.1109/ICETCI55101.2022.9832370.
  17. Chan A., “GPT-3 and InstructGPT: technological dystopianism, utopianism, and “Contextual” perspectives in AI ethics and industry”, AI Ethics 3, pp. 53–64, 2023, doi: 10.1007/s43681-022-00148-6.
  18. Dave Wild, “Futurework – A Guidebook for The Future of Work”, Aotearoa New Zealand: Smith & Wild. 218 p., 2023, ISBN 978-0-473-66594-4.
  19. Mowafak Allaham and Nicholas Diakopoulos, “Supporting Anticipatory Governance using LLMs: Evaluating and Aligning Large Language Models with the News Media to Anticipate the Negative Impacts of AI”, arXiv preprint arXiv:2401.18028, 2024, doi: 10.48550/arXiv.2401.18028.
  20. Cambria Erik, Bebo White., “Jumping NLP curves: A review of natural language processing research”, IEEE Computational intelligence magazine 9.2, pp. 48–57, 2014, doi: 10.1109/MCI.2014.2307227.
  21. A. Jalilifard, V.F. Caridá, A.F. Mansano, R.S. Cristo, F.P.C. da Fonseca, “Semantic Sensitive TF-IDF to Determine Word Relevance in Documents”, Thampi, S.M., Gelenbe, E., Atiquzzaman, M., Chaudhary, V., Li, KC. (eds) Advances in Computing and Network Communications. Lecture Notes in Electrical Engineering, vol 736. Springer, Singapore, 2021, doi.org: 10.1007/978-981-33-6987-0_27.
  22. B. Ahmed, G. Ali, A. Hussain, A. Baseer and J. Ahmed, “Analysis of Text Feature Extractors using Deep Learning on Fake News”, Eng. Technol. Appl. Sci. Res., vol. 11, no. 2, pp. 7001–7005, Apr. 2021.
  23. Lucky Agarwal, Kartik Thakral, Gaurav Bhatt, and Ankush Mittal, “Authorship Clustering using TF-IDF weighted Word-Embeddings”, In Proceedings of the 11th Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE ’19). Association for Computing Machinery, New York, NY, USA, pp. 24–29, 2019, doi: 10.1145/3368567.3368572.
  24. Fanghui Liu, Xiaolin Huang, and Jie Yang, “Indefinite Kernel Logistic Regression”, In Proceedings of the 25th ACM International Conference on Multimedia (MM ’17). Association for Computing Machinery, New York, NY, USA, pp. 846–853, 2017, doi: 10.1145/3123266.3123295
§ 06 — Related

Similar articles in this journal

Related peer-reviewed studies published in this journal.
View all issues

AN APPROACH TO CONSTRUCTING AN INFORMATIVE FEATURE SPACE OF TECHNICAL INDICATORS FOR FINANCIAL PROCESSES BASED ON NEURAL NETWORK MODELS

Roman Tertychnyi, Denys Chuprin, Petro Bidyuk

Background. Financial time series are characterized by nonlinearity, non-stationarity, and a high level of noise, which significantly complicates their modeling and…

#informative feature space construction approach #technical indicators #Parkinson volatility #neural networks
10.20535/kpisn.2026.3.359610

Similar Articles

1-10 of 62

You may also start an advanced similarity search for this article.