Insights into the Development Trends of Industrial Large Language Models
Article Sidebar
Main Article Content
Abstract
In recent years, Large Language Models (LLMs) with massive parameters and complex structures have achieved significant breakthroughs in fields such as natural language processing and image generation, driving their widespread application in industrial sectors. Despite the enormous potential of industrial AI models in areas like design and development, monitoring and management, quality control, and maintenance, their actual construction and deployment still face a lot of challenges, including inherent model deficiencies and difficulties in aligning with industrial requirements. Future technological development trends include the generation of customized industrial datasets, the collaborative optimization of large and small models, the enhancement of adaptive capabilities, and the application of Retrieval-Augmented Generation (RAG) technology. These trends are expected to improve the effectiveness and scalability of AI models, better meeting the needs of the industrial domain. This paper systematically discusses the challenges, technological development trends, and practical applications and deployment of industrial AI models, providing valuable insights for future directions.
Downloads
Article Details
Copyright (c) 2024 Lai Y, et al.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Licensing and protecting the author rights is the central aim and core of the publishing business. Peertechz dedicates itself in making it easier for people to share and build upon the work of others while maintaining consistency with the rules of copyright. Peertechz licensing terms are formulated to facilitate reuse of the manuscripts published in journals to take maximum advantage of Open Access publication and for the purpose of disseminating knowledge.
We support 'libre' open access, which defines Open Access in true terms as free of charge online access along with usage rights. The usage rights are granted through the use of specific Creative Commons license.
Peertechz accomplice with- [CC BY 4.0]
Explanation
'CC' stands for Creative Commons license. 'BY' symbolizes that users have provided attribution to the creator that the published manuscripts can be used or shared. This license allows for redistribution, commercial and non-commercial, as long as it is passed along unchanged and in whole, with credit to the author.
Please take in notification that Creative Commons user licenses are non-revocable. We recommend authors to check if their funding body requires a specific license.
With this license, the authors are allowed that after publishing with Peertechz, they can share their research by posting a free draft copy of their article to any repository or website.
'CC BY' license observance:
|
License Name |
Permission to read and download |
Permission to display in a repository |
Permission to translate |
Commercial uses of manuscript |
|
CC BY 4.0 |
Yes |
Yes |
Yes |
Yes |
The authors please note that Creative Commons license is focused on making creative works available for discovery and reuse. Creative Commons licenses provide an alternative to standard copyrights, allowing authors to specify ways that their works can be used without having to grant permission for each individual request. Others who want to reserve all of their rights under copyright law should not use CC licenses.
Chen M, Tworek J, Jun H, Yuan Q, Ponde de Oliveira Pinto H, Kaplan J, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. 2021. Available from: https://doi.org/10.48550/arXiv.2107.03374
Li R, Allal LB, Zi Y, Muennighoff N, Kocetkov D, Mou C, et al. Starcoder: May the source be with you! arXiv preprint arXiv:2305.06161. 2023. Available from: https://doi.org/10.48550/arXiv.2305.06161
Roziere B, Gehring J, Gloeckle F, Sootla S, Gat I, Tan XE, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950. 2023. Available from: https://doi.org/10.48550/arXiv.2308.12950
Zhao Z, Ma D, Chen L, Sun L, Li Z, Xu H, et al. Chemdfm: Dialogue foundation model for chemistry. arXiv preprint arXiv:2401.14818. 2024. Available from: https://doi.org/10.48550/arXiv.2401.14818
Frey NC, Soklaski R, Axelrod S, Samsi S, Gómez-Bombarelli R, Coley CW, et al. Neural scaling of deep chemical models. Nat Mach Intell. 2023;5(11):1297-1305. Available from: https://www.nature.com/articles/s42256-023-00740-3
Wang P, Wei X, Hu F, Han W. Transcript: Multi-modal generative pre-trained transformer for transportation. arXiv preprint arXiv:2402.07233. 2024. Available from: https://doi.org/10.48550/arXiv.2402.07233
Ni S, Tan M, Bai Y, Niu F, Yang M, Zhang B, et al. MoZIP: A multilingual benchmark to evaluate large language models in intellectual property. arXiv preprint arXiv:2402.16389. 2024. Available from: https://aclanthology.org/2024.lrec-main.1018/
Wang L, Chou J, Zhou X, Tien A, Baumgartner DM. AviationGPT: A large language model for the aviation domain. In: Aiaa Aviation Forum and Ascend 2024. 2024:4250. Available from: https://doi.org/10.48550/arXiv.2311.17686
Zhang H, Dereck SS, Wang Z, Lv X, Xu K, Wu L, et al. Large scale foundation models for intelligent manufacturing applications: A survey. arXiv e-prints. 2023. Available from: https://doi.org/10.48550/arXiv.2312.06718
Xi Z, Chen W, Guo X, He W, Ding Y, Hong B, et al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864. 2023. Available from: https://doi.org/10.48550/arXiv.2309.07864
Kaddour J, Harris J, Mozes M, Bradley H, Raileanu R, McHardy R. Challenges and applications of large language models. arXiv preprint arXiv:2307.10169. 2023. Available from: https://doi.org/10.48550/arXiv.2307.10169
Fan T, Kang Y, Ma G, Chen W, Wei W, Fan L, et al. Fate-llm: An industrial grade federated learning framework for large language models. arXiv preprint arXiv:2310.10049. 2023. Available from: https://doi.org/10.48550/arXiv.2310.10049
Tinnes C, Welter A, Apel S. Leveraging large language models for software model completion: Results from industrial and public datasets. arXiv preprint arXiv:2406.17651. 2024. Available from: https://doi.org/10.48550/arXiv.2406.17651
Golde J, Haller P, Hamborg F, Risch J, Akbik A. Fabricator: An open source toolkit for generating labeled training data with teacher LLMs. arXiv preprint arXiv:2309.09582. 2023. Available from: https://doi.org/10.48550/arXiv.2309.09582
Zhao Y, Zhang H, Si S, Nan L, Tang X, Cohan A. Investigating table-to-text generation capabilities of large language models in real-world information seeking scenarios. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track. 2023. Available from: https://doi.org/10.48550/arXiv.2305.14987
Feng W, Zhu W, Fu T, Jampani V, Akula A, He X, et al. LayoutGPT: Compositional visual planning and generation with large language models. Adv Neural Inf Process Syst. 2024;36. Available from: https://doi.org/10.48550/arXiv.2305.15393
Wan Y, Liu Y, Ajith A, Grazian C, Hoex B, Zhang W, et al. SciQAG: A framework for auto-generated scientific question answering dataset with fine-grained evaluation. arXiv preprint arXiv:2405.09939. 2024. Available from: https://doi.org/10.48550/arXiv.2405.09939
Devidze R, Radanovic G, Kamalaruban P, Singla A. Explicable reward design for reinforcement learning agents. Adv Neural Inf Process Syst. 2021;34:20118-20131. Available from: https://proceedings.neurips.cc/paper/2021/hash/a7f0d2b95c60161b3f3c82f764b1d1c9-Abstract.html
Anand R, Aggarwal D, Kumar V. A comparative analysis of optimization solvers. J Statist Manag Syst. 2017;20(4):623-635. Available from: http://dx.doi.org/10.1080/09720510.2017.1395182
Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, et al. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712. 2023. Available from: https://doi.org/10.48550/arXiv.2303.12712
Xian Y, Lampert CH, Schiele B, Akata Z. Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell. 2018;41(9):2251-2265. Available from: https://doi.ieeecomputersociety.org/10.1109/TPAMI.2018.2857768
Beltagy I, Lo K, Cohan A. SciBERT: A pre-trained language model for scientific text. arXiv preprint arXiv:1903.10676. 2019. Available from: https://doi.org/10.48550/arXiv.1903.10676
Salemi A, Zamani H. Evaluating retrieval quality in retrieval-augmented generation. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM; 2024;21. 2024:2395–2400. Available from: https://doi.org/10.1145/3626772.3657957
Chen J, Zhang R, Guo J, Fan Y, Cheng X. GERE: Generative evidence retrieval for fact verification. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval; 2022. ACM; 2022;2184-2189. Available from: https://doi.org/10.1145/3477495.3531827
Li C, Yates A, MacAvaney S, He B, Sun Y. PARADE: Passage representation aggregation for document reranking. ACM Trans Inf Syst. 2024;42(2):1–26. Available from: https://doi.org/10.1145/3600088
Hu Z, Iscen A, Sun C, Wang Z, Chang KW, Sun Y, et al. Reveal: Retrieval-augmented visual-language pre-training with multi-source multimodal knowledge memory. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2023. IEEE; 2023:23369–23379. Available from: https://openaccess.thecvf.com/content/CVPR2023/supplemental/Hu_REVEAL_Retrieval-Augmented_Visual-Language_CVPR_2023_supplemental.pdf
Yasunaga M, Aghajanyan A, Shi W, James R, Leskovec J, Liang P, et al. Retrieval-augmented multimodal language modeling. arXiv preprint arXiv:2211.12561. 2022. Available from: https://doi.org/10.48550/arXiv.2211.12561
Li J, Li D, Savarese S, Hoi S. BLIP-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597. 2023. Available from: https://doi.org/10.48550/arXiv.2301.12597
Zhu W, Yan A, Lu Y, Xu W, Wang XE, Eckstein M, et al. Visualize before you write: Imagination-guided open-ended text generation. arXiv preprint arXiv:2210.03765. 2022. Available from: https://doi.org/10.48550/arXiv.2210.03765
Anil C, Durmus E, Sharma M, Benton J, Kundu S, Batson J, et al. Many-shot jailbreaking. Anthropic. April 2024. Available from: https://www-cdn.anthropic.com/af5633c94ed2beb282f6a53c595eb437e8e7b630/Many_Shot_Jailbreaking__2024_04_02_0936.pdf
Xu Y, Wang W. LinkPrompt: Natural and universal adversarial attacks on prompt-based language models. In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2024:1;6473-6486. Available from: https://doi.org/10.18653/v1/2024.naacl-long.360
Qian C, Cong X, Liu W, Yang C, Chen W, Su Y, et al. ChatDev: Communicative agents for software development. arXiv preprint arXiv:2307.07924. 2023. Available from: https://doi.org/10.48550/arXiv.2307.07924
Hong S, Zhuge M, Chen J, Zheng X, Cheng Y, Zhang C, et al. MetaGPT: Meta programming for a multi-agent collaborative framework. arXiv preprint arXiv:2308.00352. 2023. Available from: https://doi.org/10.48550/arXiv.2308.00352
Rasheed Z, Waseem M, Saari M, Systa K, Abrahamsson P. Codepori: Large scale model for autonomous software development by using multi-agents. arXiv preprint arXiv:2402.01411. 2024. Available from: https://doi.org/10.48550/arXiv.2402.01411
He Z, Wu H, Zhang X, Yao X, Zheng S, Zheng H, et al. ChatEDA: A large language model powered autonomous agent for EDA. In: Proceedings of the 5th ACM/IEEE Workshop on Machine Learning for CAD; 2023:1-6. Available from: https://doi.org/10.48550/arXiv.2308.10204
Liffiton M, Sheese BE, Savelka J, Denny P. CodeHelp: Using large language models with guardrails for scalable support in programming classes. In: Proceedings of the 23rd Koli Calling International Conference on Computing Education Research; 2023:8:1-11. Available from: https://doi.org/10.1145/3631802.3631830
Deng G, Liu Y, Mayoral-Vilches V, Liu P, Li Y, Xu Y, et al. PentestGPT: An LLM-empowered automatic penetration testing tool. arXiv preprint arXiv:2308.06782. 2023. Available from: https://doi.org/10.48550/arXiv.2308.06782
Xia Y, Shenoy M, Jazdi N, Weyrich M. Towards autonomous systems: Flexible modular production system enhanced with large language model agents. In: Proceedings of the 2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation; 2023:1-8. Available from: https://doi.org/10.48550/arXiv.2304.14721
Ogundare O, Madasu S, Wiggins N. Industrial engineering with large language models: A case study of ChatGPT’s performance on oil & gas problems. In: Proceedings of the 2023 11th International Conference on Control, Mechatronics and Automation; 2023:458-461. Available from: https://doi.org/10.48550/arXiv.2304.14354
Ding X, Han J, Xu H, Zhang W, Li X. HiLM-D: Towards high-resolution understanding in multimodal large language models for autonomous driving. arXiv preprint arXiv:2309.05186. 2023. Available from: https://doi.org/10.48550/arXiv.2309.05186
Keysan A, Look A, Kosman E, Gursun G, Wagner J, Yao Y, et al. Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving. 2023. Available from: https://doi.org/10.48550/arXiv.2309.05282
Dasgupta I, Kaeser-Chen C, Marino K, Ahuja A, Babayan S, Hill F, et al. Collaborating with language models for embodied reasoning. arXiv preprint arXiv:2302.00763. 2023. Available from: https://doi.org/10.48550/arXiv.2302.00763
Zhou W, Peng X, Riedl MO. Dialogue shaping: Empowering agents through NPC interaction. arXiv preprint arXiv:2307.15833. 2023. Available from: https://doi.org/10.48550/arXiv.2307.15833
Nottingham K, Ammanabrolu P, Suhr A, Choi Y, Hajishirzi H, Singh S, et al. Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modeling. In: Proceedings of the 40th International Conference on Machine Learning; 2023:26311–26325. Available from: https://dl.acm.org/doi/10.5555/3618408.3619504
Wu Z, Wang Z, Xu X, Lu J, Yan H. Embodied task planning with large language models. arXiv preprint arXiv:2307.01848. 2023. Available from: https://doi.org/10.48550/arXiv.2307.01848