Earlier this year, BigML’s Chief Scientist and Oregon State University Emeritus Professor Tom Dietterich gave a keynote presentation titled “What’s wrong with LLMs and what we should be building instead” at the ValgrAI (Valencian Graduate School and Research Network of Artificial Intelligence) event in Valencia, Spain.
Large Language Models (LLMs) provide a pre-trained foundation for training many interesting AI systems. Among LLMs’ achievements we can count the ability to carry out conversations and answer questions covering a wide range of human knowledge, which Professor Dietterich stresses as our first case of creating a broadly-knowledgeable AI system. Other notable capabilities include summarization and revision of documents, writing code from English descriptions, and in context learning based on a small number of training samples.
However, LLMs have many shortcomings. They are expensive to train and to update, their non-linguistic knowledge is poor, they make false and self-contradictory statements, and these statements can be socially and ethically inappropriate. Professor Dietterich starts his keynote with an overview of well-documented LLM deficiencies and the current efforts to address them within the existing framework.
In the second part of his eye-opening presentation, Dr. Dietterich proposes a more modular architecture that decomposes the functions of existing LLMs and adds several additional components that can potentially address all of the shortcomings of LLMs. Dr. Dietterich’s modular architecture could be built through a combination of state-of-the-art machine learning and software engineering best practices.
Please follow along this noteworthy keynote on YouTube for the specifics of the proposed solution architecture and more.
If interested, you can also access the slides for the keynote at your convenience. Let us know in the comments of your take on what the future is likely to bring in making LLMs more and more bulletproof and whether they are ready to fulfill the capital markets’ tremendous growth expectations in the near future.
Should your organization choose to graduate from a model-centric hodgepodge approach to ML and scale Machine Learning solutions without introducing unnecessary complexity, be sure to send us a note so we can arrange a demo of the BigML platform, which has been making Machine Learning easy and beautiful for everyone for over a decade.