"Narrow" versus "Broad" AI: should LLMs do everything?

After chatGPT came to dominate public discourse about AI this year, we might be forgiven for thinking that with AI models: “bigger is better”. Articles demonstrating chatGPT-powered VMs, chatGPT passing the bar exam, and chatGPT creatively solving CAPTCHAs show a perhaps unnerving level of problem-solving versatility and adaptability.

We make the case that while chatGPT (a form of “broad AI”) can be used to solve a lot of problems, there is also a strong case for smaller, more specialised models that are designed for particular tasks (“narrow AI”). Read on to explore the balance between these two paradigms, and how they can both be relevant in the coming years.

Broad AI: The downsides

Regulatory risk

LLMs have sparked debates around ethics, safety, ownership and a slew of other topics which we’re only just beginning to answer.

The ethical considerations encompass the potential for bias in decision-making, the implications of deepfakes in media, and the privacy concerns raised by generative models trained on vast swaths of data. Safety also ties into this closely, as there are potentially strong ramifications in the event of errors or “hallucinations”, as well as LLMs making dangerous recommendations without necessarily understanding the consequences of doing so. There are then questions around intellectual property rights: who owns the content generated by these models?

These debates are complex and multifaceted, and while this article does not attempt to provide definitive answers, we aim to highlight the emerging regulatory landscape that could shape the use of LLMs.

It's reasonable to anticipate that regulatory frameworks will evolve, potentially imposing constraints on the deployment of LLMs. For instance, future regulations may govern who is qualified to use these models, ensuring that operators have the necessary expertise to manage the ethical and societal implications. Similarly, the purpose for which LLMs can be employed might be subject to scrutiny, with certain applications being restricted to prevent misuse.

With the potential for stringent controls and compliance requirements, the evolving regulatory environment surrounding LLMs would render these systems an expensive and less accessible resource for smaller technology firms.

Cost

The human-like understanding and response generation capabilities of LLMs have made them transform areas such as content creation, programming, and even artistic design, offering a level of scalability and efficiency previously unattainable.

This comes at a cost however: they are very expensive to train and use. The race to develop LLMs has been marked by an exponential increase in scale and complexity, requiring substantial R&D investment. For example, one unverified information leak suggests that GPT-4 was trained on 25,000 A100s over 100 days. Allegedly consisting of ~1 trillion parameters and costing over $100 million to train, GPT-4 is a recent instalment in a exponential journey over the last five years:

These are not advances that can be made by small research groups or the average startup, instead we must rely on well-resourced technology companies with substantial financial resources.

There is then the matter of using LLMs. An analysis by Dylan Patel and Afzal Ahmad suggests that chatGPT costs ~$700,000 to run per day, or $0.36/query. This is orders of magnitude more expensive than a conventional AI solution, deployed on an edge device for example. The article focuses on LLM applications for search, but this quote is particularly noteworthy:

The amazing thing is that Microsoft knows that LLM insertion into search will crush the profitability of search and require massive Capex.

The calculus of integrating LLMs into our tech stacks is not just about technological feasibility but also financial viability. Large entities such as Microsoft can absorb large capital expenditure and low profit margins, but this is not normally an option for most tech companies or startups.

“Narrower” AI: a solution?

An alternative approach to using LLMs is to use “narrow” AI — specialised AI models tailored for particular tasks. These models present an alternative approach, one where AI’s utility is defined by precision and efficiency rather than scale and versatility.

Because they can only be used to solve more narrowly-scoped problems, potential for misuse and misinformation is limited and more easily controlled. Additionally, narrow AI has been used in industry for over a decade and thus the regulatory space around it is better understood. While there are some restrictions on narrow AI models, the regulatory landscape is much more mature and this makes them much more straightforward to integrate into products.

There is a sprawling research community around conventional AI models - HuggingFace for example boasts nearly 400,000 models, all publicly available and sourced from their users. There’s a wealth of publicly available research, pre-trained models, and collaborative projects, all available to anybody wishing to leverage AI in their product. This is a much more democratic approach to AI: small teams can build powerful solutions without the need for exorbitant capital expenditure.

The granular focus of narrow AI also carries a very important benefit if correctness and reliability are of importance. Their operational domain is well-defined, which helps with the characterisation and mitigation of error cases. This is particularly challenging for broad AI, as the scope of what it can be used for is almost limitless in comparison. Having a solution capable of “hallucinating” results is rarely advantageous outside of creative applications, for example.

Moreover, the operational costs of narrow AI solutions are significantly lower. Many deploy models onto on-site edge devices for example, consuming orders of magnitude less power than a cloud-based GPU farm and improving the privacy of their solution to boot.

This not only reduces the carbon footprint of our projects, for smaller companies narrow AI is a game-changer. They can compete on a level playing field with corporations such as Microsoft, offering specialised services that are more aligned with the needs of niche markets. The cost-effective development cycle enables us to respond to our markets without the need for large capital expenditure.

Conclusion: Narrow AI is still relevant

Broad AI has challenged a lot of our perceptions about what we can automate, and perhaps even how we allocate our workforce going forward. It can obtain “good” solutions to an unprecedented range of problems, and is making areas accessible to us that are sometimes far out of our spheres of competence.

It is not necessarily the right tool for every job though, and in many cases we can (and probably should) look to narrow AI to solve our problems - it is cheaper, more environmentally friendly, more well understood and potentially exposes us less to future regulatory risks.