‘AI hallucinations are hard to remove completely’: Naveen Rao, VP of AI, Databricks

THB: AI adoption is in full swing even as hallucinations persist. Why is this the case?

Naveen Rao: There are a few reasons. We’ve gotten more mature on what our application areas are for AI. Basically, we have figured out the use cases where we can tolerate some error. Initially, people tried to apply them to all kinds of areas where precision was required which was a mistake. Secondly, we are adding value in these areas so the models have actually gotten better. When you put information into context either through retrieval or just entering all the required context into the prompt, we see less hallucinations. The models adhere to the information in the prompts with higher fidelity. Over time, we’ll continue to work at driving down hallucinations. There can be other checks and balances like having another model judge whether the output is good. That gives us higher precision — it’s not 100% but in many cases it’s approaching north of 90 — 95% accuracy. But hallucinations are intrinsic to these AI models because LLMs are probabilistic, right? It’s an auto regressive, next token prediction. So, it’s very hard to remove completely.

THB: Databricks recently announced a $ 100 million partnership with Anthropic. How will this play out in the agentic AI market?

Naveen: We can’t have AI agents in companies because there are errors, especially when they’re doing multi-step tasks. I do think AI agents are definitely hyped. The definition of an AI agent has shifted to fit the narrative. The original intent of the word was to describe an entity that has agency meaning it can act completely on its own. Now, it has evolved into multi-part systems where multiple LLMs work together to solve a task that usually makes humans faster or more efficient.

Even if AI agents were perfectly behaved, we don’t want something that can act completely on its own with no governance. We have built this into Databricks’ governance layer for data extended to GenAI. So, whenever an AI agent is built, it has certain access rights and entitlements and it can’t go willy-nilly everywhere. In the coming years, as we start to solve these problems, things will become reliable and accurate. 

THB: Can we attain AGI with the current level of LLM advances?

Naveen: I mean, whether it’s a route to AGI or not will be seen in the technology itself. I personally don’t think it is. I don’t think that autoregressive loss is the right way to make something that can truly understand causation in a system. Humans learn in a different way. We learn by coming up with a mechanistic understanding of how we can solve a task. So, one thing leads to another thing then another. When I want to solve the same task again, I have a sense of the inputs that cause the causation to the next state. LLMs don’t do this. Maybe we’ll get there, but the current paradigm of very large pre-training on a huge corpus of unstructured data and then doing some sort of reinforcement learning to modify its behaviour is not what will lead to something that can truly act on its own. 

THB: So, you don’t believe that AGI is just around the corner?

Naveen: No, I think it’s a much harder problem than a lot of people want to give it credit for and we are not at a point where we’re close. Yes, we have made huge progress towards autonomous systems that can understand natural language. LLMs have solved natural language, which is a big thing. I don’t want to minimize what’s been done or their economic impact — LLMs are very useful tools.

THB: How do you compare enterprise AI against the consumer AI market? Do you find it easier to navigate?

Naveen: Enterprises tend to be slower to adopt. They’re generally very rational actors, whereas consumers are somewhat irrational. That makes it harder to go after consumer, because it’s hard to understand exactly why they’re going to buy. Over time the enterprise market will be bigger than the consumer market, I believe. There are really only a few different product surfaces. Search tools is a big one in the consumer segment, like Perplexity or ChatGPT. Image generation and others are really mostly for fun. But I don’t know how much people pay for fun. Usually, the novelty wears off unless they’re used for business purposes. Whereas in enterprises we see companies really trying to look for an ROI so they’re willing to invest a lot because it means something about the company versus their competitors.

THB: What is AI’s killer app now? 

Naveen: Right now, it’s coding. AI tools tend to be effective when they’re structured or their output is easily measured. With writing code, you can tell if the code complies pretty easily. Although coding agents hallucinate a lot but most of it is still useful. 

THB: What do you think about the view that students should stop studying software engineering?

Naveen: I don’t agree with it — someone has to understand how these systems work. Even if code generation is automated, it doesn’t mean that the physics of writing software goes away. Somebody will still have to work with the code and check it. We have to understand the basics. I think it’s a very poor advice to say we should stop studying computer science altogether.

Published - June 17, 2025 01:21 pm IST