Good question! These kind of tools for knowledge retrieval are super powerful and becoming more and more realistic with LLMs that not only can support larger context windows but can also handle huge prompts too.
However, RAG still has its place because sometimes there are specific pieces of information you need to fetch from a huge corpus of documents, and even really well defined metadata might not be able to speak to the LLM that a specific document actually has that info.
The best approach in my mind is giving the LLM tools for both RAG and document lookup as you described it! That kind of agentic RAG is something I have been covering on my channel!