@NachoT Great feedback thank you! I am glad it worked for you after a little fiddling. You are right more clear instructions would certainly help That won’t be hard to update here in the future.
I implemented a fairly basic version of the chunking, splitting, and reranking operations. I tested it out with a few different LLM’s for retrieval and was getting really pretty decent results, so I stopped there. I don’t think it was working well using Ollama and I think it was due to context window issues. That would probably be something to implement in the future.
Ill let everyone know once I have an update to test here in the next few days.
–Cheers
Amazing work as I’ve already said @HillviewCap ! Is there a specific point you are trying to reach before a PR? I haven’t been super active bringing in PRs the last couple weeks but pretty soon here I’m going to merge some and I’d love to merge yours.
No, I think everything is good to go other than the few items Nacho referenced. I’ll get those updated and submit a PR shortly.
2 Likes
Sounds great, I’m looking forward to it!
1 Like
Alright hierarchical RAG integration for LLMS.txt documents by HillviewCap · Pull Request #119 · coleam00/Archon
Submitted its a biggun. I added a search feature for the https://llms-text.ai site now that I got the API working the other day. You can search for an llms.txt and request to add it to your embeddings right from the search results.
1 Like