My belief on solutioning any problem with AI begins with custom train your model. As someone without a development background, I aim to build a robust, easy to use, no code solution to train LLM with custom data with different dataset. I look forward for suggestions to create a platform for people to train their local LLMs with custom raw data & thoughfully design trained LLM for tailored response.
This sounds highly resource demanding. Do you have access to server equipment or proper server services?
I think this could be great however it could be very time consuming and resource demanding if people are to use a platform locally to train locally. If it’s a platform for use online it could help streamline some interesting LLM models! You could perhaps offer some unique datasets for people to build from and pick and choose, along with the option to add customs.
Thanks for your response, I have few used cases for small organization, it may not be very heavy, like if I want my model to respond to RFP in a specific way, I may only train it with limited data.
I may be wrong in overthinking 
is there an opensource tool to achieve this today locally?
Many people jump to fine tuning when a RAG will suffice. Be sure that you need that level of specificity. If it’s a matter of domain knowledge as opposed to learning how to do something in a specific way, RAGs are easier and faster to implement.
If you decide to fine tune, it depends on the model – whether you’re hosting it on Bedrock or locally with Ollama or using one of the closed-source LLMs like OpenAI. Ollama offers a training function where you feed in JSONL and fine tune whatever model you want. Here’s a pretty good article on the topic using Ollama.
Maybe look at using something like Google’s colab to host and run your training on and then pull the trained model from that, it is pretty cost efficient and allows a variety of runtime configurations with regards to diskspace, processing and GPUs.
1 Like
RAG can be very expensive if you need to go to production i have recently found. It seems to be the quicker way to get of the starting blocks however it can also have issues with inaccuracy. A knowledge graph can help. Fine tunes can be achieved locally and a few open source frameworks have been producing successful results. A missing piece i often see overlooked is evaluation. I agree with the use case of domain knowledge for RAG. I think of RAG like a long term memory in one sense. Fine tuning is like a PHD on one subset of a subject matter. sometimes a simple system prompt is all you need. the only way to really know what you might and then if it works is with an evaluation. GitHub - promptfoo/promptfoo: Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration. is a pretty easy to start with solution for evaluations.