Ok thank you very much, I was thinking of this while on holiday. Will try and start working on this next weekend.

@Marcio Pacheco does this allow you to get live data from my website such as pricing? I also was more interested in trying to go more into detail such as building it with Llama 7B and groq. I think it would be more rewarding and give me a better insight about how they work. Plus I like the challenge. If you have any advice relating to this method is appreciate as I'm very quick to learn and competent at doing new things but this is quite new to me. Plus I have low key started an accounts firm as well mostly to be able to build some AI that could work with me as I think most of accounts / audit work can be automated and this boutique hotel chatbot seems like a easy place for a first tool. That is because all I need is to request prices for days requested and give it data through some vector database of very simple policy questions.

Yes I understand that the synthetic data is generated separately and then the model would trained on it. That being said I only think that's the case at this point. This is similar to how 3 shot or 5 shot answers are generated by asking the model 3 or 5 times respectively and then taking the most common answer as the correct one (refer to simple multiple choice here). Yet when more compute is available for each response I would imagine having the LLM run this internally. Similarly at more available and efficient compute it would make sense to create synthetic data (including tests) be it numbers or even counter arguments, play it out in its "mind" and then critic its own response and give a final answer.

Yes agreed was just asking in general to see if there were any interesting podcasts people recommend. I find these complex topics very interesting, in fact LLM Tokenization by Andrej Karpathy just came up on my feed and I started listening to it over the weekend.