Hang on in there and you just might make it

Oct '23 (edited) • 🥇 Wins

This is my first contribution. TL;DR — keep trying, because only you can make it work for you.

My current infatuation with AI started when ChatGTP came out late last year. There, I said it. I'm no long-time Machine Learning enthusiast, no Data Scientist. It took a Large Language Model, fine-tuned to human conversation to pull me in. Superficial, I know.

But here I am and I'm still hanging on. In fact, it took me four months to realise that ChatGPT's replies were not random. By February of this year, I turned my attention to formulating my prompts more deliberately, probably because of the endless onslaught of YouTube videos claiming to have found The Ultimate Prompt. I taught myself how best to trigger the AI into giving me lean, relevant and useful responses. I felt I had become a Prompt Engineer, nay, a Prompt Warrior.

Interestingly, that triggered a deeper curiosity in me: how is it that Language Models deliver the output you desire? If it can't be a personal liking on the side of the Artificial Intelligence, surely I must find the answer in how they are built. Cue the PyTorch course that I dove into, learning a little about Machine Learning, more about Python and lots and lots about setting up my development environment. I did not finish the course.

In parallel, I started working on a Better Way© to manage my prompts. Prompts are all about context, and examples, and tasks, and ..., and ... So when you devise one that works well, and that you would like to reuse, how do you store it? How do you trigger it once you need it? I wanted to build and app, and for that you need to develop. And you know what? I just learned a lick of Python, and I had ChatGPT to help me along!

This is where I learned the limitations of ChatGPT for real. There was Python code that worked flawlessly, code that needed iterations to make run, and there was code that I could not get to work no matter what. I am embarrassed at the abuses that I threw at the AI. Sorry, Chatty. But the real problem when coding with ChatGPT is that it doesn't work well with more code than, say, a screenful. At least not for me. I did not finish developing the app, but I do have tons of Python, JavaScript, HTML and CSS snippets. And I know the basics of setting up a Flask web app.

I found out that when you develop, even together with ChatGPT, you search Stack Overflow, Google and even YouTube for specific help or for inspiration. During one of those tangents, I learned about local LLMs: totally private, low latency (hah!) and dedicated AIs that promised to work on your data in ways that only you would set up. No more jailbreaking, no more lobotomised public AIs. The Real Thing.

And so I went on the journey that brought me here today. It took me to Oobabooga, Stable Diffusion, WebUI and other dead ends. For now, I settled on LM Studio because it taught me the basics of setting up local LLMs and serving those LLMs locally using the OpenAI API. And on Ollama because it taught me how to integrate local LLMs in Python code that YouTubers demo using OpenAI. And on LLM, a command line utility by Simon Willison that makes it easy to experiment with various LLMs.

And I am leaving out on how clueless I was about Huggingface, about Langchain, about Github and about other two-words-that-are-one-word sites (and here I thought the Dutch language was the worst...) and that I am now increasingly comfortable with. I found the YouTubers that help me get the most out of the jungle we call AI — Dave Ebbelaar, looking at you! But also at Dave Shapiro, Sam Witteveen and others.

My latest private victory was getting local LLMs to stream when they responded. It took me many evenings browsing through documentation I do not know how to read. Explaining and re-explaining to ChatGPT what I needed explained to me. But in the end, Mistral-OpenOrca-7B streamed its sweet responses in beautiful white on black in my Terminal: when was Marcus Aurelius' reign?

So all I can say for now is: I am no longer a novice. It's taken me ten months, but I now know much better what I don't know than at the beginning of this journey when everything seemed possible if only. It's a humbling exercise, and having to admit that I am a lousy programmer, that I barely know enough about LLMs to get them running on my laptop and that I still don't understand why PALChain does work with OpenAI, but not with Llama 2, is hardly inspiring.

But I hung on in there and now I want to challenge myself and contribute to this awesome collective. Thanks guys for helping me cross my next hurdle. I hope I can help you cross yours.

14 comments