Getting my head around "prompt engineering" - hypothesis

Dec '23 (edited) • 💬 General

Hi all, today I have finally decided to look again into the buzzword of prompt engineering (PE). There is lots of videos around about role-playing, describing the context, setting the temperature, but also about fancy stuff like CoT, ToT, React, graphs of thoughts, or system prompts and context windows.

A lot of different things that have very little to do with each other. And afai-see no resource that looks at it comprehensively (well, this may be the closest one: https://learnprompting.org/docs/intro)

I have thought about it, here are my ideas.

So let me suggest this structure. Very early and crude but helps me at least a bit with "sense-making".

I am calling the groups by the most relevant user group

PE as writing effective user prompts (zero shot, or input-output) (Common User)
PE as using a lot of advanced techniques to solve more complex or logical problems (few shot, CoT, ToT, React, ...) (Researcher)
PE as a programming tool to give context to the LLM and to write mini-applications (with the system prompt and variables) (Developer)

On 1)

Perfect user prompt

This is a skill to be learned by everyone. Just like using google. It is mostly about being concise and descriptive. Imo this video gives an excellent explanation:

Master the Perfect ChatGPT Prompt Formula (in just 8 minutes)!

On 2)

Advanced (user) prompting techniques

Here we are talking about tools to help LLM crack more complex problems. Some of them may not be necessary any more with GPT4-Turbo and advanced models.

It is about getting a solution step-by-step, decomposing the problem, trying out different solutions, have a LLM brainstorm an idea, and agentic behavior of engaging in a thinking-acting loop.

This stuff I have not seen used much in "real life". But this is just an impression seeing the examples and watching YT. AutoGPT uses ReAct apparently. For many of the things if you like to understand or implement in detail you have to read the papers of the researchers.

Some approaches can be easily prompted some need to be coded or need a rather complex prompt.

https://www.promptingguide.ai/

There is not really an overview YT video about all these techniques, mostly individual videos that do a deep dive on particular aspects. this playlist covers some techniques:

https://www.youtube.com/watch?v=WfwHJmyTvIk&list=PLaJCKi8Nk1hyvGVZub2Ar7Az57_nKemzX

Here some techniques are covered on a more simple level

https://www.youtube.com/watch?v=23rEezTU3-A

On 3)

System prompt

This relates to using the system prompt of GPT (are there similar mechanisms in other models) to create mini applications. Examples are the Synapse Labs Prompt: How to Turn ChatGPT into AutoGPT with 1 Prompt or Dave Shapiros example ChatGPT SYSTEM Prompt Engineering (Deep Dive)

or - Python AI Choose Your Own Adventure Game - Tutorial https://www.youtube.com/watch?v=nhYcTh6vw9A&t=1109s

And this aspect of prompt engineering represents a neet tool for developers to create mini applications. But you guys know more about it ...

What do you think? Write me your thoughts

5 comments