Prompt Engineering and LLMOps: Building LLM Applications

Building LLM Applications

Today we dive into Prompt Engineering and how to build LLM applications with LangChain. There has been a relentless effort from non-technical influencers to make us believe Prompt Engineering was as complex as a Kindergarten grammar lesson (e.g. "The top 5 prompts to improve your productivity!"), but it is actually a bit more complex than that! We cover:

• Prompt engineering is more than we think

• The power of LangChain: LLMOps

• More about Prompt Engineering and LLMOps

---

Prompt engineering is more than we think

It took me some time to warm up to the idea that Prompt Engineering is actually worth digging into! Not only I realized that it is not a trivial field of research but it is also critical if we need to build applications using LLMs. We are surely barely getting started with our understanding of the subject, so time to jump into it!

Typically people are asking direct questions to the LLM. That is called zero-shot prompting. If you provide a few examples in the prompt, it is called few-shot prompting:

"""
Example question
Example answer

Question
What is the answer?
"""

In a few-shot prompt, you can get better results if you showcase the reasoning that leads to a specific answer. This is called "Chain of thoughts". You can induce a similar behavior in zero-shot by prompting it to "think step by step" and this is referred to as "Inception". "Chain of Thoughts" can take the form of intermediate questions and answers (e.g. "Do I need more information to solve the problem? -> Yes I do"). This is called "Self-ask". You can induce targeted answers by referring to concepts or analogies (e.g. "imagine you are a physics professor. Answer this question:"). This is called "Memetic Proxy". If you rerun the same query multiple times, you'll get different answers, so choosing a consistent answer over multiple queries increases the quality of the answers. This is called "Self-consistency". If you exchange multiple messages with the LLM, it is a good idea to use a memory pattern such that it can refer to previous interactions.

It becomes interesting when LLMs are given access to tools or databases. Based on questions they can decide to use tools. We can provide examples of question-action pairs (e.g. "What is the age of the universe?" -> [search on Wikipedia]) and this is called "Act". We get better results with "ReAct" when we induce intermediate thoughts from the LLM (e.g. "What is the age of the universe?" -> "I need to find more information on the universe" -> [search on Wikipedia]).

You can chain prompts! For example, you can prompt for using a tool, extract the information coming from the tool API call in the following prompt and use the result in the next prompt to answer the initial question. You can solve complex problems by inducing a plan of action. Each step of the plan can be used to generate its own chain of actions using ReAct. For example, AutoGPT is using the "Plan and execute" pattern on autopilot to solve complex problems. I guarantee that "Plan and execute" will be the source of many new startups in the coming years! 

LLMs can be thought of as some sort of flexible subroutines taking inputs and producing outputs. Prompts are the molds that shape the subroutines to solve a specific problem. LLM applications consist of piecing together those subroutines to build novel capabilities.

The power of LangChain: LLMOps

LangChain is really changing the game when it comes to building applications using LLMs. There are really 2 sides to Prompt Engineering: the one pushed by the non-technical influencers (e.g. "The top 5 prompts to improve your productivity!") and the one that actually allows you to build applications. To build applications, you dynamically need to chain multiple prompts while controlling the output of each to obtain somewhat deterministic outcomes. I have to say, I have seen so many of those prompt engineering guides by influencers, that I have wrongly ignored the complexity of going beyond the basic ChatGPT UI interface.