Skip to content

Commit

Permalink
Update inline-prompts documentation with examples, options, tools, an…
Browse files Browse the repository at this point in the history
…d concurrency information
  • Loading branch information
pelikhan committed Sep 19, 2024
1 parent 45de150 commit 4f2f85c
Showing 1 changed file with 40 additions and 10 deletions.
50 changes: 40 additions & 10 deletions docs/src/content/docs/reference/scripts/inline-prompts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,19 @@ import { Code } from "@astrojs/starlight/components"
import summaryOfSummaryGpt35 from "../../../../../../packages/sample/genaisrc/summary-of-summary-gpt35.genai.js?raw"
import summaryOfSummaryPhi3 from "../../../../../../packages/sample/genaisrc/summary-of-summary-phi3.genai.js?raw"

The `runPrompt` function allows to build an inner LLM invocation. It returns the output of the prompt.
The `prompt` or `runPrompt` function allows to build an inner LLM invocation. It returns the output of the prompt.

`prompt` is a syntactic sugar for `runPrompt` that takes a template string literal as the prompt text.

```js
const { text } = await prompt`Write a short poem.`
```

You can pass a function to `runPrompt` that takes a single argument `_` which is the prompt builder.
It defines the same helpers like `$`, `def`, but applies to the inner prompt.

```js
const { text } = await runPrompt(_ => {
const { text } = await runPrompt((_) => {
// use def, $ and other helpers
_.def("FILE", file)
_.$`Summarize the FILE. Be concise.`
Expand All @@ -26,19 +32,42 @@ const { text } = await runPrompt(_ => {
You can also shortcut the function and pass the prompt text directly

```js
const { text } = await runPrompt(`Select all the image files in ${env.files.map(f => f.filename)}`)
const { text } = await runPrompt(
`Select all the image files in ${env.files.map((f) => f.filename)}`
)
```

## Options

Both `prompt` and `runPrompt` support various options similar to the `script` function.

```js
const { text } = await prompt`Write a short poem.`.options({ temperature: 1.5 })
const { text } = await runPrompt((_) => { ...}, { temperature: 1.5 })
```

:::caution
## Tools

Make sure to `await` all calls to `runPrompt` or you will find yourself in some weird race conditions.
(To be improved)
You can use inner prompts in [tools](/genaiscript/reference/scripts/tools).

:::
```js
defTool(
"poet",
"Writes 4 line poem about a given theme",
{
theme: {
type: "string",
description: "Theme of the poem",
}
},
(({theme})) => prompt`Write a ${4} line ${"poem"} about ${theme}`
)
```

## Limitations
## Concurrency

- Nested [functions](/genaiscript/reference/scripts/tools) are not supported in the inner prompt.
`prompt` and `runPrompt` are async functions that can be used in a loop to run multiple prompts concurrently. However, running
too many requests at a time might result in rate limiting. You can use [p-all](https://www.npmjs.com/package/p-all) to limit the number of concurrent requests.

## Example: Summary of file summaries using gpt-3.5

Expand All @@ -49,7 +78,8 @@ adding them to the main prompt.

## Example: Summary of file summaries using Phi-3

The snippet below uses [Phi-3](https://azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/)
The snippet below uses [Phi-3](https://azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/)
through [Ollama](https://ollama.com/) to summarize files individually before adding them to the main prompt.

<Code code={summaryOfSummaryPhi3} wrap={true} lang="js" />
```

0 comments on commit 4f2f85c

Please sign in to comment.