EnterpriseAI/ClosedCode
1
0
Fork 0
Code Issues Pull Requests Actions 33 Packages Projects Releases Wiki Activity

chore: generate

Browse Source
This commit is contained in:
opencode-agent[bot]
2026-06-24 13:59:28 +00:00
parent 36501a81c3
commit 1ef0fd5d01
1 changed files with 154 additions and 145 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/llm/DESIGN.md
+154 -145
View File
@@ -176,7 +176,7 @@ const request = LLM.request({
}) })
// Current API: this performs one provider turn, despite the broad name. // Current API: this performs one provider turn, despite the broad name.
const response = yield* LLM.generate(request) const response = yield * LLM.generate(request)
// Current API: execution also needs LLMClient.layer and RequestExecutor services. // Current API: execution also needs LLMClient.layer and RequestExecutor services.
``` ```
@@ -242,14 +242,15 @@ executable tools. Call-level values override model defaults.
Provider-specific options are inferred from the concrete model: Provider-specific options are inferred from the concrete model:
```ts ```ts
yield* LLM.generate({ yield *
model: OpenAI.model("gpt-4.1-mini"), LLM.generate({
prompt: "Hello", model: OpenAI.model("gpt-4.1-mini"),
provider: { prompt: "Hello",
store: false, provider: {
// OpenAI-specific autocomplete here; no `{ openai: ... }` nesting. store: false,
}, // OpenAI-specific autocomplete here; no `{ openai: ... }` nesting.
}) },
})
``` ```
Code choosing between providers dynamically must narrow the model before using Code choosing between providers dynamically must narrow the model before using
@@ -278,12 +279,14 @@ provider.
### Inline input ### Inline input
```ts ```ts
const result = yield* LLM.generate({ const result =
model, yield *
system: "You are concise.", LLM.generate({
prompt: "Summarize this pull request.", model,
generation: { maxTokens: 500 }, system: "You are concise.",
}) prompt: "Summarize this pull request.",
generation: { maxTokens: 500 },
})
``` ```
### Reusable portable request ### Reusable portable request
@@ -296,7 +299,7 @@ const request = LLM.request({
}) })
// Bind process-local execution behavior only when running. // Bind process-local execution behavior only when running.
const result = yield* LLM.generate({ model, request }) const result = yield * LLM.generate({ model, request })
``` ```
`LLM.request(...)` returns a plain immutable object. Use ordinary object spread `LLM.request(...)` returns a plain immutable object. Use ordinary object spread
@@ -362,11 +365,13 @@ const tools = {
}), }),
} }
const result = yield* LLM.generate({ const result =
model, yield *
prompt: "What is the weather in London?", LLM.generate({
tools, model,
}) prompt: "What is the weather in London?",
tools,
})
// The runtime advertises definitions, dispatches calls, records results, and // The runtime advertises definitions, dispatches calls, records results, and
// continues provider turns automatically. // continues provider turns automatically.
@@ -387,15 +392,14 @@ successful result with `stopReason: "max-turns"`, not an Effect failure.
### Custom stopping ### Custom stopping
```ts ```ts
const result = yield* LLM.generate({ const result =
model, yield *
prompt, LLM.generate({
tools, model,
stopWhen: StopWhen.any( prompt,
StopWhen.turnCount(8), tools,
StopWhen.hasToolCall("finalize"), stopWhen: StopWhen.any(StopWhen.turnCount(8), StopWhen.hasToolCall("finalize")),
), })
})
``` ```
`stopWhen` accepts one predicate. Composition is explicit through combinators `stopWhen` accepts one predicate. Composition is explicit through combinators
@@ -427,17 +431,13 @@ const request = LLM.request({
tools: Tool.toDefinitions(tools), tools: Tool.toDefinitions(tools),
}) })
const events = yield* LLM.stream(request).pipe(Stream.runCollect) const events = yield * LLM.stream(request).pipe(Stream.runCollect)
const call = Array.from(events).find(LLMEvent.is.toolCall) const call = Array.from(events).find(LLMEvent.is.toolCall)
if (call && !call.providerExecuted) { if (call && !call.providerExecuted) {
const dispatched = yield* ToolRuntime.dispatch(tools, call) const dispatched = yield * ToolRuntime.dispatch(tools, call)
const followUp = LLM.updateRequest(request, { const followUp = LLM.updateRequest(request, {
messages: [ messages: [...request.messages, Message.assistant([call]), Message.tool({ ...call, result: dispatched.result })],
...request.messages,
Message.assistant([call]),
Message.tool({ ...call, result: dispatched.result }),
],
}) })
// Caller must invoke the provider again and repeat the loop. // Caller must invoke the provider again and repeat the loop.
} }
@@ -452,17 +452,19 @@ OpenCode and other durable runtimes need to own persistence, tool settlement,
and continuation. They use the explicit turn API: and continuation. They use the explicit turn API:
```ts ```ts
const result = yield* LLM.generateTurn({ const result =
model, yield *
request, LLM.generateTurn({
// Definitions only. generateTurn never dispatches local handlers. model,
tools: { request,
getWeather: Tool.definition({ // Definitions only. generateTurn never dispatches local handlers.
description: "Get current weather for a city.", tools: {
parameters: WeatherInput, getWeather: Tool.definition({
}), description: "Get current weather for a city.",
}, parameters: WeatherInput,
}) }),
},
})
// Persist the TurnResult and settle calls durably before the next turn. // Persist the TurnResult and settle calls durably before the next turn.
for (const call of result.toolCalls) { for (const call of result.toolCalls) {
@@ -494,19 +496,21 @@ const request = LLM.request({
}, },
}) })
const result = yield* LLM.generate({ const result =
model, yield *
request, LLM.generate({
tools: { model,
getWeather: Tool.make({ request,
description: "Get current weather for a city.", tools: {
parameters: WeatherInput, getWeather: Tool.make({
success: WeatherOutput, description: "Get current weather for a city.",
execute: getWeather, parameters: WeatherInput,
formatError, success: WeatherOutput,
}), execute: getWeather,
}, formatError,
}) }),
},
})
``` ```
Definitions and handlers match by record key. Before the first provider call, Definitions and handlers match by record key. Before the first provider call,
@@ -516,13 +520,15 @@ binding. Missing or incompatible bindings fail with a typed tool-binding error.
Provider-hosted tools are distinct typed values: Provider-hosted tools are distinct typed values:
```ts ```ts
const result = yield* LLM.generate({ const result =
model: OpenAI.model("gpt-4.1"), yield *
prompt: "Find today's relevant announcements.", LLM.generate({
tools: { model: OpenAI.model("gpt-4.1"),
search: OpenAI.tool.webSearch({ searchContextSize: "medium" }), prompt: "Find today's relevant announcements.",
}, tools: {
}) search: OpenAI.tool.webSearch({ searchContextSize: "medium" }),
},
})
``` ```
Hosted tools do not pretend to have local handlers, and callers do not inspect a Hosted tools do not pretend to have local handlers, and callers do not inspect a
@@ -589,11 +595,13 @@ const Weather = Schema.Struct({
highCelsius: Schema.Number, highCelsius: Schema.Number,
}) })
const result = yield* LLM.generate({ const result =
model, yield *
prompt: "Give me today's weather for London.", LLM.generate({
output: Weather, model,
}) prompt: "Give me today's weather for London.",
output: Weather,
})
// Inferred from Weather. // Inferred from Weather.
result.output.city result.output.city
@@ -612,11 +620,13 @@ Advanced callers may override the strategy when exact provider semantics matter.
```ts ```ts
// Current API is a separate operation and always forces a synthetic tool. // Current API is a separate operation and always forces a synthetic tool.
const result = yield* LLM.generateObject({ const result =
model, yield *
prompt, LLM.generateObject({
schema: Weather, model,
}) prompt,
schema: Weather,
})
``` ```
The proposal unifies generation and lets capabilities choose the strategy rather The proposal unifies generation and lets capabilities choose the strategy rather
@@ -679,11 +689,12 @@ cache boundaries where explicit caching is supported and does nothing on the wir
where providers cache implicitly. where providers cache implicitly.
```ts ```ts
yield* LLM.generate({ yield *
model, LLM.generate({
prompt, model,
cache: "none", // Explicit opt-out. prompt,
}) cache: "none", // Explicit opt-out.
})
``` ```
Granular cache policy remains available as an advanced request option. Granular cache policy remains available as an advanced request option.
@@ -705,13 +716,14 @@ silently inherit custom retry policies.
### Timeouts ### Timeouts
```ts ```ts
yield* LLM.generate({ yield *
model, LLM.generate({
prompt, model,
timeout: "2 minutes", // Entire run, including tools. prompt,
turnTimeout: "30 seconds", // Each provider turn. timeout: "2 minutes", // Entire run, including tools.
tools, turnTimeout: "30 seconds", // Each provider turn.
}) tools,
})
``` ```
Exact Duration input spelling follows Effect conventions. Individual tools may Exact Duration input spelling follows Effect conventions. Individual tools may
@@ -740,10 +752,11 @@ retry, or redirect control flow.
```ts ```ts
const model = OpenAI.model("gpt-4.1", { const model = OpenAI.model("gpt-4.1", {
hooks: { hooks: {
request: (request) => Effect.succeed({ request: (request) =>
...request, Effect.succeed({
metadata: { ...request.metadata, tenant: "acme" }, ...request,
}), metadata: { ...request.metadata, tenant: "acme" },
}),
body: (body, context) => auditBody(body, context), body: (body, context) => auditBody(body, context),
transport: (request) => signInternalGatewayRequest(request), transport: (request) => signInternalGatewayRequest(request),
event: (event) => redactProviderMetadata(event), event: (event) => redactProviderMetadata(event),
@@ -775,15 +788,16 @@ The request customization ladder is:
5. Experimental provider-definition or protocol patching 5. Experimental provider-definition or protocol patching
```ts ```ts
yield* LLM.generate({ yield *
model, LLM.generate({
prompt, model,
http: { prompt,
headers: { "x-experimental": "1" }, http: {
query: { debug: "true" }, headers: { "x-experimental": "1" },
body: { newlyReleasedProviderField: true }, query: { debug: "true" },
}, body: { newlyReleasedProviderField: true },
}) },
})
``` ```
Raw overlays are intentional last-resort support for provider features that ship Raw overlays are intentional last-resort support for provider features that ship
@@ -905,7 +919,7 @@ Schemas live in a dedicated namespace/subpath instead of flooding root exports:
```ts ```ts
import { LLMSchema } from "@opencode-ai/ai/schema" import { LLMSchema } from "@opencode-ai/ai/schema"
const request = yield* Schema.decodeUnknown(LLMSchema.Request)(input) const request = yield * Schema.decodeUnknown(LLMSchema.Request)(input)
``` ```
Schemas cover only serializable domain values: Schemas cover only serializable domain values:
@@ -928,10 +942,7 @@ Provider authoring is public but experimental.
### Declarative provider definition ### Declarative provider definition
```ts ```ts
import { import { Provider, Protocol } from "@opencode-ai/ai/provider"
Provider,
Protocol,
} from "@opencode-ai/ai/provider"
export const ExampleAI = Provider.define({ export const ExampleAI = Provider.define({
id: "example", id: "example",
@@ -996,9 +1007,7 @@ SDK integrations that motivated this package.
const PatchedResponses = OpenAIResponses.with({ const PatchedResponses = OpenAIResponses.with({
body: { body: {
fromRequest: (request) => fromRequest: (request) =>
OpenAIResponses.body.fromRequest(request).pipe( OpenAIResponses.body.fromRequest(request).pipe(Effect.map((body) => ({ ...body, custom_field: true }))),
Effect.map((body) => ({ ...body, custom_field: true })),
),
}, },
stream: { stream: {
step: patchResponsesStep, step: patchResponsesStep,
@@ -1042,45 +1051,45 @@ providers, and there is no preferred all-providers barrel.
## Defaults ## Defaults
| Concern | Default | | Concern | Default |
| --- | --- | | ---------------------------- | --------------------------------------------------- |
| `LLM.generate` semantics | Complete Model Run | | `LLM.generate` semantics | Complete Model Run |
| `LLM.generateTurn` semantics | Exactly one Provider Turn | | `LLM.generateTurn` semantics | Exactly one Provider Turn |
| Maximum turns | 20 | | Maximum turns | 20 |
| Turn-limit outcome | Successful `max-turns` result | | Turn-limit outcome | Successful `max-turns` result |
| Tool execution | Automatic in runs | | Tool execution | Automatic in runs |
| Tool concurrency | Concurrent, bounded, deterministic result order | | Tool concurrency | Concurrent, bounded, deterministic result order |
| Prompt caching | `auto` | | Prompt caching | `auto` |
| Retries | Conservative, pre-output transient failures only | | Retries | Conservative, pre-output transient failures only |
| Structured output | Capability-selected native or tool strategy | | Structured output | Capability-selected native or tool strategy |
| Capability mismatch | Typed failure before network execution | | Capability mismatch | Typed failure before network execution |
| Unknown model capability | Conservative protocol baseline | | Unknown model capability | Conservative protocol baseline |
| Telemetry content | Metadata only | | Telemetry content | Metadata only |
| Cost | Estimated aggregate or unavailable | | Cost | Estimated aggregate or unavailable |
| Cancellation | Interruption/rejection, never successful completion | | Cancellation | Interruption/rejection, never successful completion |
## Clean-break Migration ## Clean-break Migration
The redesign intentionally removes or changes these current concepts: The redesign intentionally removes or changes these current concepts:
| Current | Proposed | | Current | Proposed |
| --- | --- | | --------------------------------------- | ----------------------------------------------------------- |
| `@opencode-ai/llm` | `@opencode-ai/ai` | | `@opencode-ai/llm` | `@opencode-ai/ai` |
| Mandatory `LLM.request({ model, ... })` | Inline calls or model-free portable requests | | Mandatory `LLM.request({ model, ... })` | Inline calls or model-free portable requests |
| `LLM.generate` means one turn | `LLM.generate` means complete run | | `LLM.generate` means one turn | `LLM.generate` means complete run |
| `LLMClient.generate/stream` | `LLM.generateTurn/streamTurn` for one turn | | `LLMClient.generate/stream` | `LLM.generateTurn/streamTurn` for one turn |
| `LLMClient.layer` requirement | Standard Effect requirements exposed directly | | `LLMClient.layer` requirement | Standard Effect requirements exposed directly |
| Public `Route` mental model | Hidden behind executable `Model` | | Public `Route` mental model | Hidden behind executable `Model` |
| `Provider.make` structural helper | Experimental declarative `Provider.define` | | `Provider.make` structural helper | Experimental declarative `Provider.define` |
| Schema classes as canonical values | Plain immutable values plus schema subpath | | Schema classes as canonical values | Plain immutable values plus schema subpath |
| `LLM.updateRequest` | Object spread | | `LLM.updateRequest` | Object spread |
| `Tool.toDefinitions` in normal calls | Named executable tool records | | `Tool.toDefinitions` in normal calls | Named executable tool records |
| Manual `ToolRuntime.dispatch` loop | Automatic run dispatch; explicit turn API for orchestration | | Manual `ToolRuntime.dispatch` loop | Automatic run dispatch; explicit turn API for orchestration |
| `providerOptions: { openai: ... }` | Model-typed `provider: ...` | | `providerOptions: { openai: ... }` | Model-typed `provider: ...` |
| `generateObject` | Typed `output` option on `generate` | | `generateObject` | Typed `output` option on `generate` |
| One event union for provider output | Separate `TurnEvent` and `RunEvent` unions | | One event union for provider output | Separate `TurnEvent` and `RunEvent` unions |
| `providerExecuted` dispatch check | Distinct hosted-tool constructors | | `providerExecuted` dispatch check | Distinct hosted-tool constructors |
| One wrapped `LLMError` | Tagged domain error union | | One wrapped `LLMError` | Tagged domain error union |
OpenCode should migrate to `generateTurn` / `streamTurn`, preserving its durable OpenCode should migrate to `generateTurn` / `streamTurn`, preserving its durable
prompt admission, persistence, permission, tool settlement, and continuation prompt admission, persistence, permission, tool settlement, and continuation