生产配置

HTTP 客户端默认值

relay 内置了生产级 HTTP 客户端配置：

参数	默认值
Timeout	120s
MaxIdleConns	100
MaxIdleConnsPerHost	10
IdleConnTimeout	90s
TLSHandshakeTimeout	10s
ResponseHeaderTimeout	30s

也可传入自定义客户端：

client := relay.New(
    relay.WithProvider(provider),
    relay.WithHTTPClient(customHTTPClient),
)

重试

WithRetry 接受 common/retry 的选项：

import (
    "time"

    "github.com/LingByte/ling-base/common/retry"
    "github.com/LingByte/ling-base/relay"
)

client := relay.New(
    relay.WithProvider(provider),
    relay.WithRetry(
        retry.WithMaxAttempts(3),
        retry.WithExponentialBackoff(100*time.Millisecond, 2*time.Second, 2.0, true),
    ),
)

重试触发条件：网络错误、HTTP 429、HTTP 5xx。

熔断器

import "github.com/LingByte/ling-base/common/circuitbreaker"

cb := circuitbreaker.New(circuitbreaker.Config{
    MaxRequests:      5,
    FailureThreshold: 0.5, // 失败率 50% 触发熔断
    MinRequests:      10,
    RecoveryTimeout:  30 * time.Second,
    Name:             "openai",
})

client := relay.New(
    relay.WithProvider(provider),
    relay.WithCircuitBreaker(cb),
)

Model Fallback

client := relay.New(
    relay.WithProvider(provider),
    relay.WithFallback(relay.FallbackConfig{
        FallbackModels: []string{"gpt-4o-mini", "gpt-3.5-turbo"},
    }),
)

主模型在建立连接阶段失败时，按顺序尝试备选模型。流式请求一旦开始，不会在流中途切换模型。

Request Hook（可观测性）

client := relay.New(
    relay.WithProvider(provider),
    relay.WithRequestHook(func(ctx context.Context, provider, model string, mode int) func(err error) {
        start := time.Now()
        return func(err error) {
            metric.Record(provider, model, time.Since(start), err)
        }
    }),
)

Hook 在请求前调用，返回的闭包在请求结束后执行，适合 metrics / tracing。

用量计量

import "github.com/LingByte/ling-base/relay/meter"

client := relay.New(
    relay.WithProvider(provider),
    relay.WithMeter(meter.NewMemoryMeter()),
)

统一错误处理

var relayErr *relay.RelayError
if errors.As(err, &relayErr) {
    log.Printf(
        "provider=%s model=%s status=%d code=%s msg=%s retryable=%v",
        relayErr.Provider,
        relayErr.Model,
        relayErr.StatusCode,
        relayErr.Code,
        relayErr.Message,
        relayErr.IsRetryable(),
    )
}

生产配置

On this page