feat: implement config center phases 1-5

This commit is contained in:
2026-08-30 08:08:25 +08:00
commit 9a588233d0
54 changed files with 10473 additions and 0 deletions

7
.dockerignore Normal file
View File

@@ -0,0 +1,7 @@
.git
.env
bin
coverage.out
web/dist
web/node_modules
*.log

14
.env.example Normal file
View File

@@ -0,0 +1,14 @@
HTTP_ADDR=:8080
DATABASE_URL=postgres://configcenter:configcenter@localhost:5432/configcenter?sslmode=disable
ETCD_ENDPOINTS=http://localhost:2379
ETCD_DIAL_TIMEOUT=5s
OUTBOX_INTERVAL=500ms
OUTBOX_BATCH_SIZE=50
OUTBOX_MAX_RETRY=12
CORS_ALLOWED_ORIGINS=http://localhost:5173
AUTH_ENABLED=false
JWT_SECRET=replace-with-at-least-32-random-characters
JWT_TTL=8h
BOOTSTRAP_ADMIN_USERNAME=admin
BOOTSTRAP_ADMIN_PASSWORD=replace-with-at-least-12-characters
BOOTSTRAP_ADMIN_DISPLAY_NAME=Administrator

10
.gitignore vendored Normal file
View File

@@ -0,0 +1,10 @@
.env
.idea/
.vscode/
bin/
coverage.out
web/dist/
web/node_modules/
*.log
__pycache__/
*.pyc

1371
ConfigCenter.jsx Normal file

File diff suppressed because it is too large Load Diff

14
Dockerfile Normal file
View File

@@ -0,0 +1,14 @@
FROM golang:1.24-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY cmd ./cmd
COPY internal ./internal
RUN CGO_ENABLED=0 go build -buildvcs=false -trimpath -ldflags="-s -w" -o /out/configcenter ./cmd/server
FROM alpine:3.21
RUN apk add --no-cache ca-certificates && adduser -D -u 10001 configcenter
USER configcenter
COPY --from=build /out/configcenter /usr/local/bin/configcenter
EXPOSE 8080
ENTRYPOINT ["configcenter"]

29
Makefile Normal file
View File

@@ -0,0 +1,29 @@
.PHONY: dev test build web-install web-build compose-up compose-down integration-smoke loadtest
dev:
go run ./cmd/server
test:
GOCACHE=/tmp/configcenter-go-cache go test ./...
build:
mkdir -p bin
GOCACHE=/tmp/configcenter-go-cache go build -buildvcs=false -trimpath -o bin/configcenter ./cmd/server
web-install:
npm --prefix web install
web-build:
npm --prefix web run build
compose-up:
docker compose up --build
compose-down:
docker compose down
integration-smoke:
sh scripts/integration-smoke.sh
loadtest:
GOCACHE=/tmp/configcenter-go-cache go run ./cmd/loadtest -duration=30s -concurrency=32

208
README.md Normal file
View File

@@ -0,0 +1,208 @@
# Config Center
一个按 `config-center-etcd-implementation.md` 落地的自建配置中心。PostgreSQL 保存编辑态、版本和审计记录;发布事务写入 release 与 outboxworker 再将完整 namespace 快照原子下发到 etcd。Web Console 已接入真实 API不再使用浏览器内存数据。
## 已实现
- 应用、环境、命名空间、配置项 CRUD
- 草稿/已发布值分离,新增、修改、待删除状态与发布 diff
- PostgreSQL 事务化发布,基于 advisory lock 的并发版本分配;
- Outbox claim、租约恢复、指数退避、最大重试和 release 状态回写;
- etcd 完整快照 + `__meta` 同事务写入;
- 运行时配置读取与 SSE Watch服务端同 key Watch 扇出;
- 发布历史和“生成新版本”的可追溯回滚;
- 审计日志查询;
- 可选 JWT 登录、bcrypt 密码、全局管理员与应用级 `viewer` / `app-owner` RBAC
- IP/CIDR、实例 ID、稳定百分比三类灰度规则支持优先级覆盖
- Go/Python SDK支持自动重连、全量替换和本地文件缓存兜底
- React Web Console含登录、灰度、用户授权页面、Docker Compose 与本地内存开发模式;
- Prometheus 指标与告警规则、三副本服务/三节点 etcd Kubernetes 清单、SLO 压测工具。
## 快速启动
不安装 PostgreSQL/etcd 也可以启动。未设置连接信息时,服务会使用带演示数据的内存控制面和内存运行时:
```bash
make dev
```
另开终端启动 Web Console
```bash
make web-install
npm --prefix web run dev
```
浏览器访问 `http://localhost:5173`。API 默认监听 `http://localhost:8080`
完整依赖环境:
```bash
docker compose up --build
```
Compose 会启动 PostgreSQL、etcd、Redis、Config Server 和 Web Console。打开 `http://localhost:5173`。PostgreSQL migration 会在服务启动时幂等执行。
启用 Prometheus`http://localhost:9090`
```bash
docker compose --profile monitoring up --build
```
## 配置
| 环境变量 | 默认值 | 说明 |
|---|---:|---|
| `HTTP_ADDR` | `:8080` | HTTP 监听地址 |
| `DATABASE_URL` | 空 | 为空时使用内存控制面 |
| `ETCD_ENDPOINTS` | 空 | 逗号分隔;为空时使用内存运行时 |
| `ETCD_DIAL_TIMEOUT` | `5s` | etcd 连接超时 |
| `OUTBOX_INTERVAL` | `500ms` | outbox 扫描周期 |
| `OUTBOX_BATCH_SIZE` | `50` | 单次 claim 数量 |
| `OUTBOX_MAX_RETRY` | `12` | 下发失败最大重试次数 |
| `CORS_ALLOWED_ORIGINS` | `http://localhost:5173` | 允许的 Web Origin逗号分隔 |
| `AUTH_ENABLED` | `false` | 是否启用 JWT 登录与 RBAC |
| `JWT_SECRET` | 空 | HS256 密钥;启用认证时至少 32 个字符 |
| `JWT_TTL` | `8h` | 登录令牌有效期 |
| `BOOTSTRAP_ADMIN_USERNAME` | `admin` | 首次启动时幂等创建的管理员用户名 |
| `BOOTSTRAP_ADMIN_PASSWORD` | 空 | 启用认证时必填,至少 12 个字符 |
| `BOOTSTRAP_ADMIN_DISPLAY_NAME` | `Administrator` | 初始管理员显示名称 |
可复制 `.env.example` 后按部署环境调整。认证启用后,除健康检查、登录和 `/metrics` 外的接口都要求 `Authorization: Bearer <token>`。初始管理员只会在不存在时创建,不会在重启时覆盖已有密码。生产环境应通过 Secret 管理数据库密码、JWT 密钥和初始密码,并启用 PostgreSQL/etcd TLS。
## API
管理面响应统一为 `{ "data": ... }`,错误统一为 `{ "error": { "code", "message" } }`
| 方法 | 路径 | 说明 |
|---|---|---|
| `GET/POST` | `/v1/applications` | 应用列表/创建 |
| `PUT/DELETE` | `/v1/applications/{id}` | 应用更新/删除 |
| `GET/POST` | `/v1/environments` | 环境列表/创建 |
| `GET/POST` | `/v1/namespaces` | 命名空间列表/创建 |
| `GET/POST` | `/v1/config-items` | 配置项列表/创建 |
| `PUT/DELETE` | `/v1/config-items/{id}` | 修改/标记待删除 |
| `POST` | `/v1/config-items/{id}/restore` | 撤销待删除 |
| `POST` | `/v1/publish` | 创建发布版本,返回 `202` |
| `GET` | `/v1/releases?appId=&nsId=&envId=` | 发布历史与下发状态 |
| `POST` | `/v1/rollback` | 回滚并生成新版本 |
| `GET` | `/v1/audit-logs` | 审计日志 |
| `POST` | `/v1/auth/login` | 用户登录并签发 JWT |
| `GET` | `/v1/me` | 当前用户与应用角色 |
| `GET/POST` | `/v1/users` | 用户列表/创建(管理员) |
| `GET/PUT/DELETE` | `/v1/users/{id}/roles[/{appId}]` | 应用角色管理(管理员) |
| `GET/POST` | `/v1/gray-rules` | 当前范围灰度规则列表/创建 |
| `PUT/DELETE` | `/v1/gray-rules/{id}` | 灰度规则更新/删除 |
| `GET` | `/v1/config?env=&app=&namespace=&ip=&instance=` | 读取运行时快照并匹配灰度规则 |
| `GET` | `/v1/watch?env=&app=&namespace=&ip=&instance=` | SSE 全量同步与更新事件 |
| `GET` | `/metrics` | Prometheus 指标 |
认证关闭时服务以开发管理员身份运行,并可用 `X-User` 记录操作者;认证开启时操作者来自 JWT客户端不能通过请求头伪造。`viewer` 可以读取应用配置,`app-owner` 还可以维护命名空间、配置、发布、回滚和灰度规则,全局管理员可管理应用、环境、审计和用户授权。
### 灰度匹配
`ip` 规则接受精确 IP 或 CIDR`instance` 规则接受实例 ID 列表;`percentage` 使用 salt 与实例 ID无实例时使用 IP做稳定哈希同一实例不会随机漂移。多条规则命中时按优先级从低到高覆盖并在响应的 `grayRuleIds` 中返回命中规则。
### 发布示例
```bash
curl -X POST http://localhost:8080/v1/publish \
-H 'Content-Type: application/json' \
-H 'X-User: alice' \
-d '{"appId":1,"nsId":1,"envId":1,"comment":"enable feature"}'
```
数据库事务提交后返回 `pending`outbox 写入 etcd 后,`GET /v1/releases/{id}` 会变为 `applied` 并带上 `etcdRevision`
## SDK
Go SDK
```go
client, err := configsdk.New(configsdk.Options{
BaseURL: "http://localhost:8080",
Env: "PROD",
App: "order-service",
Token: os.Getenv("CONFIGCENTER_TOKEN"),
Instance: "order-3",
CacheFile: "/var/lib/my-service/configcenter.json",
})
if err != nil { /* handle */ }
_ = client.Load(ctx, "application")
go client.WatchAndSync(ctx, "application")
port := client.GetInt("application", "server.port", 8080)
```
Python SDK 位于 `sdk/python`
```python
from configcenter import ConfigClient
client = ConfigClient(
"http://localhost:8080", "PROD", "order-service", "/tmp/order-config.json",
token=os.environ.get("CONFIGCENTER_TOKEN"), instance="order-3",
)
client.load("application")
client.start_background_watch("application")
timeout = client.get("application", "order.timeout.minutes", "30")
```
本地缓存采用完整快照和原子 renameConfig Server/etcd 暂时不可用时,进程可以读取上一次成功同步的缓存启动。
## 开发与验证
```bash
make test
make build
make web-build
docker compose config
make integration-smoke
make loadtest
```
核心端到端测试覆盖CRUD → 发布事务 → outbox worker → 运行时读取 → 灰度覆盖与指标;单元/HTTP 集成测试还覆盖 JWT 过期与篡改、应用级 RBAC、多版本回滚和 SDK 鉴权/灰度参数。
压测工具默认验证运行时读取的错误率不超过 1%、p95 不超过 200ms可覆盖目标和阈值
```bash
go run ./cmd/loadtest \
-base-url=http://127.0.0.1:8080 -env=DEV -app=demo-service -namespace=application \
-token="$CONFIGCENTER_TOKEN" -concurrency=64 -duration=60s -max-p95=200ms
```
## 生产部署
`deploy/kubernetes/configcenter.yaml` 提供三副本无状态 Config Server、三节点 etcd、跨节点调度、PDB、HPA、探针和资源限制。部署前替换镜像与 Origin并创建密钥
```bash
kubectl create namespace configcenter
kubectl -n configcenter create secret generic configcenter-secrets \
--from-literal=DATABASE_URL='postgres://...' \
--from-literal=JWT_SECRET='至少32位随机密钥' \
--from-literal=BOOTSTRAP_ADMIN_PASSWORD='至少12位初始密码'
kubectl apply -f deploy/kubernetes/configcenter.yaml
```
生产 PostgreSQL 应使用托管高可用/主从方案etcd 应跨故障域部署,定期 snapshot、compact 和 defrag。示例清单使用明文集群内 etcd 地址,生产落地需补充双向 TLS 和只允许 Config Server 访问的网络策略。
## 目录
```text
cmd/server 服务入口
internal/api/httpapi REST + SSE
internal/store/postgres PostgreSQL repository 与 migration
internal/store/memory 本地开发/测试实现
internal/runtime/etcd etcd 快照与 Watch
internal/outbox 异步下发 worker
internal/watch 同 key Watch 扇出
internal/auth JWT、密码与应用级 RBAC
internal/gray 灰度规则校验与确定性匹配
internal/metrics Prometheus 指标
cmd/loadtest 运行时读取 SLO 压测
deploy Kubernetes 与监控告警配置
pkg/sdk/go Go SDK
sdk/python Python SDK
api/proto gRPC 契约
web React Web Console
```

View File

@@ -0,0 +1,77 @@
syntax = "proto3";
package configcenter.v1;
option go_package = "github.com/longpeng/configcenter/pkg/proto/v1;configcenterv1";
message ConfigItem {
string key = 1;
string value = 2;
}
message GetConfigRequest {
string env = 1;
string app = 2;
string namespace = 3;
}
message GetConfigResponse {
repeated ConfigItem items = 1;
int64 revision = 2;
int64 release_version = 3;
}
message WatchConfigRequest {
string env = 1;
string app = 2;
string namespace = 3;
int64 start_revision = 4;
}
message ConfigEvent {
enum EventType {
FULL_SYNC = 0;
UPDATED = 1;
}
EventType type = 1;
repeated ConfigItem items = 2;
int64 revision = 3;
}
message PublishRequest {
int64 env_id = 1;
int64 app_id = 2;
int64 namespace_id = 3;
string comment = 4;
string operator = 5;
}
message PublishResponse {
int64 release_id = 1;
int64 release_version = 2;
string status = 3;
}
message RollbackRequest {
int64 env_id = 1;
int64 app_id = 2;
int64 namespace_id = 3;
int64 target_version = 4;
string operator = 5;
}
message RollbackResponse {
int64 release_id = 1;
int64 new_release_version = 2;
string status = 3;
}
service ConfigService {
rpc GetConfig(GetConfigRequest) returns (GetConfigResponse);
rpc WatchConfig(WatchConfigRequest) returns (stream ConfigEvent);
}
service AdminService {
rpc PublishConfig(PublishRequest) returns (PublishResponse);
rpc RollbackConfig(RollbackRequest) returns (RollbackResponse);
}

107
cmd/loadtest/main.go Normal file
View File

@@ -0,0 +1,107 @@
package main
import (
"context"
"flag"
"fmt"
"io"
"net/http"
"net/url"
"os"
"sort"
"sync"
"time"
)
type result struct {
duration time.Duration
ok bool
}
func main() {
baseURL := flag.String("base-url", "http://127.0.0.1:8080", "Config Center base URL")
env := flag.String("env", "DEV", "environment code")
app := flag.String("app", "demo-service", "application code")
namespace := flag.String("namespace", "application", "namespace")
token := flag.String("token", "", "optional bearer token")
concurrency := flag.Int("concurrency", 32, "concurrent workers")
duration := flag.Duration("duration", 30*time.Second, "test duration")
maxErrorRate := flag.Float64("max-error-rate", 0.01, "failure threshold")
maxP95 := flag.Duration("max-p95", 200*time.Millisecond, "p95 latency threshold")
flag.Parse()
if *concurrency <= 0 || *duration <= 0 {
fmt.Fprintln(os.Stderr, "concurrency and duration must be positive")
os.Exit(2)
}
query := url.Values{"env": {*env}, "app": {*app}, "namespace": {*namespace}}
endpoint := fmt.Sprintf("%s/v1/config?%s", *baseURL, query.Encode())
client := &http.Client{Transport: &http.Transport{MaxIdleConns: *concurrency * 2, MaxIdleConnsPerHost: *concurrency, IdleConnTimeout: 30 * time.Second}, Timeout: 5 * time.Second}
ctx, cancel := context.WithTimeout(context.Background(), *duration)
defer cancel()
results := make(chan result, *concurrency*128)
var workers sync.WaitGroup
for i := 0; i < *concurrency; i++ {
workers.Add(1)
go func() {
defer workers.Done()
for ctx.Err() == nil {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return
}
if *token != "" {
request.Header.Set("Authorization", "Bearer "+*token)
}
begin := time.Now()
response, err := client.Do(request)
elapsed := time.Since(begin)
if err != nil && ctx.Err() != nil {
return
}
ok := err == nil && response.StatusCode == http.StatusOK
if response != nil {
_, _ = io.Copy(io.Discard, response.Body)
response.Body.Close()
}
select {
case results <- result{duration: elapsed, ok: ok}:
case <-ctx.Done():
return
}
}
}()
}
go func() {
workers.Wait()
close(results)
}()
latencies := make([]time.Duration, 0)
failures := 0
for item := range results {
latencies = append(latencies, item.duration)
if !item.ok {
failures++
}
}
if len(latencies) == 0 {
fmt.Fprintln(os.Stderr, "no requests completed")
os.Exit(1)
}
sort.Slice(latencies, func(i, j int) bool { return latencies[i] < latencies[j] })
p50, p95, p99 := percentile(latencies, 0.50), percentile(latencies, 0.95), percentile(latencies, 0.99)
errorRate := float64(failures) / float64(len(latencies))
qps := float64(len(latencies)) / duration.Seconds()
fmt.Printf("requests=%d failures=%d error_rate=%.4f qps=%.1f p50=%s p95=%s p99=%s\n", len(latencies), failures, errorRate, qps, p50, p95, p99)
if errorRate > *maxErrorRate || p95 > *maxP95 {
fmt.Fprintf(os.Stderr, "SLO failed: error_rate<=%.4f p95<=%s\n", *maxErrorRate, *maxP95)
os.Exit(1)
}
}
func percentile(values []time.Duration, quantile float64) time.Duration {
index := int(float64(len(values)-1) * quantile)
return values[index]
}

121
cmd/server/main.go Normal file
View File

@@ -0,0 +1,121 @@
package main
import (
"context"
"flag"
"log/slog"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/longpeng/configcenter/internal/api/httpapi"
"github.com/longpeng/configcenter/internal/auth"
"github.com/longpeng/configcenter/internal/config"
"github.com/longpeng/configcenter/internal/metrics"
"github.com/longpeng/configcenter/internal/outbox"
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
etcdstore "github.com/longpeng/configcenter/internal/runtime/etcd"
memoryruntime "github.com/longpeng/configcenter/internal/runtime/memory"
storepkg "github.com/longpeng/configcenter/internal/store"
memory_store "github.com/longpeng/configcenter/internal/store/memory"
postgres_store "github.com/longpeng/configcenter/internal/store/postgres"
"github.com/longpeng/configcenter/internal/watch"
)
func main() {
migrateOnly := flag.Bool("migrate-only", false, "apply database migrations and exit")
flag.Parse()
logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))
cfg := config.Load()
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
repository, err := openStore(ctx, cfg, logger)
if err != nil {
logger.Error("open control-plane store", "error", err)
os.Exit(1)
}
defer repository.Close()
if *migrateOnly {
logger.Info("database migrations applied")
return
}
authorizer, err := auth.New(repository, cfg.AuthEnabled, cfg.JWTSecret, cfg.JWTTTL)
if err != nil {
logger.Error("initialize authentication", "error", err)
os.Exit(1)
}
if err := authorizer.Bootstrap(ctx, cfg.BootstrapAdminUsername, cfg.BootstrapAdminPassword, cfg.BootstrapAdminDisplayName); err != nil {
logger.Error("bootstrap administrator", "error", err)
os.Exit(1)
}
runtimeStore, err := openRuntime(cfg, logger)
if err != nil {
logger.Error("open runtime store", "error", err)
os.Exit(1)
}
defer runtimeStore.Close() //nolint:errcheck
hub := watch.New(ctx, runtimeStore)
collector := metrics.New()
worker := outbox.New(repository, runtimeStore, cfg.OutboxInterval, cfg.OutboxBatchSize, cfg.OutboxMaxRetry, collector, logger)
go worker.Run(ctx)
api := httpapi.New(repository, runtimeStore, hub, authorizer, collector, cfg.AllowedOrigins, logger)
server := &http.Server{
Addr: cfg.HTTPAddr,
Handler: api.Handler(),
ReadHeaderTimeout: 5 * time.Second,
ReadTimeout: 15 * time.Second,
IdleTimeout: 60 * time.Second,
}
serverErrors := make(chan error, 1)
go func() {
logger.Info("config center listening", "address", cfg.HTTPAddr)
serverErrors <- server.ListenAndServe()
}()
select {
case <-ctx.Done():
logger.Info("shutting down config center")
case err := <-serverErrors:
if err != nil && err != http.ErrServerClosed {
logger.Error("http server stopped", "error", err)
}
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.ShutdownTimeout)
defer cancel()
if err := server.Shutdown(shutdownCtx); err != nil {
logger.Error("graceful shutdown", "error", err)
}
}
func openStore(ctx context.Context, cfg config.Config, logger *slog.Logger) (storepkg.Store, error) {
if cfg.DatabaseURL == "" {
logger.Warn("DATABASE_URL is empty; using the in-memory control-plane store")
return memory_store.New(true), nil
}
repository, err := postgres_store.Open(ctx, cfg.DatabaseURL)
if err != nil {
return nil, err
}
if err := repository.Migrate(ctx); err != nil {
repository.Close()
return nil, err
}
return repository, nil
}
func openRuntime(cfg config.Config, logger *slog.Logger) (runtimepkg.Store, error) {
if len(cfg.EtcdEndpoints) == 0 {
logger.Warn("ETCD_ENDPOINTS is empty; using the in-memory runtime store")
return memoryruntime.New(), nil
}
return etcdstore.Open(cfg.EtcdEndpoints, cfg.EtcdDialTimeout)
}

97
compose.yaml Normal file
View File

@@ -0,0 +1,97 @@
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: configcenter
POSTGRES_USER: configcenter
POSTGRES_PASSWORD: configcenter
healthcheck:
test: ["CMD-SHELL", "pg_isready -U configcenter -d configcenter"]
interval: 5s
timeout: 3s
retries: 20
volumes:
- postgres-data:/var/lib/postgresql/data
etcd:
image: quay.io/coreos/etcd:v3.5.17
command:
- /usr/local/bin/etcd
- --name=etcd-1
- --data-dir=/etcd-data
- --listen-client-urls=http://0.0.0.0:2379
- --advertise-client-urls=http://etcd:2379
healthcheck:
test: ["CMD", "etcdctl", "endpoint", "health"]
interval: 5s
timeout: 3s
retries: 20
volumes:
- etcd-data:/etcd-data
redis:
image: redis:7.4-alpine
command: ["redis-server", "--appendonly", "yes"]
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 20
volumes:
- redis-data:/data
server:
build:
context: .
dockerfile: Dockerfile
environment:
HTTP_ADDR: :8080
DATABASE_URL: postgres://configcenter:configcenter@postgres:5432/configcenter?sslmode=disable
ETCD_ENDPOINTS: http://etcd:2379
CORS_ALLOWED_ORIGINS: http://localhost:5173
AUTH_ENABLED: ${AUTH_ENABLED:-false}
JWT_SECRET: ${JWT_SECRET:-}
JWT_TTL: ${JWT_TTL:-8h}
BOOTSTRAP_ADMIN_USERNAME: ${BOOTSTRAP_ADMIN_USERNAME:-admin}
BOOTSTRAP_ADMIN_PASSWORD: ${BOOTSTRAP_ADMIN_PASSWORD:-}
BOOTSTRAP_ADMIN_DISPLAY_NAME: ${BOOTSTRAP_ADMIN_DISPLAY_NAME:-Administrator}
depends_on:
postgres:
condition: service_healthy
etcd:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8080/health/ready"]
interval: 5s
timeout: 3s
retries: 20
ports:
- "8080:8080"
web:
build:
context: .
dockerfile: web/Dockerfile
depends_on:
server:
condition: service_healthy
ports:
- "5173:80"
prometheus:
image: prom/prometheus:v3.2.1
profiles: ["monitoring"]
command: ["--config.file=/etc/prometheus/prometheus.yml"]
volumes:
- ./deploy/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./deploy/monitoring/alerts.yml:/etc/prometheus/alerts.yml:ro
depends_on:
server:
condition: service_healthy
ports:
- "9090:9090"
volumes:
postgres-data:
etcd-data:
redis-data:

View File

@@ -0,0 +1,506 @@
# 自建配置中心实现方案etcd + PostgreSQL + Redis + Go
对应架构:
```
Web Console
Config Server (Go)
┌──────────────┼──────────────┐
│ │ │
PostgreSQL etcd Redis
配置/版本/RBAC 生效配置 缓存/事件
发布记录/审计 Watch
```
## 一、核心设计原则
**PostgreSQL 是控制面source of truthetcd 是数据面(运行时分发)。**
配置的"编辑态"(草稿)永远只存在 PostgreSQL 里;只有点了「发布」,才会把该 namespace 当前生效的完整配置写进 etcd。etcd 里任何时刻只保存"当前生效"的那一份历史版本、审计、RBAC 全部留在 PG。
这样划分带来两个好处:
- etcd 数据量可控etcd 官方建议库大小控制在几个 GB 内,不适合塞历史/审计数据);
- 复杂查询按部门筛选应用、按时间查审计日志交给关系型数据库etcd 只做它最擅长的事:强一致 + Watch。
## 二、发布语义与一致性Outbox 模式)
发布动作要跨两个存储写入,如果直接「先写 PG 再写 etcd」中间进程崩溃会导致状态不一致PG 说已发布etcd 里其实是旧值)。
解决办法:**Outbox 模式**——发布在同一个 PG 事务里写入 `releases` 记录和 `release_outbox` 记录;一个独立的异步 worker 轮询 outbox把内容幂等地 Put 进 etcd成功后回写 `etcd_revision` 并标记 `applied`。这样即使 Config Server 中途重启,未完成的发布也能被 worker 续上不会丢失也不会重复产生副作用Put 本身是幂等的)。
```
发布请求
PG 事务:写 releases + release_outboxDB 提交即成功返回给用户)
Outbox Worker异步、可重试
etcd.Put(key, snapshot) ──成功──▶ 回写 etcd_revisionrelease.status = applied
└─失败──▶ retry_count++,下一轮重试
```
## 三、etcd Key 设计
推荐按 **namespace 整体做一个 blob**,而不是每个 key 单独存一条,理由:一次发布往往同时改多个 key整体 blob 能保证客户端拿到的永远是"某个版本的完整快照",不会出现 watch 到一半、配置项之间不一致的中间态。
```
/config/{env}/{app}/{namespace} -> JSON: {"server.port":"8080", "log.level":"INFO", ...}
/config/{env}/{app}/{namespace}/__meta -> JSON: {"version":12,"releaseId":88,"publishedAt":"...","publishedBy":"admin"}
```
客户端只 watch `/config/{env}/{app}/{namespace}` 这一个 key收到事件后解码出完整 map本地直接替换。
## 四、PostgreSQL 数据模型
```sql
CREATE TABLE applications (
id BIGSERIAL PRIMARY KEY,
app_code VARCHAR(64) UNIQUE NOT NULL,
name VARCHAR(128) NOT NULL,
department VARCHAR(128),
owner VARCHAR(64),
description TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE environments (
id BIGSERIAL PRIMARY KEY,
env_code VARCHAR(32) UNIQUE NOT NULL, -- DEV/TEST/STAGING/PROD
name VARCHAR(64) NOT NULL,
sort_order INT NOT NULL DEFAULT 0,
description TEXT
);
CREATE TABLE namespaces (
id BIGSERIAL PRIMARY KEY,
app_id BIGINT NOT NULL REFERENCES applications(id) ON DELETE CASCADE,
name VARCHAR(64) NOT NULL,
format VARCHAR(16) NOT NULL DEFAULT 'properties', -- properties/yaml/json/xml
description TEXT,
UNIQUE(app_id, name)
);
-- 草稿态配置项value 是编辑中的值released_value 是最近一次发布生效的值
CREATE TABLE config_items (
id BIGSERIAL PRIMARY KEY,
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
key VARCHAR(256) NOT NULL,
value TEXT NOT NULL,
released_value TEXT, -- NULL 表示从未发布过(待发布·新增)
pending_delete BOOLEAN NOT NULL DEFAULT false,
comment TEXT,
updated_by VARCHAR(64),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE(namespace_id, env_id, key)
);
CREATE TABLE releases (
id BIGSERIAL PRIMARY KEY,
namespace_id BIGINT NOT NULL REFERENCES namespaces(id),
env_id BIGINT NOT NULL REFERENCES environments(id),
version INT NOT NULL,
snapshot JSONB NOT NULL, -- 该版本完整的 key -> value
diff_added INT DEFAULT 0,
diff_modified INT DEFAULT 0,
diff_removed INT DEFAULT 0,
comment TEXT,
operator VARCHAR(64) NOT NULL,
etcd_revision BIGINT, -- 写入 etcd 成功后回填,用于对账
status VARCHAR(16) NOT NULL DEFAULT 'pending', -- pending/applied/failed
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE(namespace_id, env_id, version)
);
-- Outbox保证 DB 事务与 etcd 写入之间的最终一致
CREATE TABLE release_outbox (
id BIGSERIAL PRIMARY KEY,
release_id BIGINT NOT NULL REFERENCES releases(id),
etcd_key VARCHAR(512) NOT NULL,
payload JSONB NOT NULL,
status VARCHAR(16) NOT NULL DEFAULT 'pending', -- pending/done/failed
retry_count INT NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
username VARCHAR(64) UNIQUE NOT NULL,
password_hash VARCHAR(256) NOT NULL,
display_name VARCHAR(64),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE roles (
id BIGSERIAL PRIMARY KEY,
name VARCHAR(32) UNIQUE NOT NULL -- admin / app-owner / viewer
);
CREATE TABLE user_app_roles ( -- 用户对某个应用的角色(应用级 RBAC
user_id BIGINT NOT NULL REFERENCES users(id),
app_id BIGINT NOT NULL REFERENCES applications(id),
role_id BIGINT NOT NULL REFERENCES roles(id),
PRIMARY KEY (user_id, app_id)
);
CREATE TABLE audit_logs (
id BIGSERIAL PRIMARY KEY,
actor VARCHAR(64) NOT NULL,
action VARCHAR(32) NOT NULL, -- create/update/delete/publish/rollback
target_type VARCHAR(32) NOT NULL, -- app/namespace/env/config/release
target_id BIGINT,
detail JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- 灰度发布规则可选Phase 4 再做)
CREATE TABLE gray_rules (
id BIGSERIAL PRIMARY KEY,
namespace_id BIGINT NOT NULL REFERENCES namespaces(id),
env_id BIGINT NOT NULL REFERENCES environments(id),
rule_type VARCHAR(16) NOT NULL, -- ip / instance / percentage
rule_value JSONB NOT NULL, -- {"ips": [...]} 或 {"percentage": 20}
overrides JSONB NOT NULL, -- 灰度期间覆盖的 key -> value
enabled BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```
## 五、Protobuf / gRPC 接口定义
建议用 **grpc-gateway** 从同一份 proto 同时生成 gRPC给 Go/Python SDK 用)和 REST给 Web Console 用),避免维护两套接口。
```protobuf
syntax = "proto3";
package configcenter.v1;
option go_package = "github.com/yourorg/configcenter/pkg/proto/v1;configcenterv1";
message ConfigItem {
string key = 1;
string value = 2;
}
// ---- 读取 / 监听 ----
message GetConfigRequest {
string env = 1;
string app = 2;
string namespace = 3;
}
message GetConfigResponse {
repeated ConfigItem items = 1;
int64 revision = 2;
int64 release_version = 3;
}
message WatchConfigRequest {
string env = 1;
string app = 2;
string namespace = 3;
int64 start_revision = 4; // 断线重连时从该 revision 继续,避免错过中间事件
}
message ConfigEvent {
enum EventType { FULL_SYNC = 0; UPDATED = 1; }
EventType type = 1;
repeated ConfigItem items = 2;
int64 revision = 3;
}
service ConfigService {
rpc GetConfig(GetConfigRequest) returns (GetConfigResponse);
rpc WatchConfig(WatchConfigRequest) returns (stream ConfigEvent);
}
// ---- 发布 / 回滚管理面Web Console 也走这里)----
message PublishRequest {
string env = 1;
string app = 2;
string namespace = 3;
string comment = 4;
string operator = 5;
}
message PublishResponse {
int64 release_version = 1;
int64 etcd_revision = 2;
}
message RollbackRequest {
string env = 1;
string app = 2;
string namespace = 3;
int64 target_version = 4;
string operator = 5;
}
message RollbackResponse {
int64 new_release_version = 1;
}
service AdminService {
rpc PublishConfig(PublishRequest) returns (PublishResponse);
rpc RollbackConfig(RollbackRequest) returns (RollbackResponse);
// 应用/命名空间/环境/配置项的 CRUD 接口类似,此处略
}
```
## 六、Go Config Server 实现要点
### 目录结构
```
configcenter/
├── cmd/server/main.go
├── internal/
│ ├── api/ # gRPC handler + grpc-gateway REST
│ ├── service/ # 发布 / 回滚 / watch 扇出等业务逻辑
│ ├── store/
│ │ ├── postgres/ # repository推荐 sqlc 生成,类型安全)
│ │ └── etcdstore/ # etcd client 封装
│ ├── outbox/ # outbox worker
│ ├── cache/ # redis 封装(读缓存 + 内部事件)
│ └── auth/ # JWT + RBAC 中间件
├── pkg/
│ ├── proto/v1/ # protoc 生成代码
│ └── sdk/go/ # Go SDK独立 module方便业务方单独引用
└── web/ # Web Console 前端
```
### 发布流程
```go
func (s *ConfigService) Publish(ctx context.Context, req *PublishRequest) (*PublishResponse, error) {
return s.db.WithTx(ctx, func(tx *sql.Tx) (*PublishResponse, error) {
items, err := s.repo.ListConfigItems(tx, req.NamespaceID, req.EnvID)
if err != nil {
return nil, err
}
pending := filterPending(items) // pendingDelete || releasedValue==nil || value!=releasedValue
if len(pending) == 0 {
return nil, ErrNoPendingChanges
}
version := s.repo.NextReleaseVersion(tx, req.NamespaceID, req.EnvID)
snapshot := buildSnapshot(items) // 去掉 pendingDelete 的 key
releaseID, err := s.repo.InsertRelease(tx, version, snapshot, req.Comment, req.Operator)
if err != nil {
return nil, err
}
// 关键:写 outbox 而不是直接写 etcd和 DB 事务在同一个提交单元里
etcdKey := buildEtcdKey(req.Env, req.App, req.Namespace)
if err := s.repo.InsertOutbox(tx, releaseID, etcdKey, snapshot); err != nil {
return nil, err
}
s.repo.MarkConfigItemsReleased(tx, pending)
return &PublishResponse{ReleaseVersion: version}, nil
})
}
```
### Outbox Worker异步写 etcd失败可重试
```go
func (w *OutboxWorker) Run(ctx context.Context) {
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
rows, _ := w.repo.FetchPendingOutbox(ctx, 50)
for _, row := range rows {
payload, _ := json.Marshal(row.Payload)
resp, err := w.etcd.Put(ctx, row.EtcdKey, string(payload))
if err != nil {
w.repo.MarkOutboxFailed(ctx, row.ID) // retry_count++
continue
}
w.repo.MarkOutboxDone(ctx, row.ID)
w.repo.UpdateReleaseRevision(ctx, row.ReleaseID, resp.Header.Revision, "applied")
}
}
}
}
```
### Watch 扇出(避免 N 个 Server 实例 = N 倍 etcd watch 连接)
```go
type WatchHub struct {
mu sync.RWMutex
subs map[string]map[chan *pb.ConfigEvent]struct{}
etcd *clientv3.Client
}
func (h *WatchHub) Subscribe(key string) (<-chan *pb.ConfigEvent, func()) {
ch := make(chan *pb.ConfigEvent, 8)
h.mu.Lock()
if h.subs[key] == nil {
h.subs[key] = map[chan *pb.ConfigEvent]struct{}{}
go h.watchKey(key) // 同一个 key 只建一条 etcd watch
}
h.subs[key][ch] = struct{}{}
h.mu.Unlock()
return ch, func() { h.unsubscribe(key, ch) }
}
func (h *WatchHub) watchKey(key string) {
wc := h.etcd.Watch(context.Background(), key)
for resp := range wc {
for _, ev := range resp.Events {
evt := &pb.ConfigEvent{Type: pb.ConfigEvent_UPDATED, Items: decode(ev.Kv.Value), Revision: ev.Kv.ModRevision}
h.broadcast(key, evt)
}
}
}
```
gRPC 层只需要订阅 Hub 并转发给客户端流:
```go
func (s *ConfigService) WatchConfig(req *pb.WatchConfigRequest, stream pb.ConfigService_WatchConfigServer) error {
key := buildEtcdKey(req.Env, req.App, req.Namespace)
initial, rev := s.loadCurrent(key)
if err := stream.Send(&pb.ConfigEvent{Type: pb.ConfigEvent_FULL_SYNC, Items: initial, Revision: rev}); err != nil {
return err
}
ch, cancel := s.hub.Subscribe(key)
defer cancel()
for {
select {
case evt := <-ch:
if err := stream.Send(evt); err != nil {
return err
}
case <-stream.Context().Done():
return nil
}
}
}
```
### 回滚
回滚不是"覆盖旧记录",而是**生成一条新的 release**(内容等于目标历史版本的 snapshot走和发布完全一样的 outbox 流程——这样回滚本身也留痕,历史可追溯。
## 七、Go SDK
```go
package configsdk
type Client struct {
stub pb.ConfigServiceClient
cache sync.Map // namespace -> map[string]string
}
func New(addr string) (*Client, error) {
conn, err := grpc.Dial(addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
return nil, err
}
return &Client{stub: pb.NewConfigServiceClient(conn)}, nil
}
func (c *Client) GetString(namespace, key, defaultVal string) string {
if m, ok := c.cache.Load(namespace); ok {
if v, ok := m.(map[string]string)[key]; ok {
return v
}
}
return defaultVal
}
// 后台常驻 goroutine断线自动重连
func (c *Client) WatchAndSync(ctx context.Context, env, app, namespace string) {
for {
stream, err := c.stub.WatchConfig(ctx, &pb.WatchConfigRequest{Env: env, App: app, Namespace: namespace})
if err == nil {
for {
evt, err := stream.Recv()
if err != nil {
break
}
c.applyEvent(namespace, evt)
}
}
select {
case <-ctx.Done():
return
case <-time.After(2 * time.Second): // 重连退避
}
}
}
```
## 八、Python SDK
不直接连 etcdetcd 官方不维护 Python client而是通过 gRPC 连 Go Config Server
```python
import grpc
import threading
import time
from configcenter.v1 import config_pb2, config_pb2_grpc
class ConfigClient:
def __init__(self, addr: str):
self._channel = grpc.insecure_channel(addr)
self._stub = config_pb2_grpc.ConfigServiceStub(self._channel)
self._cache = {}
self._lock = threading.Lock()
def get(self, namespace: str, key: str, default=None):
with self._lock:
return self._cache.get(namespace, {}).get(key, default)
def start_background_watch(self, env: str, app: str, namespace: str):
t = threading.Thread(target=self._watch_loop, args=(env, app, namespace), daemon=True)
t.start()
def _watch_loop(self, env, app, namespace):
req = config_pb2.WatchConfigRequest(env=env, app=app, namespace=namespace)
while True:
try:
for event in self._stub.WatchConfig(req):
with self._lock:
self._cache[namespace] = {i.key: i.value for i in event.items}
except grpc.RpcError:
time.sleep(2) # 断线重连
```
## 九、Web Console 对接方式
之前给你做的前端 Demo应用/命名空间/环境/配置项 CRUD + 发布 diff 预览 + 发布历史回滚)里的数据结构和这里的 PG 表结构是对齐的,接入真实后端时只需要:
1. 把内存里的 `useState` 数据源换成对 `grpc-gateway` 生成的 REST 接口的 `fetch` 调用;
2. 发布按钮调用 `POST /v1/publish`,历史列表调用 `GET /v1/releases`,回滚调用 `POST /v1/rollback`
3. 因为发布是异步落 etcdoutboxWeb Console 发布后可以轮询 release 的 `status` 字段pending → applied或者用 Redis pub/sub + SSE 做实时状态推送。
## 十、部署与运维要点
- **etcd 独立部署**3 或 5 节点集群,不与业务系统共享,专门服务配置分发;
- **控制 etcd 数据量**:只放"当前生效配置",历史/审计一律留在 PGetcd 库大小尽量控制在几个 GB 以内;
- **定期 compact + defrag**etcd 的 MVCC 历史 revision 会持续占用磁盘,需要定期 `etcdctl compact``defrag`
- **监控**etcd 自带 `/metrics`Prometheus重点关注 db 大小、leader 变更次数、watch 连接数Config Server 侧关注发布成功率、outbox 积压量、gRPC 延迟;
- **无状态水平扩展**Config Server 本身无状态,可以随意加实例,配合前面的 WatchHub 设计,不会导致 etcd 连接数暴涨;
- **灾备**etcd 定期 `snapshot save`PostgreSQL 走常规主从 + 定期备份;
- **安全**etcd 只对 Config Server 开放访问不直接暴露给业务服务Config Server 与 etcd 之间开 TLS。
## 十一、分阶段落地路线图
| 阶段 | 内容 | 周期建议 |
|---|---|---|
| Phase 1 | PG 建表 + CRUD REST API + Web Console 接入真实接口(发布先只落 PG不接 etcd | 23 周 |
| Phase 2 | 接入 etcdOutbox Worker、GetConfig 接口、Go/Python SDK先只支持 Get不支持 Watch | 2 周 |
| Phase 3 | WatchConfig 全链路WatchHub 扇出、SDK 长连接自动重连、**SDK 本地文件缓存兜底**Server/etcd 故障时业务进程仍能用最后一次缓存启动这是必须做的Apollo/Nacos 都有) | 2 周 |
| Phase 4 | RBAC应用级角色、审计日志、灰度发布规则 | 23 周 |
| Phase 5 | 高可用加固etcd 多机房、监控告警、压测 | 持续 |
Phase 1 完全可以复用你现在这份前端 Demo 直接改造,不用重写界面。

View File

@@ -0,0 +1,218 @@
apiVersion: v1
kind: Namespace
metadata:
name: configcenter
---
apiVersion: v1
kind: ConfigMap
metadata:
name: configcenter
namespace: configcenter
data:
HTTP_ADDR: ":8080"
ETCD_ENDPOINTS: "http://etcd:2379"
OUTBOX_INTERVAL: "500ms"
OUTBOX_BATCH_SIZE: "50"
OUTBOX_MAX_RETRY: "12"
AUTH_ENABLED: "true"
JWT_TTL: "8h"
CORS_ALLOWED_ORIGINS: "https://config.example.com"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: etcd
namespace: configcenter
spec:
serviceName: etcd-peer
replicas: 3
selector:
matchLabels:
app: etcd
template:
metadata:
labels:
app: etcd
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "2379"
prometheus.io/path: /metrics
spec:
terminationGracePeriodSeconds: 30
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels: { app: etcd }
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: etcd
topologyKey: kubernetes.io/hostname
containers:
- name: etcd
image: quay.io/coreos/etcd:v3.5.17
command: ["/bin/sh", "-ec"]
args:
- >-
exec etcd
--name="${HOSTNAME}"
--data-dir=/var/lib/etcd
--listen-client-urls=http://0.0.0.0:2379
--advertise-client-urls="http://${HOSTNAME}.etcd-peer.configcenter.svc.cluster.local:2379"
--listen-peer-urls=http://0.0.0.0:2380
--initial-advertise-peer-urls="http://${HOSTNAME}.etcd-peer.configcenter.svc.cluster.local:2380"
--initial-cluster="etcd-0=http://etcd-0.etcd-peer.configcenter.svc.cluster.local:2380,etcd-1=http://etcd-1.etcd-peer.configcenter.svc.cluster.local:2380,etcd-2=http://etcd-2.etcd-peer.configcenter.svc.cluster.local:2380"
--initial-cluster-state=new
--initial-cluster-token=configcenter-etcd
--auto-compaction-retention=1
--auto-compaction-mode=periodic
ports:
- { name: client, containerPort: 2379 }
- { name: peer, containerPort: 2380 }
readinessProbe:
exec:
command: ["etcdctl", "endpoint", "health"]
initialDelaySeconds: 10
periodSeconds: 5
resources:
requests: { cpu: "250m", memory: "512Mi" }
limits: { cpu: "2", memory: "2Gi" }
volumeMounts:
- { name: data, mountPath: /var/lib/etcd }
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 20Gi
---
apiVersion: v1
kind: Service
metadata:
name: etcd-peer
namespace: configcenter
spec:
clusterIP: None
publishNotReadyAddresses: true
selector: { app: etcd }
ports:
- { name: peer, port: 2380, targetPort: peer }
- { name: client, port: 2379, targetPort: client }
---
apiVersion: v1
kind: Service
metadata:
name: etcd
namespace: configcenter
spec:
selector: { app: etcd }
ports:
- { name: client, port: 2379, targetPort: client }
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: etcd
namespace: configcenter
spec:
maxUnavailable: 1
selector:
matchLabels: { app: etcd }
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: configcenter
namespace: configcenter
spec:
replicas: 3
selector:
matchLabels: { app: configcenter }
template:
metadata:
labels: { app: configcenter }
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "8080"
prometheus.io/path: /metrics
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels: { app: configcenter }
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchLabels: { app: configcenter }
topologyKey: kubernetes.io/hostname
containers:
- name: server
image: ghcr.io/yourorg/configcenter:latest
envFrom:
- configMapRef: { name: configcenter }
- secretRef: { name: configcenter-secrets }
ports:
- { name: http, containerPort: 8080 }
readinessProbe:
httpGet: { path: /health/ready, port: http }
periodSeconds: 5
livenessProbe:
httpGet: { path: /health/live, port: http }
periodSeconds: 10
resources:
requests: { cpu: "200m", memory: "256Mi" }
limits: { cpu: "1", memory: "1Gi" }
---
apiVersion: v1
kind: Service
metadata:
name: configcenter
namespace: configcenter
spec:
selector: { app: configcenter }
ports:
- { name: http, port: 8080, targetPort: http }
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: configcenter
namespace: configcenter
spec:
minAvailable: 2
selector:
matchLabels: { app: configcenter }
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: configcenter
namespace: configcenter
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: configcenter
minReplicas: 3
maxReplicas: 10
behavior:
scaleDown:
stabilizationWindowSeconds: 300
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 65

View File

@@ -0,0 +1,54 @@
groups:
- name: configcenter
rules:
- alert: ConfigCenterDown
expr: up{job="configcenter"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: Config Center is unavailable
- alert: ConfigCenterOutboxBacklog
expr: configcenter_outbox_pending > 100
for: 5m
labels:
severity: warning
annotations:
summary: Config Center outbox backlog is growing
- alert: ConfigCenterOutboxFailed
expr: configcenter_outbox_failed > 0
for: 1m
labels:
severity: critical
annotations:
summary: Config Center has terminally failed releases
- alert: ConfigCenterHighHTTPErrorRate
expr: |
sum(rate(configcenter_http_requests_total{status=~"5.."}[5m]))
/
clamp_min(sum(rate(configcenter_http_requests_total[5m])), 0.001)
> 0.01
for: 5m
labels:
severity: warning
annotations:
summary: Config Center 5xx rate exceeds 1%
- alert: EtcdDown
expr: up{job="etcd"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: Config Center etcd is unavailable
- alert: EtcdNoLeader
expr: etcd_server_has_leader{job="etcd"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: Config Center etcd member has no leader

View File

@@ -0,0 +1,17 @@
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files:
- /etc/prometheus/alerts.yml
scrape_configs:
- job_name: configcenter
metrics_path: /metrics
static_configs:
- targets: ["server:8080"]
- job_name: etcd
metrics_path: /metrics
static_configs:
- targets: ["etcd:2379"]

32
go.mod Normal file
View File

@@ -0,0 +1,32 @@
module github.com/longpeng/configcenter
go 1.24.0
require (
github.com/jackc/pgx/v5 v5.7.2
go.etcd.io/etcd/client/v3 v3.6.7
)
require (
github.com/coreos/go-semver v0.3.1 // indirect
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
go.etcd.io/etcd/api/v3 v3.6.7 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.42.0 // indirect
golang.org/x/net v0.45.0 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/sys v0.36.0 // indirect
golang.org/x/text v0.29.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/grpc v1.71.1 // indirect
google.golang.org/protobuf v1.36.5 // indirect
)

112
go.sum Normal file
View File

@@ -0,0 +1,112 @@
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 h1:5ZPtiqj0JL5oKWmcsq4VMaAW5ukBEgSGXEN89zeH1Jo=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3/go.mod h1:ndYquD05frm2vACXE1nsccT4oJzjhw2arTS2cpUD1PI=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.7.2 h1:mLoDLV6sonKlvjIEsV56SkWNCnuNv531l94GaIzO+XI=
github.com/jackc/pgx/v5 v5.7.2/go.mod h1:ncY89UGWxg82EykZUwSpUKEfccBGGYq1xjrOpsbsfGQ=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
go.etcd.io/etcd/api/v3 v3.6.7 h1:7BNJ2gQmc3DNM+9cRkv7KkGQDayElg8x3X+tFDYS+E0=
go.etcd.io/etcd/api/v3 v3.6.7/go.mod h1:xJ81TLj9hxrYYEDmXTeKURMeY3qEDN24hqe+q7KhbnI=
go.etcd.io/etcd/client/pkg/v3 v3.6.7 h1:vvzgyozz46q+TyeGBuFzVuI53/yd133CHceNb/AhBVs=
go.etcd.io/etcd/client/pkg/v3 v3.6.7/go.mod h1:2IVulJ3FZ/czIGl9T4lMF1uxzrhRahLqe+hSgy+Kh7Q=
go.etcd.io/etcd/client/v3 v3.6.7 h1:9WqA5RpIBtdMxAy1ukXLAdtg2pAxNqW5NUoO2wQrE6U=
go.etcd.io/etcd/client/v3 v3.6.7/go.mod h1:2XfROY56AXnUqGsvl+6k29wrwsSbEh1lAouQB1vHpeE=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
go.opentelemetry.io/otel v1.34.0/go.mod h1:OWFPOQ+h4G8xpyjgqo4SxJYdDQ/qmRH+wivy7zzx9oI=
go.opentelemetry.io/otel/metric v1.34.0 h1:+eTR3U0MyfWjRDhmFMxe2SsW64QrZ84AOhvqS7Y+PoQ=
go.opentelemetry.io/otel/metric v1.34.0/go.mod h1:CEDrp0fy2D0MvkXE+dPV7cMi8tWZwX3dmaIhwPOaqHE=
go.opentelemetry.io/otel/sdk v1.34.0 h1:95zS4k/2GOy069d321O8jWgYsW3MzVV+KuSPKp7Wr1A=
go.opentelemetry.io/otel/sdk v1.34.0/go.mod h1:0e/pNiaMAqaykJGKbi+tSjWfNNHMTxoC9qANsCzbyxU=
go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.42.0 h1:chiH31gIWm57EkTXpwnqf8qeuMUi0yekh6mT2AvFlqI=
golang.org/x/crypto v0.42.0/go.mod h1:4+rDnOTJhQCx2q7/j6rAN5XDw8kPjeaXEUR2eL94ix8=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.45.0 h1:RLBg5JKixCy82FtLJpeNlVM0nrSqpCRYzVU1n8kj0tM=
golang.org/x/net v0.45.0/go.mod h1:ECOoLqd5U3Lhyeyo/QDCEVQ4sNgYsqvCZ722XogGieY=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb h1:p31xT4yrYrSM/G4Sn2+TNUkVhFCbG9y8itM2S6Th950=
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb/go.mod h1:jbe3Bkdp+Dh2IrslsFCklNhweNTBgSYanP1UXhJDhKg=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb h1:TLPQVbx1GJ8VKZxz52VAxl1EBgKXXbTiU9Fc5fZeLn4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb/go.mod h1:LuRYeWDFV6WOn90g357N17oMCaxpgCnbi/44qJvDn2I=
google.golang.org/grpc v1.71.1 h1:ffsFWr7ygTUscGPI0KKK6TLrGz0476KUvvsbqWK0rPI=
google.golang.org/grpc v1.71.1/go.mod h1:H0GRtasmQOh9LkFoCPDu3ZrwUtD1YGE+b2vYBYd/8Ec=
google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM=
google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,232 @@
package httpapi_test
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/longpeng/configcenter/internal/api/httpapi"
"github.com/longpeng/configcenter/internal/auth"
"github.com/longpeng/configcenter/internal/domain"
"github.com/longpeng/configcenter/internal/metrics"
"github.com/longpeng/configcenter/internal/outbox"
memoryruntime "github.com/longpeng/configcenter/internal/runtime/memory"
memory_store "github.com/longpeng/configcenter/internal/store/memory"
"github.com/longpeng/configcenter/internal/watch"
)
func TestCRUDPublishAndRuntimeRead(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
repository := memory_store.New(false)
runtimeStore := memoryruntime.New()
defer runtimeStore.Close() //nolint:errcheck
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
hub := watch.New(ctx, runtimeStore)
authorizer, err := auth.New(repository, false, "", 8*time.Hour)
if err != nil {
t.Fatal(err)
}
collector := metrics.New()
worker := outbox.New(repository, runtimeStore, time.Millisecond, 10, 3, collector, logger)
go worker.Run(ctx)
handler := httpapi.New(repository, runtimeStore, hub, authorizer, collector, nil, logger).Handler()
var app domain.Application
post(t, handler, "/v1/applications", map[string]any{"code": "orders", "name": "Orders"}, http.StatusCreated, &app)
var env domain.Environment
post(t, handler, "/v1/environments", map[string]any{"code": "PROD", "name": "Production", "order": 1}, http.StatusCreated, &env)
var namespace domain.Namespace
post(t, handler, "/v1/namespaces", map[string]any{"appId": app.ID, "name": "application", "type": "properties"}, http.StatusCreated, &namespace)
var config domain.ConfigItem
post(t, handler, "/v1/config-items", map[string]any{
"appId": app.ID, "nsId": namespace.ID, "envId": env.ID, "key": "server.port", "value": "8080",
}, http.StatusCreated, &config)
var release domain.Release
post(t, handler, "/v1/publish", map[string]any{
"appId": app.ID, "nsId": namespace.ID, "envId": env.ID, "comment": "first release",
}, http.StatusAccepted, &release)
if release.Status != "pending" || release.Version != 1 {
t.Fatalf("unexpected accepted release: %#v", release)
}
deadline := time.Now().Add(time.Second)
for time.Now().Before(deadline) {
get(t, handler, fmt.Sprintf("/v1/releases/%d", release.ID), http.StatusOK, &release)
if release.Status == "applied" {
break
}
time.Sleep(5 * time.Millisecond)
}
if release.Status != "applied" || release.EtcdRevision == nil {
t.Fatalf("outbox was not applied: %#v", release)
}
var current domain.RuntimeConfig
get(t, handler, "/v1/config?env=PROD&app=orders&namespace=application", http.StatusOK, &current)
if current.Items["server.port"] != "8080" || current.ReleaseVersion != 1 {
t.Fatalf("unexpected runtime config: %#v", current)
}
var grayRule domain.GrayRule
post(t, handler, "/v1/gray-rules", map[string]any{
"appId": app.ID, "nsId": namespace.ID, "envId": env.ID,
"ruleType": "instance", "ruleValue": map[string]any{"instances": []string{"orders-7"}},
"overrides": map[string]string{"server.port": "9080", "feature.gray": "on"}, "enabled": true, "priority": 10,
}, http.StatusCreated, &grayRule)
get(t, handler, "/v1/config?env=PROD&app=orders&namespace=application&instance=orders-7", http.StatusOK, &current)
if current.Items["server.port"] != "9080" || current.Items["feature.gray"] != "on" || len(current.GrayRuleIDs) != 1 || current.GrayRuleIDs[0] != grayRule.ID {
t.Fatalf("gray overrides were not applied: %#v", current)
}
}
func TestAuthenticationRBACAndMetrics(t *testing.T) {
ctx := context.Background()
repository := memory_store.New(false)
runtimeStore := memoryruntime.New()
defer runtimeStore.Close() //nolint:errcheck
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
authorizer, err := auth.New(repository, true, "0123456789abcdef0123456789abcdef", 90*time.Minute)
if err != nil {
t.Fatal(err)
}
if err := authorizer.Bootstrap(ctx, "root", "a-strong-password", "Root Admin"); err != nil {
t.Fatal(err)
}
collector := metrics.New()
handler := httpapi.New(repository, runtimeStore, watch.New(ctx, runtimeStore), authorizer, collector, nil, logger).Handler()
request := httptest.NewRequest(http.MethodGet, "/v1/applications", nil)
doStatus(t, handler, request, http.StatusUnauthorized)
var adminLogin struct {
Token string `json:"token"`
ExpiresIn int64 `json:"expiresIn"`
}
requestJSON(t, handler, http.MethodPost, "/v1/auth/login", "", map[string]any{"username": "root", "password": "a-strong-password"}, http.StatusOK, &adminLogin)
if adminLogin.Token == "" || adminLogin.ExpiresIn != 5400 {
t.Fatalf("unexpected login response: %#v", adminLogin)
}
var application domain.Application
requestJSON(t, handler, http.MethodPost, "/v1/applications", adminLogin.Token, map[string]any{"code": "secure-app", "name": "Secure App"}, http.StatusCreated, &application)
var viewer domain.User
requestJSON(t, handler, http.MethodPost, "/v1/users", adminLogin.Token, map[string]any{
"username": "reader", "displayName": "Read Only", "password": "reader-password-123",
}, http.StatusCreated, &viewer)
requestJSON(t, handler, http.MethodPut, fmt.Sprintf("/v1/users/%d/roles/%d", viewer.ID, application.ID), adminLogin.Token, map[string]any{"role": auth.RoleViewer}, http.StatusOK, &domain.UserAppRole{})
var audits []domain.AuditLog
requestJSON(t, handler, http.MethodGet, "/v1/audit-logs?limit=20", adminLogin.Token, nil, http.StatusOK, &audits)
foundActor := false
for _, audit := range audits {
if audit.Actor == "root" && audit.TargetType == "app" && audit.Action == "create" {
foundActor = true
break
}
}
if !foundActor {
t.Fatalf("JWT actor was not propagated to audit log: %#v", audits)
}
var viewerLogin struct {
Token string `json:"token"`
}
requestJSON(t, handler, http.MethodPost, "/v1/auth/login", "", map[string]any{"username": "reader", "password": "reader-password-123"}, http.StatusOK, &viewerLogin)
var applications []domain.Application
requestJSON(t, handler, http.MethodGet, "/v1/applications", viewerLogin.Token, nil, http.StatusOK, &applications)
if len(applications) != 1 || applications[0].ID != application.ID {
t.Fatalf("viewer application filtering failed: %#v", applications)
}
requestJSON(t, handler, http.MethodPost, "/v1/namespaces", viewerLogin.Token, map[string]any{"appId": application.ID, "name": "application", "type": "properties"}, http.StatusForbidden, nil)
doStatus(t, handler, httptest.NewRequest(http.MethodGet, "/health/live", nil), http.StatusOK)
recorder := httptest.NewRecorder()
handler.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/metrics", nil))
if recorder.Code != http.StatusOK || !strings.Contains(recorder.Body.String(), `configcenter_http_requests_total{method="GET",route="GET /health/live",status="200"}`) || !strings.Contains(recorder.Body.String(), "configcenter_outbox_pending 0") {
t.Fatalf("unexpected metrics response: status=%d body=%s", recorder.Code, recorder.Body.String())
}
}
func post(t *testing.T, handler http.Handler, path string, input any, wantStatus int, output any) {
t.Helper()
payload, err := json.Marshal(input)
if err != nil {
t.Fatal(err)
}
request := httptest.NewRequest(http.MethodPost, path, bytes.NewReader(payload))
request.Header.Set("Content-Type", "application/json")
request.Header.Set("X-User", "tester")
do(t, handler, request, wantStatus, output)
}
func get(t *testing.T, handler http.Handler, path string, wantStatus int, output any) {
t.Helper()
request := httptest.NewRequest(http.MethodGet, path, nil)
do(t, handler, request, wantStatus, output)
}
func do(t *testing.T, handler http.Handler, request *http.Request, wantStatus int, output any) {
t.Helper()
recorder := httptest.NewRecorder()
handler.ServeHTTP(recorder, request)
response := recorder.Result()
defer response.Body.Close()
body, err := io.ReadAll(response.Body)
if err != nil {
t.Fatal(err)
}
if response.StatusCode != wantStatus {
t.Fatalf("%s %s: got status %d, want %d; body=%s", request.Method, request.URL, response.StatusCode, wantStatus, body)
}
var envelope struct {
Data json.RawMessage `json:"data"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
t.Fatalf("decode response envelope: %v; body=%s", err, body)
}
if err := json.Unmarshal(envelope.Data, output); err != nil {
t.Fatalf("decode response data: %v; body=%s", err, body)
}
}
func requestJSON(t *testing.T, handler http.Handler, method, path, token string, input any, wantStatus int, output any) {
t.Helper()
var body io.Reader
if input != nil {
payload, err := json.Marshal(input)
if err != nil {
t.Fatal(err)
}
body = bytes.NewReader(payload)
}
request := httptest.NewRequest(method, path, body)
if input != nil {
request.Header.Set("Content-Type", "application/json")
}
if token != "" {
request.Header.Set("Authorization", "Bearer "+token)
}
if output == nil {
doStatus(t, handler, request, wantStatus)
return
}
do(t, handler, request, wantStatus, output)
}
func doStatus(t *testing.T, handler http.Handler, request *http.Request, wantStatus int) {
t.Helper()
recorder := httptest.NewRecorder()
handler.ServeHTTP(recorder, request)
if recorder.Code != wantStatus {
t.Fatalf("%s %s: got status %d, want %d; body=%s", request.Method, request.URL, recorder.Code, wantStatus, recorder.Body.String())
}
}

224
internal/auth/auth.go Normal file
View File

@@ -0,0 +1,224 @@
package auth
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"time"
"golang.org/x/crypto/bcrypt"
"github.com/longpeng/configcenter/internal/domain"
"github.com/longpeng/configcenter/internal/store"
)
var (
ErrUnauthorized = errors.New("authentication required")
ErrForbidden = errors.New("permission denied")
)
const (
RoleViewer = "viewer"
RoleAppOwner = "app-owner"
RoleAdmin = "admin"
)
type Manager struct {
enabled bool
secret []byte
ttl time.Duration
store store.Store
now func() time.Time
}
type tokenClaims struct {
Issuer string `json:"iss"`
Subject string `json:"sub"`
UserID int64 `json:"uid"`
DisplayName string `json:"name"`
Admin bool `json:"admin"`
IssuedAt int64 `json:"iat"`
ExpiresAt int64 `json:"exp"`
}
type principalKey struct{}
func New(repository store.Store, enabled bool, secret string, ttl time.Duration) (*Manager, error) {
if enabled && len(secret) < 32 {
return nil, errors.New("JWT_SECRET must contain at least 32 characters when authentication is enabled")
}
if ttl <= 0 {
return nil, errors.New("JWT_TTL must be positive")
}
return &Manager{enabled: enabled, secret: []byte(secret), ttl: ttl, store: repository, now: time.Now}, nil
}
func (m *Manager) Enabled() bool { return m.enabled }
func (m *Manager) TTLSeconds() int64 { return int64(m.ttl.Seconds()) }
func (m *Manager) Bootstrap(ctx context.Context, username, password, displayName string) error {
if !m.enabled {
return nil
}
if strings.TrimSpace(username) == "" || len(password) < 12 {
return errors.New("bootstrap admin username is required and password must contain at least 12 characters")
}
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return fmt.Errorf("hash bootstrap password: %w", err)
}
return m.store.EnsureBootstrapAdmin(ctx, normalizeUsername(username), string(hash), strings.TrimSpace(displayName))
}
func (m *Manager) Login(ctx context.Context, username, password string) (string, domain.Principal, error) {
if !m.enabled {
principal := domain.Principal{Username: "admin", DisplayName: "Development Admin", IsAdmin: true}
token, err := m.issue(principal)
return token, principal, err
}
credential, err := m.store.FindUserByUsername(ctx, normalizeUsername(username))
if err != nil || credential.Disabled || bcrypt.CompareHashAndPassword([]byte(credential.PasswordHash), []byte(password)) != nil {
return "", domain.Principal{}, ErrUnauthorized
}
principal := domain.Principal{UserID: credential.ID, Username: credential.Username, DisplayName: credential.DisplayName, IsAdmin: credential.IsAdmin}
token, err := m.issue(principal)
return token, principal, err
}
func (m *Manager) CreateUser(ctx context.Context, input domain.User, password string) (domain.User, error) {
if len(password) < 12 {
return domain.User{}, errors.New("password must contain at least 12 characters")
}
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return domain.User{}, err
}
input.Username = normalizeUsername(input.Username)
return m.store.CreateUser(ctx, input, string(hash))
}
func (m *Manager) AuthenticateRequest(r *http.Request) (domain.Principal, error) {
if !m.enabled {
return domain.Principal{Username: "admin", DisplayName: "Development Admin", IsAdmin: true}, nil
}
header := strings.TrimSpace(r.Header.Get("Authorization"))
if !strings.HasPrefix(header, "Bearer ") {
return domain.Principal{}, ErrUnauthorized
}
return m.parse(strings.TrimSpace(strings.TrimPrefix(header, "Bearer ")))
}
func WithPrincipal(ctx context.Context, principal domain.Principal) context.Context {
return context.WithValue(ctx, principalKey{}, principal)
}
func Principal(ctx context.Context) (domain.Principal, bool) {
principal, ok := ctx.Value(principalKey{}).(domain.Principal)
return principal, ok
}
func (m *Manager) RequireAdmin(ctx context.Context) error {
principal, ok := Principal(ctx)
if !ok {
return ErrUnauthorized
}
if !principal.IsAdmin {
return ErrForbidden
}
return nil
}
func (m *Manager) RequireAppRole(ctx context.Context, appID int64, minimum string) error {
principal, ok := Principal(ctx)
if !ok {
return ErrUnauthorized
}
if principal.IsAdmin {
return nil
}
role, err := m.store.GetUserAppRole(ctx, principal.UserID, appID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
return ErrForbidden
}
return err
}
if roleRank(role) < roleRank(minimum) {
return ErrForbidden
}
return nil
}
func (m *Manager) CanAccessApp(ctx context.Context, appID int64, minimum string) bool {
return m.RequireAppRole(ctx, appID, minimum) == nil
}
func (m *Manager) issue(principal domain.Principal) (string, error) {
now := m.now().UTC()
claims := tokenClaims{Issuer: "configcenter", Subject: principal.Username, UserID: principal.UserID, DisplayName: principal.DisplayName, Admin: principal.IsAdmin, IssuedAt: now.Unix(), ExpiresAt: now.Add(m.ttl).Unix()}
header, _ := json.Marshal(map[string]string{"alg": "HS256", "typ": "JWT"})
payload, err := json.Marshal(claims)
if err != nil {
return "", err
}
unsigned := encode(header) + "." + encode(payload)
signature := m.sign(unsigned)
return unsigned + "." + encode(signature), nil
}
func (m *Manager) parse(token string) (domain.Principal, error) {
parts := strings.Split(token, ".")
if len(parts) != 3 {
return domain.Principal{}, ErrUnauthorized
}
unsigned := parts[0] + "." + parts[1]
signature, err := base64.RawURLEncoding.DecodeString(parts[2])
if err != nil || !hmac.Equal(signature, m.sign(unsigned)) {
return domain.Principal{}, ErrUnauthorized
}
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return domain.Principal{}, ErrUnauthorized
}
var claims tokenClaims
if err := json.Unmarshal(payload, &claims); err != nil || claims.Issuer != "configcenter" || claims.Subject == "" {
return domain.Principal{}, ErrUnauthorized
}
now := m.now().Unix()
if claims.ExpiresAt <= now || claims.IssuedAt > now+60 {
return domain.Principal{}, ErrUnauthorized
}
return domain.Principal{UserID: claims.UserID, Username: claims.Subject, DisplayName: claims.DisplayName, IsAdmin: claims.Admin}, nil
}
func (m *Manager) sign(input string) []byte {
mac := hmac.New(sha256.New, m.secret)
_, _ = mac.Write([]byte(input))
return mac.Sum(nil)
}
func encode(value []byte) string { return base64.RawURLEncoding.EncodeToString(value) }
func normalizeUsername(value string) string { return strings.ToLower(strings.TrimSpace(value)) }
func roleRank(role string) int {
switch role {
case RoleViewer:
return 1
case RoleAppOwner:
return 2
case RoleAdmin:
return 3
default:
return 0
}
}
func ValidRole(role string) bool { return roleRank(role) > 0 }

View File

@@ -0,0 +1,72 @@
package auth
import (
"context"
"errors"
"net/http/httptest"
"testing"
"time"
"github.com/longpeng/configcenter/internal/domain"
memory_store "github.com/longpeng/configcenter/internal/store/memory"
)
func TestLoginTokenExpiryAndApplicationRoles(t *testing.T) {
ctx := context.Background()
repository := memory_store.New(false)
manager, err := New(repository, true, "0123456789abcdef0123456789abcdef", time.Hour)
if err != nil {
t.Fatal(err)
}
clock := time.Date(2026, 8, 30, 2, 0, 0, 0, time.UTC)
manager.now = func() time.Time { return clock }
if err := manager.Bootstrap(ctx, "root", "a-strong-password", "Root"); err != nil {
t.Fatal(err)
}
token, principal, err := manager.Login(ctx, "root", "a-strong-password")
if err != nil || !principal.IsAdmin {
t.Fatalf("bootstrap login failed: principal=%#v err=%v", principal, err)
}
request := httptest.NewRequest("GET", "/v1/applications", nil)
request.Header.Set("Authorization", "Bearer "+token)
parsed, err := manager.AuthenticateRequest(request)
if err != nil || parsed.Username != "root" {
t.Fatalf("token validation failed: principal=%#v err=%v", parsed, err)
}
app, err := repository.CreateApplication(ctx, domain.Application{Code: "orders", Name: "Orders"})
if err != nil {
t.Fatal(err)
}
viewer, err := manager.CreateUser(ctx, domain.User{Username: "Reader", DisplayName: "Reader"}, "another-strong-password")
if err != nil {
t.Fatal(err)
}
if viewer.Username != "reader" {
t.Fatalf("username was not normalized: %q", viewer.Username)
}
if err := repository.SetUserAppRole(ctx, domain.UserAppRole{UserID: viewer.ID, AppID: app.ID, Role: RoleViewer}); err != nil {
t.Fatal(err)
}
viewerToken, viewerPrincipal, err := manager.Login(ctx, "READER", "another-strong-password")
if err != nil || viewerToken == "" {
t.Fatal(err)
}
viewerContext := WithPrincipal(ctx, viewerPrincipal)
if err := manager.RequireAppRole(viewerContext, app.ID, RoleViewer); err != nil {
t.Fatalf("viewer should read application: %v", err)
}
if err := manager.RequireAppRole(viewerContext, app.ID, RoleAppOwner); !errors.Is(err, ErrForbidden) {
t.Fatalf("viewer must not edit application: %v", err)
}
clock = clock.Add(2 * time.Hour)
if _, err := manager.AuthenticateRequest(request); !errors.Is(err, ErrUnauthorized) {
t.Fatalf("expired token must be rejected: %v", err)
}
tampered := httptest.NewRequest("GET", "/", nil)
tampered.Header.Set("Authorization", "Bearer "+token+"x")
if _, err := manager.AuthenticateRequest(tampered); !errors.Is(err, ErrUnauthorized) {
t.Fatalf("tampered token must be rejected: %v", err)
}
}

103
internal/config/config.go Normal file
View File

@@ -0,0 +1,103 @@
package config
import (
"os"
"strconv"
"strings"
"time"
)
type Config struct {
HTTPAddr string
DatabaseURL string
EtcdEndpoints []string
EtcdDialTimeout time.Duration
OutboxInterval time.Duration
OutboxBatchSize int
OutboxMaxRetry int
ShutdownTimeout time.Duration
AllowedOrigins []string
AuthEnabled bool
JWTSecret string
JWTTTL time.Duration
BootstrapAdminUsername string
BootstrapAdminPassword string
BootstrapAdminDisplayName string
}
func Load() Config {
return Config{
HTTPAddr: env("HTTP_ADDR", ":8080"),
DatabaseURL: strings.TrimSpace(os.Getenv("DATABASE_URL")),
EtcdEndpoints: csv(os.Getenv("ETCD_ENDPOINTS")),
EtcdDialTimeout: duration("ETCD_DIAL_TIMEOUT", 5*time.Second),
OutboxInterval: duration("OUTBOX_INTERVAL", 500*time.Millisecond),
OutboxBatchSize: integer("OUTBOX_BATCH_SIZE", 50),
OutboxMaxRetry: integer("OUTBOX_MAX_RETRY", 12),
ShutdownTimeout: duration("SHUTDOWN_TIMEOUT", 10*time.Second),
AllowedOrigins: csvDefault(os.Getenv("CORS_ALLOWED_ORIGINS"), []string{"http://localhost:5173"}),
AuthEnabled: boolean("AUTH_ENABLED", false),
JWTSecret: strings.TrimSpace(os.Getenv("JWT_SECRET")),
JWTTTL: duration("JWT_TTL", 8*time.Hour),
BootstrapAdminUsername: env("BOOTSTRAP_ADMIN_USERNAME", "admin"),
BootstrapAdminPassword: os.Getenv("BOOTSTRAP_ADMIN_PASSWORD"),
BootstrapAdminDisplayName: env("BOOTSTRAP_ADMIN_DISPLAY_NAME", "Administrator"),
}
}
func env(key, fallback string) string {
if value := strings.TrimSpace(os.Getenv(key)); value != "" {
return value
}
return fallback
}
func duration(key string, fallback time.Duration) time.Duration {
value := strings.TrimSpace(os.Getenv(key))
if value == "" {
return fallback
}
parsed, err := time.ParseDuration(value)
if err != nil {
return fallback
}
return parsed
}
func integer(key string, fallback int) int {
parsed, err := strconv.Atoi(strings.TrimSpace(os.Getenv(key)))
if err != nil || parsed <= 0 {
return fallback
}
return parsed
}
func boolean(key string, fallback bool) bool {
value := strings.TrimSpace(os.Getenv(key))
if value == "" {
return fallback
}
parsed, err := strconv.ParseBool(value)
if err != nil {
return fallback
}
return parsed
}
func csv(value string) []string {
parts := strings.Split(value, ",")
result := make([]string, 0, len(parts))
for _, part := range parts {
if trimmed := strings.TrimSpace(part); trimmed != "" {
result = append(result, trimmed)
}
}
return result
}
func csvDefault(value string, fallback []string) []string {
if result := csv(value); len(result) > 0 {
return result
}
return fallback
}

168
internal/domain/models.go Normal file
View File

@@ -0,0 +1,168 @@
package domain
import (
"encoding/json"
"time"
)
type Application struct {
ID int64 `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
Department string `json:"department"`
Owner string `json:"owner"`
Desc string `json:"desc"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
UpdatedBy string `json:"-"`
}
type Environment struct {
ID int64 `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
Order int `json:"order"`
Desc string `json:"desc"`
UpdatedBy string `json:"-"`
}
type Namespace struct {
ID int64 `json:"id"`
AppID int64 `json:"appId"`
Name string `json:"name"`
Type string `json:"type"`
Desc string `json:"desc"`
UpdatedBy string `json:"-"`
}
type ConfigItem struct {
ID int64 `json:"id"`
AppID int64 `json:"appId"`
NamespaceID int64 `json:"nsId"`
EnvironmentID int64 `json:"envId"`
Key string `json:"key"`
Value string `json:"value"`
ReleasedValue *string `json:"releasedValue"`
PendingDelete bool `json:"pendingDelete"`
Comment string `json:"comment"`
UpdatedBy string `json:"updatedBy"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (item ConfigItem) Pending() bool {
return item.PendingDelete || item.ReleasedValue == nil || *item.ReleasedValue != item.Value
}
type Release struct {
ID int64 `json:"id"`
AppID int64 `json:"appId"`
NamespaceID int64 `json:"nsId"`
EnvironmentID int64 `json:"envId"`
Version int `json:"version"`
Snapshot map[string]string `json:"snapshot"`
Added int `json:"added"`
Modified int `json:"modified"`
Removed int `json:"removed"`
Comment string `json:"comment"`
Operator string `json:"operator"`
EtcdRevision *int64 `json:"etcdRevision"`
Status string `json:"status"`
Time time.Time `json:"time"`
}
type PublishRequest struct {
EnvironmentID int64 `json:"envId"`
AppID int64 `json:"appId"`
NamespaceID int64 `json:"nsId"`
Comment string `json:"comment"`
Operator string `json:"operator"`
}
type RollbackRequest struct {
EnvironmentID int64 `json:"envId"`
AppID int64 `json:"appId"`
NamespaceID int64 `json:"nsId"`
TargetVersion int `json:"targetVersion"`
Operator string `json:"operator"`
}
type RuntimeConfig struct {
Items map[string]string `json:"items"`
Revision int64 `json:"revision"`
ReleaseVersion int `json:"releaseVersion"`
GrayRuleIDs []int64 `json:"grayRuleIds,omitempty"`
}
type OutboxEntry struct {
ID int64
ReleaseID int64
EtcdKey string
Payload json.RawMessage
RetryCount int
}
type AuditLog struct {
ID int64 `json:"id"`
Actor string `json:"actor"`
Action string `json:"action"`
TargetType string `json:"targetType"`
TargetID *int64 `json:"targetId"`
Detail json.RawMessage `json:"detail"`
CreatedAt time.Time `json:"createdAt"`
}
type ConfigEvent struct {
Type string `json:"type"`
Items map[string]string `json:"items"`
Revision int64 `json:"revision"`
GrayRuleIDs []int64 `json:"grayRuleIds,omitempty"`
}
type User struct {
ID int64 `json:"id"`
Username string `json:"username"`
DisplayName string `json:"displayName"`
IsAdmin bool `json:"isAdmin"`
Disabled bool `json:"disabled"`
CreatedAt time.Time `json:"createdAt"`
UpdatedBy string `json:"-"`
}
type UserCredential struct {
User
PasswordHash string `json:"-"`
}
type UserAppRole struct {
UserID int64 `json:"userId"`
AppID int64 `json:"appId"`
Role string `json:"role"`
UpdatedBy string `json:"-"`
}
type Principal struct {
UserID int64 `json:"userId"`
Username string `json:"username"`
DisplayName string `json:"displayName"`
IsAdmin bool `json:"isAdmin"`
}
type GrayRule struct {
ID int64 `json:"id"`
AppID int64 `json:"appId"`
NamespaceID int64 `json:"nsId"`
EnvironmentID int64 `json:"envId"`
RuleType string `json:"ruleType"`
RuleValue json.RawMessage `json:"ruleValue"`
Overrides map[string]string `json:"overrides"`
Enabled bool `json:"enabled"`
Priority int `json:"priority"`
Desc string `json:"desc"`
CreatedAt time.Time `json:"createdAt"`
}
type OutboxStats struct {
Pending int64 `json:"pending"`
Processing int64 `json:"processing"`
Failed int64 `json:"failed"`
}

159
internal/gray/matcher.go Normal file
View File

@@ -0,0 +1,159 @@
package gray
import (
"crypto/sha256"
"encoding/binary"
"encoding/json"
"fmt"
"net"
"sort"
"strings"
"github.com/longpeng/configcenter/internal/domain"
)
const (
RuleIP = "ip"
RuleInstance = "instance"
RulePercentage = "percentage"
)
type Target struct {
IP string
Instance string
}
type ipRule struct {
IPs []string `json:"ips"`
}
type instanceRule struct {
Instances []string `json:"instances"`
}
type percentageRule struct {
Percentage int `json:"percentage"`
Salt string `json:"salt"`
}
func Validate(rule domain.GrayRule) error {
if rule.NamespaceID <= 0 || rule.EnvironmentID <= 0 || len(rule.Overrides) == 0 {
return fmt.Errorf("scope and overrides are required")
}
for key := range rule.Overrides {
if strings.TrimSpace(key) == "" {
return fmt.Errorf("override key must not be empty")
}
}
switch rule.RuleType {
case RuleIP:
var value ipRule
if err := strictUnmarshal(rule.RuleValue, &value); err != nil || len(value.IPs) == 0 {
return fmt.Errorf("ip rule requires a non-empty ips array")
}
for _, item := range value.IPs {
if net.ParseIP(item) == nil {
if _, _, err := net.ParseCIDR(item); err != nil {
return fmt.Errorf("invalid IP or CIDR %q", item)
}
}
}
case RuleInstance:
var value instanceRule
if err := strictUnmarshal(rule.RuleValue, &value); err != nil || len(value.Instances) == 0 {
return fmt.Errorf("instance rule requires a non-empty instances array")
}
case RulePercentage:
var value percentageRule
if err := strictUnmarshal(rule.RuleValue, &value); err != nil || value.Percentage < 0 || value.Percentage > 100 {
return fmt.Errorf("percentage rule requires percentage between 0 and 100")
}
default:
return fmt.Errorf("unsupported gray rule type %q", rule.RuleType)
}
return nil
}
func Apply(base map[string]string, rules []domain.GrayRule, target Target) (map[string]string, []int64) {
result := clone(base)
sorted := append([]domain.GrayRule(nil), rules...)
sort.SliceStable(sorted, func(i, j int) bool {
if sorted[i].Priority == sorted[j].Priority {
return sorted[i].ID < sorted[j].ID
}
return sorted[i].Priority < sorted[j].Priority
})
matched := make([]int64, 0)
for _, rule := range sorted {
if !rule.Enabled || !matches(rule, target) {
continue
}
for key, value := range rule.Overrides {
result[key] = value
}
matched = append(matched, rule.ID)
}
return result, matched
}
func matches(rule domain.GrayRule, target Target) bool {
switch rule.RuleType {
case RuleIP:
parsed := net.ParseIP(target.IP)
if parsed == nil {
return false
}
var value ipRule
if json.Unmarshal(rule.RuleValue, &value) != nil {
return false
}
for _, item := range value.IPs {
if ip := net.ParseIP(item); ip != nil && ip.Equal(parsed) {
return true
}
if _, network, err := net.ParseCIDR(item); err == nil && network.Contains(parsed) {
return true
}
}
case RuleInstance:
var value instanceRule
if target.Instance == "" || json.Unmarshal(rule.RuleValue, &value) != nil {
return false
}
for _, instance := range value.Instances {
if instance == target.Instance {
return true
}
}
case RulePercentage:
identity := target.Instance
if identity == "" {
identity = target.IP
}
if identity == "" {
return false
}
var value percentageRule
if json.Unmarshal(rule.RuleValue, &value) != nil {
return false
}
digest := sha256.Sum256([]byte(value.Salt + "\x00" + identity))
bucket := binary.BigEndian.Uint64(digest[:8]) % 100
return int(bucket) < value.Percentage
}
return false
}
func strictUnmarshal(payload []byte, target any) error {
decoder := json.NewDecoder(strings.NewReader(string(payload)))
decoder.DisallowUnknownFields()
return decoder.Decode(target)
}
func clone(input map[string]string) map[string]string {
result := make(map[string]string, len(input))
for key, value := range input {
result[key] = value
}
return result
}

View File

@@ -0,0 +1,53 @@
package gray
import (
"encoding/json"
"testing"
"github.com/longpeng/configcenter/internal/domain"
)
func TestApplyMatchesIPInstanceAndPriority(t *testing.T) {
rules := []domain.GrayRule{
{ID: 1, NamespaceID: 1, EnvironmentID: 1, RuleType: RuleIP, RuleValue: json.RawMessage(`{"ips":["10.0.0.0/24"]}`), Overrides: map[string]string{"color": "blue", "ip-only": "yes"}, Enabled: true, Priority: 10},
{ID: 2, NamespaceID: 1, EnvironmentID: 1, RuleType: RuleInstance, RuleValue: json.RawMessage(`{"instances":["orders-7"]}`), Overrides: map[string]string{"color": "green"}, Enabled: true, Priority: 20},
}
for _, rule := range rules {
if err := Validate(rule); err != nil {
t.Fatalf("valid rule rejected: %v", err)
}
}
items, matched := Apply(map[string]string{"color": "red", "stable": "value"}, rules, Target{IP: "10.0.0.42", Instance: "orders-7"})
if items["color"] != "green" || items["ip-only"] != "yes" || items["stable"] != "value" {
t.Fatalf("unexpected overrides: %#v", items)
}
if len(matched) != 2 || matched[0] != 1 || matched[1] != 2 {
t.Fatalf("unexpected matched rules: %#v", matched)
}
}
func TestPercentageRuleIsDeterministic(t *testing.T) {
rule := domain.GrayRule{ID: 3, NamespaceID: 1, EnvironmentID: 1, RuleType: RulePercentage, RuleValue: json.RawMessage(`{"percentage":50,"salt":"release-a"}`), Overrides: map[string]string{"feature": "on"}, Enabled: true}
if err := Validate(rule); err != nil {
t.Fatal(err)
}
firstItems, firstMatched := Apply(map[string]string{"feature": "off"}, []domain.GrayRule{rule}, Target{Instance: "orders-42"})
for i := 0; i < 20; i++ {
items, matched := Apply(map[string]string{"feature": "off"}, []domain.GrayRule{rule}, Target{Instance: "orders-42"})
if items["feature"] != firstItems["feature"] || len(matched) != len(firstMatched) {
t.Fatalf("percentage assignment changed between calls: first=%#v/%#v current=%#v/%#v", firstItems, firstMatched, items, matched)
}
}
}
func TestValidateRejectsInvalidCIDRAndPercentage(t *testing.T) {
cases := []domain.GrayRule{
{NamespaceID: 1, EnvironmentID: 1, RuleType: RuleIP, RuleValue: json.RawMessage(`{"ips":["not-an-ip"]}`), Overrides: map[string]string{"x": "y"}},
{NamespaceID: 1, EnvironmentID: 1, RuleType: RulePercentage, RuleValue: json.RawMessage(`{"percentage":101}`), Overrides: map[string]string{"x": "y"}},
}
for _, rule := range cases {
if err := Validate(rule); err == nil {
t.Fatalf("invalid rule accepted: %#v", rule)
}
}
}

130
internal/metrics/metrics.go Normal file
View File

@@ -0,0 +1,130 @@
package metrics
import (
"fmt"
"net/http"
"sort"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/longpeng/configcenter/internal/store"
)
type Collector struct {
mu sync.Mutex
http map[httpKey]httpValue
publishAccepted atomic.Uint64
outboxApplied atomic.Uint64
outboxFailed atomic.Uint64
runtimeReads atomic.Uint64
grayMatches atomic.Uint64
watchers atomic.Int64
}
type httpKey struct {
method string
route string
status int
}
type httpValue struct {
count uint64
durationSum float64
}
func New() *Collector { return &Collector{http: make(map[httpKey]httpValue)} }
func (c *Collector) ObserveHTTP(method, route string, status int, duration time.Duration) {
if route == "" {
route = "unmatched"
}
key := httpKey{method: method, route: route, status: status}
c.mu.Lock()
value := c.http[key]
value.count++
value.durationSum += duration.Seconds()
c.http[key] = value
c.mu.Unlock()
}
func (c *Collector) PublishAccepted() { c.publishAccepted.Add(1) }
func (c *Collector) OutboxApplied() { c.outboxApplied.Add(1) }
func (c *Collector) OutboxFailed() { c.outboxFailed.Add(1) }
func (c *Collector) RuntimeRead() { c.runtimeReads.Add(1) }
func (c *Collector) GrayMatched(count int) {
if count > 0 {
c.grayMatches.Add(uint64(count))
}
}
func (c *Collector) WatcherAdded() { c.watchers.Add(1) }
func (c *Collector) WatcherRemoved() { c.watchers.Add(-1) }
func (c *Collector) Handler(repository store.Store) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain; version=0.0.4; charset=utf-8")
stats, err := repository.OutboxStats(r.Context())
if err != nil {
http.Error(w, "collect outbox metrics", http.StatusServiceUnavailable)
return
}
var output strings.Builder
writeHelp(&output, "configcenter_http_requests_total", "HTTP requests by method, route and status", "counter")
writeHelp(&output, "configcenter_http_request_duration_seconds_sum", "Cumulative HTTP request duration", "counter")
c.mu.Lock()
keys := make([]httpKey, 0, len(c.http))
for key := range c.http {
keys = append(keys, key)
}
sort.Slice(keys, func(i, j int) bool {
if keys[i].route != keys[j].route {
return keys[i].route < keys[j].route
}
if keys[i].method != keys[j].method {
return keys[i].method < keys[j].method
}
return keys[i].status < keys[j].status
})
for _, key := range keys {
value := c.http[key]
labels := fmt.Sprintf(`method="%s",route="%s",status="%d"`, escape(key.method), escape(key.route), key.status)
fmt.Fprintf(&output, "configcenter_http_requests_total{%s} %d\n", labels, value.count)
fmt.Fprintf(&output, "configcenter_http_request_duration_seconds_sum{%s} %s\n", labels, strconv.FormatFloat(value.durationSum, 'f', 6, 64))
}
c.mu.Unlock()
writeCounter(&output, "configcenter_publish_accepted_total", "Accepted configuration releases", c.publishAccepted.Load())
writeCounter(&output, "configcenter_outbox_applied_total", "Successfully applied outbox entries", c.outboxApplied.Load())
writeCounter(&output, "configcenter_outbox_failed_attempts_total", "Failed outbox apply attempts", c.outboxFailed.Load())
writeCounter(&output, "configcenter_runtime_reads_total", "Runtime configuration reads", c.runtimeReads.Load())
writeCounter(&output, "configcenter_gray_rule_matches_total", "Gray rules matched during runtime reads and watches", c.grayMatches.Load())
writeGauge(&output, "configcenter_watch_subscribers", "Current SSE watch subscribers", c.watchers.Load())
writeGauge(&output, "configcenter_outbox_pending", "Pending outbox entries", stats.Pending)
writeGauge(&output, "configcenter_outbox_processing", "Processing outbox entries", stats.Processing)
writeGauge(&output, "configcenter_outbox_failed", "Terminally failed outbox entries", stats.Failed)
_, _ = w.Write([]byte(output.String()))
}
}
func writeHelp(output *strings.Builder, name, help, metricType string) {
fmt.Fprintf(output, "# HELP %s %s\n# TYPE %s %s\n", name, help, name, metricType)
}
func writeCounter(output *strings.Builder, name, help string, value uint64) {
writeHelp(output, name, help, "counter")
fmt.Fprintf(output, "%s %d\n", name, value)
}
func writeGauge(output *strings.Builder, name, help string, value int64) {
writeHelp(output, name, help, "gauge")
fmt.Fprintf(output, "%s %d\n", name, value)
}
func escape(value string) string {
value = strings.ReplaceAll(value, `\`, `\\`)
value = strings.ReplaceAll(value, `"`, `\"`)
return strings.ReplaceAll(value, "\n", `\n`)
}

67
internal/outbox/worker.go Normal file
View File

@@ -0,0 +1,67 @@
package outbox
import (
"context"
"log/slog"
"time"
metricspkg "github.com/longpeng/configcenter/internal/metrics"
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
"github.com/longpeng/configcenter/internal/store"
)
type Worker struct {
store store.Store
runtime runtimepkg.Store
interval time.Duration
batch int
maxRetry int
logger *slog.Logger
metrics *metricspkg.Collector
}
func New(repository store.Store, runtimeStore runtimepkg.Store, interval time.Duration, batch, maxRetry int, metrics *metricspkg.Collector, logger *slog.Logger) *Worker {
return &Worker{store: repository, runtime: runtimeStore, interval: interval, batch: batch, maxRetry: maxRetry, metrics: metrics, logger: logger}
}
func (w *Worker) Run(ctx context.Context) {
ticker := time.NewTicker(w.interval)
defer ticker.Stop()
w.process(ctx)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
w.process(ctx)
}
}
}
func (w *Worker) process(ctx context.Context) {
entries, err := w.store.ClaimOutbox(ctx, w.batch)
if err != nil {
w.logger.Error("claim outbox", "error", err)
return
}
for _, entry := range entries {
release, err := w.store.GetRelease(ctx, entry.ReleaseID)
if err == nil {
var revision int64
revision, err = w.runtime.Put(ctx, entry.EtcdKey, entry.Payload, release)
if err == nil {
err = w.store.MarkOutboxDone(ctx, entry.ID, entry.ReleaseID, revision)
if err == nil {
w.metrics.OutboxApplied()
}
}
}
if err != nil {
w.metrics.OutboxFailed()
w.logger.Warn("apply release outbox", "outbox_id", entry.ID, "release_id", entry.ReleaseID, "retry", entry.RetryCount, "error", err)
if markErr := w.store.MarkOutboxFailed(ctx, entry.ID, entry.ReleaseID, err.Error(), w.maxRetry); markErr != nil {
w.logger.Error("mark outbox failed", "outbox_id", entry.ID, "error", markErr)
}
}
}
}

View File

@@ -0,0 +1,122 @@
package etcd
import (
"context"
"encoding/json"
"fmt"
"time"
clientv3 "go.etcd.io/etcd/client/v3"
"github.com/longpeng/configcenter/internal/domain"
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
)
type Store struct {
client *clientv3.Client
}
type metadata struct {
Version int `json:"version"`
ReleaseID int64 `json:"releaseId"`
PublishedAt time.Time `json:"publishedAt"`
PublishedBy string `json:"publishedBy"`
}
func Open(endpoints []string, dialTimeout time.Duration) (*Store, error) {
client, err := clientv3.New(clientv3.Config{Endpoints: endpoints, DialTimeout: dialTimeout})
if err != nil {
return nil, fmt.Errorf("create etcd client: %w", err)
}
return &Store{client: client}, nil
}
func (s *Store) Close() error { return s.client.Close() }
func (s *Store) Ping(ctx context.Context) error {
_, err := s.client.Get(ctx, "/configcenter/health", clientv3.WithLimit(1))
return err
}
func (s *Store) Put(ctx context.Context, key string, payload []byte, release domain.Release) (int64, error) {
meta, err := json.Marshal(metadata{Version: release.Version, ReleaseID: release.ID, PublishedAt: release.Time, PublishedBy: release.Operator})
if err != nil {
return 0, err
}
response, err := s.client.Txn(ctx).Then(
clientv3.OpPut(key, string(payload)),
clientv3.OpPut(key+"/__meta", string(meta)),
).Commit()
if err != nil {
return 0, err
}
return response.Header.Revision, nil
}
func (s *Store) Get(ctx context.Context, key string) (domain.RuntimeConfig, error) {
response, err := s.client.Txn(ctx).Then(
clientv3.OpGet(key),
clientv3.OpGet(key+"/__meta"),
).Commit()
if err != nil {
return domain.RuntimeConfig{}, err
}
result := domain.RuntimeConfig{Items: map[string]string{}, Revision: response.Header.Revision}
if len(response.Responses) > 0 {
values := response.Responses[0].GetResponseRange().Kvs
if len(values) > 0 {
if err := json.Unmarshal(values[0].Value, &result.Items); err != nil {
return domain.RuntimeConfig{}, fmt.Errorf("decode runtime config: %w", err)
}
result.Revision = values[0].ModRevision
}
}
if len(response.Responses) > 1 {
values := response.Responses[1].GetResponseRange().Kvs
if len(values) > 0 {
var meta metadata
if err := json.Unmarshal(values[0].Value, &meta); err != nil {
return domain.RuntimeConfig{}, fmt.Errorf("decode runtime metadata: %w", err)
}
result.ReleaseVersion = meta.Version
}
}
return result, nil
}
func (s *Store) Watch(ctx context.Context, key string, startRevision int64) <-chan runtimepkg.WatchResult {
result := make(chan runtimepkg.WatchResult, 16)
options := []clientv3.OpOption{}
if startRevision > 0 {
options = append(options, clientv3.WithRev(startRevision))
}
watch := s.client.Watch(ctx, key, options...)
go func() {
defer close(result)
for response := range watch {
if err := response.Err(); err != nil {
select {
case result <- runtimepkg.WatchResult{Err: err}:
case <-ctx.Done():
}
return
}
for _, event := range response.Events {
items := make(map[string]string)
if err := json.Unmarshal(event.Kv.Value, &items); err != nil {
select {
case result <- runtimepkg.WatchResult{Err: err}:
case <-ctx.Done():
}
continue
}
select {
case result <- runtimepkg.WatchResult{Event: domain.ConfigEvent{Type: "UPDATED", Items: items, Revision: event.Kv.ModRevision}}:
case <-ctx.Done():
return
}
}
}
}()
return result
}

View File

@@ -0,0 +1,120 @@
package memory
import (
"context"
"encoding/json"
"sync"
"github.com/longpeng/configcenter/internal/domain"
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
)
type value struct {
items map[string]string
revision int64
version int
}
type historyEvent struct {
key string
event domain.ConfigEvent
}
type subscriber struct {
key string
ch chan runtimepkg.WatchResult
}
type Store struct {
mu sync.RWMutex
revision int64
values map[string]value
subscribers map[int64]subscriber
nextSubID int64
history []historyEvent
}
func New() *Store {
return &Store{values: make(map[string]value), subscribers: make(map[int64]subscriber)}
}
func (s *Store) Close() error {
s.mu.Lock()
defer s.mu.Unlock()
for id, sub := range s.subscribers {
close(sub.ch)
delete(s.subscribers, id)
}
return nil
}
func (s *Store) Ping(context.Context) error { return nil }
func (s *Store) Put(_ context.Context, key string, payload []byte, release domain.Release) (int64, error) {
items := make(map[string]string)
if err := json.Unmarshal(payload, &items); err != nil {
return 0, err
}
s.mu.Lock()
s.revision++
revision := s.revision
s.values[key] = value{items: clone(items), revision: revision, version: release.Version}
event := domain.ConfigEvent{Type: "UPDATED", Items: clone(items), Revision: revision}
s.history = append(s.history, historyEvent{key: key, event: event})
if len(s.history) > 256 {
s.history = append([]historyEvent(nil), s.history[len(s.history)-256:]...)
}
for _, sub := range s.subscribers {
if sub.key != key {
continue
}
select {
case sub.ch <- runtimepkg.WatchResult{Event: event}:
default:
}
}
s.mu.Unlock()
return revision, nil
}
func (s *Store) Get(_ context.Context, key string) (domain.RuntimeConfig, error) {
s.mu.RLock()
defer s.mu.RUnlock()
current, ok := s.values[key]
if !ok {
return domain.RuntimeConfig{Items: map[string]string{}, Revision: s.revision}, nil
}
return domain.RuntimeConfig{Items: clone(current.items), Revision: current.revision, ReleaseVersion: current.version}, nil
}
func (s *Store) Watch(ctx context.Context, key string, startRevision int64) <-chan runtimepkg.WatchResult {
ch := make(chan runtimepkg.WatchResult, 16)
s.mu.Lock()
s.nextSubID++
id := s.nextSubID
s.subscribers[id] = subscriber{key: key, ch: ch}
for _, historical := range s.history {
if historical.key == key && historical.event.Revision >= startRevision && startRevision > 0 {
ch <- runtimepkg.WatchResult{Event: historical.event}
}
}
s.mu.Unlock()
go func() {
<-ctx.Done()
s.mu.Lock()
if _, ok := s.subscribers[id]; ok {
delete(s.subscribers, id)
close(ch)
}
s.mu.Unlock()
}()
return ch
}
func clone(input map[string]string) map[string]string {
result := make(map[string]string, len(input))
for key, item := range input {
result[key] = item
}
return result
}

View File

@@ -0,0 +1,20 @@
package runtime
import (
"context"
"github.com/longpeng/configcenter/internal/domain"
)
type WatchResult struct {
Event domain.ConfigEvent
Err error
}
type Store interface {
Close() error
Ping(context.Context) error
Put(context.Context, string, []byte, domain.Release) (int64, error)
Get(context.Context, string) (domain.RuntimeConfig, error)
Watch(context.Context, string, int64) <-chan WatchResult
}

View File

@@ -0,0 +1,884 @@
package memory
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"time"
"github.com/longpeng/configcenter/internal/domain"
storepkg "github.com/longpeng/configcenter/internal/store"
)
type Store struct {
mu sync.RWMutex
nextID int64
applications map[int64]domain.Application
environments map[int64]domain.Environment
namespaces map[int64]domain.Namespace
configs map[int64]domain.ConfigItem
releases map[int64]domain.Release
outbox map[int64]outboxRecord
audits []domain.AuditLog
users map[int64]domain.UserCredential
userRoles map[[2]int64]string
grayRules map[int64]domain.GrayRule
}
type outboxRecord struct {
domain.OutboxEntry
Status string
NextAttempt time.Time
}
func New(withSeed bool) *Store {
s := &Store{
nextID: 100,
applications: make(map[int64]domain.Application),
environments: make(map[int64]domain.Environment),
namespaces: make(map[int64]domain.Namespace),
configs: make(map[int64]domain.ConfigItem),
releases: make(map[int64]domain.Release),
outbox: make(map[int64]outboxRecord),
users: make(map[int64]domain.UserCredential),
userRoles: make(map[[2]int64]string),
grayRules: make(map[int64]domain.GrayRule),
}
if withSeed {
s.seed()
}
return s
}
func (s *Store) Close() {}
func (s *Store) Ping(context.Context) error { return nil }
func (s *Store) id() int64 {
s.nextID++
return s.nextID
}
func (s *Store) seed() {
now := time.Now().UTC()
for _, env := range []domain.Environment{
{ID: 1, Code: "DEV", Name: "开发环境", Order: 1, Desc: "开发联调使用"},
{ID: 2, Code: "TEST", Name: "测试环境", Order: 2, Desc: "QA 测试验证"},
{ID: 3, Code: "STAGING", Name: "预发环境", Order: 3, Desc: "上线前灰度验证"},
{ID: 4, Code: "PROD", Name: "生产环境", Order: 4, Desc: "线上正式环境"},
} {
s.environments[env.ID] = env
}
app := domain.Application{ID: 10, Code: "demo-service", Name: "示例服务", Department: "平台架构部", Owner: "admin", Desc: "开箱即用的演示应用", CreatedAt: now, UpdatedAt: now}
ns := domain.Namespace{ID: 20, AppID: app.ID, Name: "application", Type: "properties", Desc: "默认应用配置"}
s.applications[app.ID] = app
s.namespaces[ns.ID] = ns
port := "8080"
level := "INFO"
s.configs[30] = domain.ConfigItem{ID: 30, AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: 1, Key: "server.port", Value: port, ReleasedValue: &port, Comment: "服务监听端口", UpdatedBy: "system", UpdatedAt: now}
s.configs[31] = domain.ConfigItem{ID: 31, AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: 1, Key: "log.level", Value: level, ReleasedValue: &level, Comment: "日志级别", UpdatedBy: "system", UpdatedAt: now}
release := domain.Release{ID: 40, AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: 1, Version: 1, Snapshot: map[string]string{"server.port": port, "log.level": level}, Added: 2, Operator: "system", Comment: "初始化示例配置", Status: "pending", Time: now}
s.releases[release.ID] = release
s.enqueue(release)
}
func (s *Store) ListApplications(context.Context) ([]domain.Application, error) {
s.mu.RLock()
defer s.mu.RUnlock()
result := make([]domain.Application, 0, len(s.applications))
for _, item := range s.applications {
result = append(result, item)
}
sort.Slice(result, func(i, j int) bool { return result[i].Code < result[j].Code })
return result, nil
}
func (s *Store) CreateApplication(_ context.Context, input domain.Application) (domain.Application, error) {
s.mu.Lock()
defer s.mu.Unlock()
for _, item := range s.applications {
if strings.EqualFold(item.Code, input.Code) {
return domain.Application{}, storepkg.ErrConflict
}
}
now := time.Now().UTC()
input.ID, input.CreatedAt, input.UpdatedAt = s.id(), now, now
s.applications[input.ID] = input
s.audit(defaultActor(input.UpdatedBy), "create", "app", input.ID, input)
return input, nil
}
func (s *Store) UpdateApplication(_ context.Context, id int64, input domain.Application) (domain.Application, error) {
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.applications[id]
if !ok {
return domain.Application{}, storepkg.ErrNotFound
}
current.Name, current.Department, current.Owner, current.Desc = input.Name, input.Department, input.Owner, input.Desc
current.UpdatedAt = time.Now().UTC()
s.applications[id] = current
s.audit(defaultActor(input.UpdatedBy), "update", "app", id, current)
return current, nil
}
func (s *Store) DeleteApplication(_ context.Context, id int64, actor string) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.applications[id]; !ok {
return storepkg.ErrNotFound
}
delete(s.applications, id)
for nsID, ns := range s.namespaces {
if ns.AppID == id {
delete(s.namespaces, nsID)
}
}
for configID, item := range s.configs {
if item.AppID == id {
delete(s.configs, configID)
}
}
for releaseID, item := range s.releases {
if item.AppID == id {
delete(s.releases, releaseID)
}
}
for ruleID, rule := range s.grayRules {
if rule.AppID == id {
delete(s.grayRules, ruleID)
}
}
for key := range s.userRoles {
if key[1] == id {
delete(s.userRoles, key)
}
}
s.audit(defaultActor(actor), "delete", "app", id, nil)
return nil
}
func (s *Store) ListEnvironments(context.Context) ([]domain.Environment, error) {
s.mu.RLock()
defer s.mu.RUnlock()
result := make([]domain.Environment, 0, len(s.environments))
for _, item := range s.environments {
result = append(result, item)
}
sort.Slice(result, func(i, j int) bool {
if result[i].Order == result[j].Order {
return result[i].Code < result[j].Code
}
return result[i].Order < result[j].Order
})
return result, nil
}
func (s *Store) CreateEnvironment(_ context.Context, input domain.Environment) (domain.Environment, error) {
s.mu.Lock()
defer s.mu.Unlock()
for _, item := range s.environments {
if strings.EqualFold(item.Code, input.Code) {
return domain.Environment{}, storepkg.ErrConflict
}
}
input.ID = s.id()
input.Code = strings.ToUpper(input.Code)
s.environments[input.ID] = input
s.audit(defaultActor(input.UpdatedBy), "create", "env", input.ID, input)
return input, nil
}
func (s *Store) UpdateEnvironment(_ context.Context, id int64, input domain.Environment) (domain.Environment, error) {
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.environments[id]
if !ok {
return domain.Environment{}, storepkg.ErrNotFound
}
current.Name, current.Order, current.Desc = input.Name, input.Order, input.Desc
s.environments[id] = current
s.audit(defaultActor(input.UpdatedBy), "update", "env", id, current)
return current, nil
}
func (s *Store) DeleteEnvironment(_ context.Context, id int64, actor string) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.environments[id]; !ok {
return storepkg.ErrNotFound
}
delete(s.environments, id)
for configID, item := range s.configs {
if item.EnvironmentID == id {
delete(s.configs, configID)
}
}
for releaseID, item := range s.releases {
if item.EnvironmentID == id {
delete(s.releases, releaseID)
}
}
for ruleID, rule := range s.grayRules {
if rule.EnvironmentID == id {
delete(s.grayRules, ruleID)
}
}
s.audit(defaultActor(actor), "delete", "env", id, nil)
return nil
}
func (s *Store) ListNamespaces(_ context.Context, appID *int64) ([]domain.Namespace, error) {
s.mu.RLock()
defer s.mu.RUnlock()
result := make([]domain.Namespace, 0)
for _, item := range s.namespaces {
if appID == nil || item.AppID == *appID {
result = append(result, item)
}
}
sort.Slice(result, func(i, j int) bool {
if result[i].AppID == result[j].AppID {
return result[i].Name < result[j].Name
}
return result[i].AppID < result[j].AppID
})
return result, nil
}
func (s *Store) GetNamespace(_ context.Context, id int64) (domain.Namespace, error) {
s.mu.RLock()
defer s.mu.RUnlock()
item, ok := s.namespaces[id]
if !ok {
return domain.Namespace{}, storepkg.ErrNotFound
}
return item, nil
}
func (s *Store) CreateNamespace(_ context.Context, input domain.Namespace) (domain.Namespace, error) {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.applications[input.AppID]; !ok {
return domain.Namespace{}, storepkg.ErrNotFound
}
for _, item := range s.namespaces {
if item.AppID == input.AppID && strings.EqualFold(item.Name, input.Name) {
return domain.Namespace{}, storepkg.ErrConflict
}
}
input.ID = s.id()
s.namespaces[input.ID] = input
s.audit(defaultActor(input.UpdatedBy), "create", "namespace", input.ID, input)
return input, nil
}
func (s *Store) UpdateNamespace(_ context.Context, id int64, input domain.Namespace) (domain.Namespace, error) {
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.namespaces[id]
if !ok {
return domain.Namespace{}, storepkg.ErrNotFound
}
current.Type, current.Desc = input.Type, input.Desc
s.namespaces[id] = current
s.audit(defaultActor(input.UpdatedBy), "update", "namespace", id, current)
return current, nil
}
func (s *Store) DeleteNamespace(_ context.Context, id int64, actor string) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.namespaces[id]; !ok {
return storepkg.ErrNotFound
}
delete(s.namespaces, id)
for configID, item := range s.configs {
if item.NamespaceID == id {
delete(s.configs, configID)
}
}
for releaseID, item := range s.releases {
if item.NamespaceID == id {
delete(s.releases, releaseID)
}
}
for ruleID, rule := range s.grayRules {
if rule.NamespaceID == id {
delete(s.grayRules, ruleID)
}
}
s.audit(defaultActor(actor), "delete", "namespace", id, nil)
return nil
}
func (s *Store) ListConfigItems(_ context.Context, appID, namespaceID, envID int64) ([]domain.ConfigItem, error) {
s.mu.RLock()
defer s.mu.RUnlock()
result := make([]domain.ConfigItem, 0)
for _, item := range s.configs {
if item.AppID == appID && item.NamespaceID == namespaceID && item.EnvironmentID == envID {
result = append(result, item)
}
}
sort.Slice(result, func(i, j int) bool { return result[i].Key < result[j].Key })
return result, nil
}
func (s *Store) GetConfigItem(_ context.Context, id int64) (domain.ConfigItem, error) {
s.mu.RLock()
defer s.mu.RUnlock()
item, ok := s.configs[id]
if !ok {
return domain.ConfigItem{}, storepkg.ErrNotFound
}
return item, nil
}
func (s *Store) CreateConfigItem(_ context.Context, input domain.ConfigItem) (domain.ConfigItem, error) {
s.mu.Lock()
defer s.mu.Unlock()
if !s.validScope(input.AppID, input.NamespaceID, input.EnvironmentID) {
return domain.ConfigItem{}, storepkg.ErrNotFound
}
for _, item := range s.configs {
if item.NamespaceID == input.NamespaceID && item.EnvironmentID == input.EnvironmentID && item.Key == input.Key {
return domain.ConfigItem{}, storepkg.ErrConflict
}
}
input.ID, input.UpdatedAt = s.id(), time.Now().UTC()
input.UpdatedBy = defaultActor(input.UpdatedBy)
input.ReleasedValue = nil
s.configs[input.ID] = input
s.audit(input.UpdatedBy, "create", "config", input.ID, input)
return input, nil
}
func (s *Store) UpdateConfigItem(_ context.Context, id int64, input domain.ConfigItem) (domain.ConfigItem, error) {
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.configs[id]
if !ok {
return domain.ConfigItem{}, storepkg.ErrNotFound
}
current.Value, current.Comment = input.Value, input.Comment
current.UpdatedBy, current.UpdatedAt = defaultActor(input.UpdatedBy), time.Now().UTC()
current.PendingDelete = false
s.configs[id] = current
s.audit(current.UpdatedBy, "update", "config", id, current)
return current, nil
}
func (s *Store) SetConfigItemPendingDelete(_ context.Context, id int64, pending bool, actor string) (domain.ConfigItem, error) {
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.configs[id]
if !ok {
return domain.ConfigItem{}, storepkg.ErrNotFound
}
if current.ReleasedValue == nil && pending {
delete(s.configs, id)
s.audit(defaultActor(actor), "delete", "config", id, map[string]bool{"draftOnly": true})
return current, nil
}
current.PendingDelete, current.UpdatedBy, current.UpdatedAt = pending, defaultActor(actor), time.Now().UTC()
s.configs[id] = current
s.audit(current.UpdatedBy, "update", "config", id, map[string]bool{"pendingDelete": pending})
return current, nil
}
func (s *Store) Publish(_ context.Context, request domain.PublishRequest) (domain.Release, error) {
s.mu.Lock()
defer s.mu.Unlock()
if !s.validScope(request.AppID, request.NamespaceID, request.EnvironmentID) {
return domain.Release{}, storepkg.ErrNotFound
}
items := s.scopeItems(request.AppID, request.NamespaceID, request.EnvironmentID)
added, modified, removed := 0, 0, 0
snapshot := make(map[string]string)
for _, item := range items {
if item.PendingDelete {
removed++
continue
}
snapshot[item.Key] = item.Value
if item.ReleasedValue == nil {
added++
} else if *item.ReleasedValue != item.Value {
modified++
}
}
if added+modified+removed == 0 {
return domain.Release{}, storepkg.ErrNoPendingChange
}
release := s.newRelease(request.AppID, request.NamespaceID, request.EnvironmentID, snapshot, added, modified, removed, request.Comment, defaultActor(request.Operator))
for id, item := range s.configs {
if item.AppID != request.AppID || item.NamespaceID != request.NamespaceID || item.EnvironmentID != request.EnvironmentID {
continue
}
if item.PendingDelete {
delete(s.configs, id)
continue
}
value := item.Value
item.ReleasedValue, item.PendingDelete = &value, false
s.configs[id] = item
}
s.enqueue(release)
s.audit(release.Operator, "publish", "release", release.ID, release)
return release, nil
}
func (s *Store) Rollback(_ context.Context, request domain.RollbackRequest) (domain.Release, error) {
s.mu.Lock()
defer s.mu.Unlock()
var target *domain.Release
var latest *domain.Release
for _, release := range s.releases {
if release.AppID != request.AppID || release.NamespaceID != request.NamespaceID || release.EnvironmentID != request.EnvironmentID {
continue
}
copy := release
if release.Version == request.TargetVersion {
target = &copy
}
if latest == nil || release.Version > latest.Version {
latest = &copy
}
}
if target == nil {
return domain.Release{}, storepkg.ErrNotFound
}
if latest != nil && target.Version == latest.Version {
return domain.Release{}, storepkg.ErrInvalidRollback
}
current := map[string]string{}
if latest != nil {
current = latest.Snapshot
}
added, modified, removed := diff(current, target.Snapshot)
for id, item := range s.configs {
if item.AppID == request.AppID && item.NamespaceID == request.NamespaceID && item.EnvironmentID == request.EnvironmentID {
delete(s.configs, id)
}
}
for key, value := range target.Snapshot {
copyValue := value
item := domain.ConfigItem{ID: s.id(), AppID: request.AppID, NamespaceID: request.NamespaceID, EnvironmentID: request.EnvironmentID, Key: key, Value: value, ReleasedValue: &copyValue, UpdatedBy: defaultActor(request.Operator), UpdatedAt: time.Now().UTC()}
s.configs[item.ID] = item
}
release := s.newRelease(request.AppID, request.NamespaceID, request.EnvironmentID, cloneMap(target.Snapshot), added, modified, removed, fmt.Sprintf("回滚至版本 v%d", target.Version), defaultActor(request.Operator))
s.enqueue(release)
s.audit(release.Operator, "rollback", "release", release.ID, map[string]any{"targetVersion": target.Version, "newVersion": release.Version})
return release, nil
}
func (s *Store) ListReleases(_ context.Context, appID, namespaceID, envID int64) ([]domain.Release, error) {
s.mu.RLock()
defer s.mu.RUnlock()
result := make([]domain.Release, 0)
for _, release := range s.releases {
if release.AppID == appID && release.NamespaceID == namespaceID && release.EnvironmentID == envID {
release.Snapshot = cloneMap(release.Snapshot)
result = append(result, release)
}
}
sort.Slice(result, func(i, j int) bool { return result[i].Version > result[j].Version })
return result, nil
}
func (s *Store) GetRelease(_ context.Context, id int64) (domain.Release, error) {
s.mu.RLock()
defer s.mu.RUnlock()
release, ok := s.releases[id]
if !ok {
return domain.Release{}, storepkg.ErrNotFound
}
release.Snapshot = cloneMap(release.Snapshot)
return release, nil
}
func (s *Store) ListAuditLogs(_ context.Context, limit int) ([]domain.AuditLog, error) {
s.mu.RLock()
defer s.mu.RUnlock()
if limit <= 0 || limit > len(s.audits) {
limit = len(s.audits)
}
result := make([]domain.AuditLog, 0, limit)
for i := len(s.audits) - 1; i >= len(s.audits)-limit; i-- {
result = append(result, s.audits[i])
}
return result, nil
}
func (s *Store) EnsureBootstrapAdmin(_ context.Context, username, passwordHash, displayName string) error {
s.mu.Lock()
defer s.mu.Unlock()
for _, credential := range s.users {
if strings.EqualFold(credential.Username, username) {
return nil
}
}
now := time.Now().UTC()
id := s.id()
s.users[id] = domain.UserCredential{User: domain.User{ID: id, Username: username, DisplayName: displayName, IsAdmin: true, CreatedAt: now}, PasswordHash: passwordHash}
s.audit(username, "create", "user", id, map[string]any{"bootstrap": true, "isAdmin": true})
return nil
}
func (s *Store) FindUserByUsername(_ context.Context, username string) (domain.UserCredential, error) {
s.mu.RLock()
defer s.mu.RUnlock()
for _, credential := range s.users {
if strings.EqualFold(credential.Username, username) {
return credential, nil
}
}
return domain.UserCredential{}, storepkg.ErrNotFound
}
func (s *Store) ListUsers(context.Context) ([]domain.User, error) {
s.mu.RLock()
defer s.mu.RUnlock()
result := make([]domain.User, 0, len(s.users))
for _, credential := range s.users {
result = append(result, credential.User)
}
sort.Slice(result, func(i, j int) bool { return result[i].Username < result[j].Username })
return result, nil
}
func (s *Store) CreateUser(_ context.Context, input domain.User, passwordHash string) (domain.User, error) {
s.mu.Lock()
defer s.mu.Unlock()
for _, credential := range s.users {
if strings.EqualFold(credential.Username, input.Username) {
return domain.User{}, storepkg.ErrConflict
}
}
input.ID, input.CreatedAt = s.id(), time.Now().UTC()
s.users[input.ID] = domain.UserCredential{User: input, PasswordHash: passwordHash}
s.audit(defaultActor(input.UpdatedBy), "create", "user", input.ID, input)
return input, nil
}
func (s *Store) SetUserAppRole(_ context.Context, input domain.UserAppRole) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.users[input.UserID]; !ok {
return storepkg.ErrNotFound
}
if _, ok := s.applications[input.AppID]; !ok {
return storepkg.ErrNotFound
}
if input.Role != "admin" && input.Role != "app-owner" && input.Role != "viewer" {
return errors.New("invalid role")
}
s.userRoles[[2]int64{input.UserID, input.AppID}] = input.Role
s.audit(defaultActor(input.UpdatedBy), "update", "user_app_role", input.UserID, input)
return nil
}
func (s *Store) DeleteUserAppRole(_ context.Context, userID, appID int64, updatedBy string) error {
s.mu.Lock()
defer s.mu.Unlock()
key := [2]int64{userID, appID}
if _, ok := s.userRoles[key]; !ok {
return storepkg.ErrNotFound
}
delete(s.userRoles, key)
s.audit(defaultActor(updatedBy), "delete", "user_app_role", userID, map[string]int64{"appId": appID})
return nil
}
func (s *Store) GetUserAppRole(_ context.Context, userID, appID int64) (string, error) {
s.mu.RLock()
defer s.mu.RUnlock()
role, ok := s.userRoles[[2]int64{userID, appID}]
if !ok {
return "", storepkg.ErrNotFound
}
return role, nil
}
func (s *Store) ListUserAppRoles(_ context.Context, userID int64) ([]domain.UserAppRole, error) {
s.mu.RLock()
defer s.mu.RUnlock()
if _, ok := s.users[userID]; !ok {
return nil, storepkg.ErrNotFound
}
result := make([]domain.UserAppRole, 0)
for key, role := range s.userRoles {
if key[0] == userID {
result = append(result, domain.UserAppRole{UserID: userID, AppID: key[1], Role: role})
}
}
sort.Slice(result, func(i, j int) bool { return result[i].AppID < result[j].AppID })
return result, nil
}
func (s *Store) ResolveScope(_ context.Context, envCode, appCode, namespaceName string) (int64, int64, int64, error) {
s.mu.RLock()
defer s.mu.RUnlock()
var appID, namespaceID, envID int64
for id, item := range s.applications {
if item.Code == appCode {
appID = id
break
}
}
for id, item := range s.environments {
if strings.EqualFold(item.Code, envCode) {
envID = id
break
}
}
for id, item := range s.namespaces {
if item.AppID == appID && item.Name == namespaceName {
namespaceID = id
break
}
}
if appID == 0 || namespaceID == 0 || envID == 0 {
return 0, 0, 0, storepkg.ErrNotFound
}
return appID, namespaceID, envID, nil
}
func (s *Store) ListGrayRules(_ context.Context, appID, namespaceID, envID int64) ([]domain.GrayRule, error) {
s.mu.RLock()
defer s.mu.RUnlock()
if !s.validScope(appID, namespaceID, envID) {
return nil, storepkg.ErrNotFound
}
result := make([]domain.GrayRule, 0)
for _, rule := range s.grayRules {
if rule.AppID == appID && rule.NamespaceID == namespaceID && rule.EnvironmentID == envID {
result = append(result, cloneGrayRule(rule))
}
}
sort.Slice(result, func(i, j int) bool {
if result[i].Priority == result[j].Priority {
return result[i].ID < result[j].ID
}
return result[i].Priority < result[j].Priority
})
return result, nil
}
func (s *Store) GetGrayRule(_ context.Context, id int64) (domain.GrayRule, error) {
s.mu.RLock()
defer s.mu.RUnlock()
rule, ok := s.grayRules[id]
if !ok {
return domain.GrayRule{}, storepkg.ErrNotFound
}
return cloneGrayRule(rule), nil
}
func (s *Store) CreateGrayRule(_ context.Context, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
s.mu.Lock()
defer s.mu.Unlock()
if !s.validScope(input.AppID, input.NamespaceID, input.EnvironmentID) {
return domain.GrayRule{}, storepkg.ErrNotFound
}
input.ID, input.CreatedAt = s.id(), time.Now().UTC()
input = cloneGrayRule(input)
s.grayRules[input.ID] = input
s.audit(defaultActor(updatedBy), "create", "gray_rule", input.ID, input)
return cloneGrayRule(input), nil
}
func (s *Store) UpdateGrayRule(_ context.Context, id int64, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
s.mu.Lock()
defer s.mu.Unlock()
current, ok := s.grayRules[id]
if !ok {
return domain.GrayRule{}, storepkg.ErrNotFound
}
current.RuleType, current.RuleValue, current.Overrides = input.RuleType, append(json.RawMessage(nil), input.RuleValue...), cloneMap(input.Overrides)
current.Enabled, current.Priority, current.Desc = input.Enabled, input.Priority, input.Desc
s.grayRules[id] = current
s.audit(defaultActor(updatedBy), "update", "gray_rule", id, current)
return cloneGrayRule(current), nil
}
func (s *Store) DeleteGrayRule(_ context.Context, id int64, updatedBy string) error {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.grayRules[id]; !ok {
return storepkg.ErrNotFound
}
delete(s.grayRules, id)
s.audit(defaultActor(updatedBy), "delete", "gray_rule", id, nil)
return nil
}
func (s *Store) ClaimOutbox(_ context.Context, limit int) ([]domain.OutboxEntry, error) {
s.mu.Lock()
defer s.mu.Unlock()
now := time.Now().UTC()
ids := make([]int64, 0)
for id, record := range s.outbox {
if record.Status == "pending" && !record.NextAttempt.After(now) {
ids = append(ids, id)
}
}
sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] })
if len(ids) > limit {
ids = ids[:limit]
}
result := make([]domain.OutboxEntry, 0, len(ids))
for _, id := range ids {
record := s.outbox[id]
record.Status = "processing"
s.outbox[id] = record
result = append(result, record.OutboxEntry)
}
return result, nil
}
func (s *Store) MarkOutboxDone(_ context.Context, id, releaseID, revision int64) error {
s.mu.Lock()
defer s.mu.Unlock()
record, ok := s.outbox[id]
if !ok {
return storepkg.ErrNotFound
}
record.Status = "done"
s.outbox[id] = record
release, ok := s.releases[releaseID]
if !ok {
return storepkg.ErrNotFound
}
release.Status, release.EtcdRevision = "applied", &revision
s.releases[releaseID] = release
return nil
}
func (s *Store) MarkOutboxFailed(_ context.Context, id, releaseID int64, _ string, maxRetry int) error {
s.mu.Lock()
defer s.mu.Unlock()
record, ok := s.outbox[id]
if !ok {
return storepkg.ErrNotFound
}
record.RetryCount++
if record.RetryCount >= maxRetry {
record.Status = "failed"
if release, exists := s.releases[releaseID]; exists {
release.Status = "failed"
s.releases[releaseID] = release
}
} else {
record.Status = "pending"
delay := time.Duration(1<<min(record.RetryCount, 6)) * time.Second
record.NextAttempt = time.Now().UTC().Add(delay)
}
s.outbox[id] = record
return nil
}
func (s *Store) OutboxStats(context.Context) (domain.OutboxStats, error) {
s.mu.RLock()
defer s.mu.RUnlock()
var result domain.OutboxStats
for _, record := range s.outbox {
switch record.Status {
case "pending":
result.Pending++
case "processing":
result.Processing++
case "failed":
result.Failed++
}
}
return result, nil
}
func (s *Store) validScope(appID, namespaceID, envID int64) bool {
ns, ok := s.namespaces[namespaceID]
_, envExists := s.environments[envID]
_, appExists := s.applications[appID]
return ok && ns.AppID == appID && envExists && appExists
}
func (s *Store) scopeItems(appID, namespaceID, envID int64) []domain.ConfigItem {
result := make([]domain.ConfigItem, 0)
for _, item := range s.configs {
if item.AppID == appID && item.NamespaceID == namespaceID && item.EnvironmentID == envID {
result = append(result, item)
}
}
return result
}
func (s *Store) newRelease(appID, namespaceID, envID int64, snapshot map[string]string, added, modified, removed int, comment, operator string) domain.Release {
version := 1
for _, release := range s.releases {
if release.NamespaceID == namespaceID && release.EnvironmentID == envID && release.Version >= version {
version = release.Version + 1
}
}
release := domain.Release{ID: s.id(), AppID: appID, NamespaceID: namespaceID, EnvironmentID: envID, Version: version, Snapshot: snapshot, Added: added, Modified: modified, Removed: removed, Comment: comment, Operator: operator, Status: "pending", Time: time.Now().UTC()}
s.releases[release.ID] = release
return release
}
func (s *Store) enqueue(release domain.Release) {
app := s.applications[release.AppID]
ns := s.namespaces[release.NamespaceID]
env := s.environments[release.EnvironmentID]
payload, _ := json.Marshal(release.Snapshot)
id := s.id()
s.outbox[id] = outboxRecord{OutboxEntry: domain.OutboxEntry{ID: id, ReleaseID: release.ID, EtcdKey: fmt.Sprintf("/config/%s/%s/%s", strings.ToLower(env.Code), app.Code, ns.Name), Payload: payload}, Status: "pending"}
}
func (s *Store) audit(actor, action, targetType string, targetID int64, detail any) {
payload, _ := json.Marshal(detail)
id := s.id()
target := targetID
s.audits = append(s.audits, domain.AuditLog{ID: id, Actor: defaultActor(actor), Action: action, TargetType: targetType, TargetID: &target, Detail: payload, CreatedAt: time.Now().UTC()})
}
func diff(current, target map[string]string) (added, modified, removed int) {
for key, value := range target {
old, ok := current[key]
if !ok {
added++
} else if old != value {
modified++
}
}
for key := range current {
if _, ok := target[key]; !ok {
removed++
}
}
return
}
func cloneMap(input map[string]string) map[string]string {
result := make(map[string]string, len(input))
for key, value := range input {
result[key] = value
}
return result
}
func cloneGrayRule(input domain.GrayRule) domain.GrayRule {
input.RuleValue = append(json.RawMessage(nil), input.RuleValue...)
input.Overrides = cloneMap(input.Overrides)
return input
}
func defaultActor(actor string) string {
if strings.TrimSpace(actor) == "" {
return "admin"
}
return strings.TrimSpace(actor)
}

View File

@@ -0,0 +1,80 @@
package memory
import (
"context"
"errors"
"testing"
"github.com/longpeng/configcenter/internal/domain"
storepkg "github.com/longpeng/configcenter/internal/store"
)
func TestPublishCreatesVersionSnapshotAndOutbox(t *testing.T) {
repository := New(false)
ctx := context.Background()
app, err := repository.CreateApplication(ctx, domain.Application{Code: "orders", Name: "Orders"})
if err != nil {
t.Fatal(err)
}
env, err := repository.CreateEnvironment(ctx, domain.Environment{Code: "PROD", Name: "Production", Order: 1})
if err != nil {
t.Fatal(err)
}
ns, err := repository.CreateNamespace(ctx, domain.Namespace{AppID: app.ID, Name: "application", Type: "properties"})
if err != nil {
t.Fatal(err)
}
item, err := repository.CreateConfigItem(ctx, domain.ConfigItem{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Key: "server.port", Value: "8080", UpdatedBy: "alice"})
if err != nil {
t.Fatal(err)
}
release, err := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Comment: "first", Operator: "alice"})
if err != nil {
t.Fatal(err)
}
if release.Version != 1 || release.Added != 1 || release.Snapshot["server.port"] != "8080" || release.Status != "pending" {
t.Fatalf("unexpected release: %#v", release)
}
items, err := repository.ListConfigItems(ctx, app.ID, ns.ID, env.ID)
if err != nil {
t.Fatal(err)
}
if len(items) != 1 || items[0].ReleasedValue == nil || *items[0].ReleasedValue != item.Value {
t.Fatalf("published item was not marked released: %#v", items)
}
entries, err := repository.ClaimOutbox(ctx, 10)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 || entries[0].ReleaseID != release.ID || entries[0].EtcdKey != "/config/prod/orders/application" {
t.Fatalf("unexpected outbox: %#v", entries)
}
if _, err := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID}); !errors.Is(err, storepkg.ErrNoPendingChange) {
t.Fatalf("expected no pending changes, got %v", err)
}
}
func TestRollbackCreatesANewRelease(t *testing.T) {
repository := New(false)
ctx := context.Background()
app, _ := repository.CreateApplication(ctx, domain.Application{Code: "orders", Name: "Orders"})
env, _ := repository.CreateEnvironment(ctx, domain.Environment{Code: "PROD", Name: "Production"})
ns, _ := repository.CreateNamespace(ctx, domain.Namespace{AppID: app.ID, Name: "application", Type: "properties"})
item, _ := repository.CreateConfigItem(ctx, domain.ConfigItem{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Key: "feature", Value: "off"})
first, _ := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Operator: "alice"})
_, _ = repository.UpdateConfigItem(ctx, item.ID, domain.ConfigItem{Value: "on", UpdatedBy: "bob"})
second, _ := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Operator: "bob"})
rolledBack, err := repository.Rollback(ctx, domain.RollbackRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, TargetVersion: first.Version, Operator: "carol"})
if err != nil {
t.Fatal(err)
}
if rolledBack.Version != second.Version+1 || rolledBack.Snapshot["feature"] != "off" || rolledBack.Modified != 1 {
t.Fatalf("unexpected rollback release: %#v", rolledBack)
}
releases, _ := repository.ListReleases(ctx, app.ID, ns.ID, env.ID)
if len(releases) != 3 {
t.Fatalf("rollback must retain history, got %d releases", len(releases))
}
}

View File

@@ -0,0 +1,134 @@
CREATE TABLE IF NOT EXISTS applications (
id BIGSERIAL PRIMARY KEY,
app_code VARCHAR(64) UNIQUE NOT NULL,
name VARCHAR(128) NOT NULL,
department VARCHAR(128) NOT NULL DEFAULT '',
owner VARCHAR(64) NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS environments (
id BIGSERIAL PRIMARY KEY,
env_code VARCHAR(32) UNIQUE NOT NULL,
name VARCHAR(64) NOT NULL,
sort_order INT NOT NULL DEFAULT 0,
description TEXT NOT NULL DEFAULT ''
);
CREATE TABLE IF NOT EXISTS namespaces (
id BIGSERIAL PRIMARY KEY,
app_id BIGINT NOT NULL REFERENCES applications(id) ON DELETE CASCADE,
name VARCHAR(64) NOT NULL,
format VARCHAR(16) NOT NULL DEFAULT 'properties',
description TEXT NOT NULL DEFAULT '',
UNIQUE(app_id, name),
CONSTRAINT namespaces_format_check CHECK (format IN ('properties', 'yaml', 'json', 'xml'))
);
CREATE TABLE IF NOT EXISTS config_items (
id BIGSERIAL PRIMARY KEY,
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
key VARCHAR(256) NOT NULL,
value TEXT NOT NULL,
released_value TEXT,
pending_delete BOOLEAN NOT NULL DEFAULT false,
comment TEXT NOT NULL DEFAULT '',
updated_by VARCHAR(64) NOT NULL DEFAULT 'admin',
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE(namespace_id, env_id, key)
);
CREATE TABLE IF NOT EXISTS releases (
id BIGSERIAL PRIMARY KEY,
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
version INT NOT NULL,
snapshot JSONB NOT NULL,
diff_added INT NOT NULL DEFAULT 0,
diff_modified INT NOT NULL DEFAULT 0,
diff_removed INT NOT NULL DEFAULT 0,
comment TEXT NOT NULL DEFAULT '',
operator VARCHAR(64) NOT NULL,
etcd_revision BIGINT,
status VARCHAR(16) NOT NULL DEFAULT 'pending',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE(namespace_id, env_id, version),
CONSTRAINT releases_status_check CHECK (status IN ('pending', 'applied', 'failed'))
);
CREATE TABLE IF NOT EXISTS release_outbox (
id BIGSERIAL PRIMARY KEY,
release_id BIGINT NOT NULL REFERENCES releases(id) ON DELETE CASCADE,
etcd_key VARCHAR(512) NOT NULL,
payload JSONB NOT NULL,
status VARCHAR(16) NOT NULL DEFAULT 'pending',
retry_count INT NOT NULL DEFAULT 0,
last_error TEXT NOT NULL DEFAULT '',
next_attempt_at TIMESTAMPTZ NOT NULL DEFAULT now(),
locked_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT outbox_status_check CHECK (status IN ('pending', 'processing', 'done', 'failed'))
);
CREATE INDEX IF NOT EXISTS release_outbox_poll_idx
ON release_outbox(status, next_attempt_at, id);
CREATE INDEX IF NOT EXISTS releases_scope_idx
ON releases(namespace_id, env_id, version DESC);
CREATE INDEX IF NOT EXISTS config_items_scope_idx
ON config_items(namespace_id, env_id, key);
CREATE TABLE IF NOT EXISTS users (
id BIGSERIAL PRIMARY KEY,
username VARCHAR(64) UNIQUE NOT NULL,
password_hash VARCHAR(256) NOT NULL,
display_name VARCHAR(64) NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS roles (
id BIGSERIAL PRIMARY KEY,
name VARCHAR(32) UNIQUE NOT NULL
);
CREATE TABLE IF NOT EXISTS user_app_roles (
user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
app_id BIGINT NOT NULL REFERENCES applications(id) ON DELETE CASCADE,
role_id BIGINT NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
PRIMARY KEY (user_id, app_id)
);
CREATE TABLE IF NOT EXISTS audit_logs (
id BIGSERIAL PRIMARY KEY,
actor VARCHAR(64) NOT NULL,
action VARCHAR(32) NOT NULL,
target_type VARCHAR(32) NOT NULL,
target_id BIGINT,
detail JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS audit_logs_created_idx ON audit_logs(created_at DESC);
CREATE TABLE IF NOT EXISTS gray_rules (
id BIGSERIAL PRIMARY KEY,
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
rule_type VARCHAR(16) NOT NULL,
rule_value JSONB NOT NULL,
overrides JSONB NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
INSERT INTO environments(env_code, name, sort_order, description) VALUES
('DEV', '开发环境', 1, '开发联调使用'),
('TEST', '测试环境', 2, 'QA 测试验证'),
('STAGING', '预发环境', 3, '上线前灰度验证'),
('PROD', '生产环境', 4, '线上正式环境')
ON CONFLICT (env_code) DO NOTHING;
INSERT INTO roles(name) VALUES ('admin'), ('app-owner'), ('viewer')
ON CONFLICT (name) DO NOTHING;

View File

@@ -0,0 +1,10 @@
ALTER TABLE users ADD COLUMN IF NOT EXISTS is_admin BOOLEAN NOT NULL DEFAULT false;
ALTER TABLE users ADD COLUMN IF NOT EXISTS disabled BOOLEAN NOT NULL DEFAULT false;
CREATE INDEX IF NOT EXISTS user_app_roles_app_idx ON user_app_roles(app_id, user_id);
ALTER TABLE gray_rules ADD COLUMN IF NOT EXISTS priority INT NOT NULL DEFAULT 0;
ALTER TABLE gray_rules ADD COLUMN IF NOT EXISTS description TEXT NOT NULL DEFAULT '';
CREATE INDEX IF NOT EXISTS gray_rules_scope_idx
ON gray_rules(namespace_id, env_id, enabled, priority, id);

View File

@@ -0,0 +1,930 @@
package postgres
import (
"context"
"embed"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/longpeng/configcenter/internal/domain"
storepkg "github.com/longpeng/configcenter/internal/store"
)
//go:embed migrations/*.sql
var migrations embed.FS
type Store struct {
pool *pgxpool.Pool
}
func Open(ctx context.Context, databaseURL string) (*Store, error) {
pool, err := pgxpool.New(ctx, databaseURL)
if err != nil {
return nil, fmt.Errorf("create postgres pool: %w", err)
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
return nil, fmt.Errorf("ping postgres: %w", err)
}
return &Store{pool: pool}, nil
}
func (s *Store) Close() { s.pool.Close() }
func (s *Store) Ping(ctx context.Context) error { return s.pool.Ping(ctx) }
func (s *Store) Migrate(ctx context.Context) error {
entries, err := migrations.ReadDir("migrations")
if err != nil {
return fmt.Errorf("list migrations: %w", err)
}
for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".sql") {
continue
}
content, err := migrations.ReadFile("migrations/" + entry.Name())
if err != nil {
return fmt.Errorf("read migration %s: %w", entry.Name(), err)
}
if _, err := s.pool.Exec(ctx, string(content)); err != nil {
return fmt.Errorf("apply migration %s: %w", entry.Name(), err)
}
}
return nil
}
func (s *Store) ListApplications(ctx context.Context) ([]domain.Application, error) {
rows, err := s.pool.Query(ctx, `SELECT id, app_code, name, department, owner, description, created_at, updated_at FROM applications ORDER BY app_code`)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.Application, 0)
for rows.Next() {
var item domain.Application
if err := rows.Scan(&item.ID, &item.Code, &item.Name, &item.Department, &item.Owner, &item.Desc, &item.CreatedAt, &item.UpdatedAt); err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) CreateApplication(ctx context.Context, input domain.Application) (domain.Application, error) {
err := s.pool.QueryRow(ctx, `
INSERT INTO applications(app_code, name, department, owner, description)
VALUES($1, $2, $3, $4, $5)
RETURNING id, app_code, name, department, owner, description, created_at, updated_at`,
input.Code, input.Name, input.Department, input.Owner, input.Desc,
).Scan(&input.ID, &input.Code, &input.Name, &input.Department, &input.Owner, &input.Desc, &input.CreatedAt, &input.UpdatedAt)
if err != nil {
return domain.Application{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "create", "app", input.ID, input)
return input, nil
}
func (s *Store) UpdateApplication(ctx context.Context, id int64, input domain.Application) (domain.Application, error) {
err := s.pool.QueryRow(ctx, `
UPDATE applications SET name=$2, department=$3, owner=$4, description=$5, updated_at=now()
WHERE id=$1
RETURNING id, app_code, name, department, owner, description, created_at, updated_at`,
id, input.Name, input.Department, input.Owner, input.Desc,
).Scan(&input.ID, &input.Code, &input.Name, &input.Department, &input.Owner, &input.Desc, &input.CreatedAt, &input.UpdatedAt)
if err != nil {
return domain.Application{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "update", "app", id, input)
return input, nil
}
func (s *Store) DeleteApplication(ctx context.Context, id int64, actor string) error {
return s.deleteAndAudit(ctx, "applications", "app", id, actor)
}
func (s *Store) ListEnvironments(ctx context.Context) ([]domain.Environment, error) {
rows, err := s.pool.Query(ctx, `SELECT id, env_code, name, sort_order, description FROM environments ORDER BY sort_order, env_code`)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.Environment, 0)
for rows.Next() {
var item domain.Environment
if err := rows.Scan(&item.ID, &item.Code, &item.Name, &item.Order, &item.Desc); err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) CreateEnvironment(ctx context.Context, input domain.Environment) (domain.Environment, error) {
input.Code = strings.ToUpper(input.Code)
err := s.pool.QueryRow(ctx, `
INSERT INTO environments(env_code, name, sort_order, description) VALUES($1, $2, $3, $4)
RETURNING id, env_code, name, sort_order, description`, input.Code, input.Name, input.Order, input.Desc,
).Scan(&input.ID, &input.Code, &input.Name, &input.Order, &input.Desc)
if err != nil {
return domain.Environment{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "create", "env", input.ID, input)
return input, nil
}
func (s *Store) UpdateEnvironment(ctx context.Context, id int64, input domain.Environment) (domain.Environment, error) {
err := s.pool.QueryRow(ctx, `
UPDATE environments SET name=$2, sort_order=$3, description=$4 WHERE id=$1
RETURNING id, env_code, name, sort_order, description`, id, input.Name, input.Order, input.Desc,
).Scan(&input.ID, &input.Code, &input.Name, &input.Order, &input.Desc)
if err != nil {
return domain.Environment{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "update", "env", id, input)
return input, nil
}
func (s *Store) DeleteEnvironment(ctx context.Context, id int64, actor string) error {
return s.deleteAndAudit(ctx, "environments", "env", id, actor)
}
func (s *Store) ListNamespaces(ctx context.Context, appID *int64) ([]domain.Namespace, error) {
query := `SELECT id, app_id, name, format, description FROM namespaces`
args := []any{}
if appID != nil {
query += ` WHERE app_id=$1`
args = append(args, *appID)
}
query += ` ORDER BY app_id, name`
rows, err := s.pool.Query(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.Namespace, 0)
for rows.Next() {
var item domain.Namespace
if err := rows.Scan(&item.ID, &item.AppID, &item.Name, &item.Type, &item.Desc); err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) GetNamespace(ctx context.Context, id int64) (domain.Namespace, error) {
var item domain.Namespace
err := s.pool.QueryRow(ctx, `SELECT id, app_id, name, format, description FROM namespaces WHERE id=$1`, id).
Scan(&item.ID, &item.AppID, &item.Name, &item.Type, &item.Desc)
return item, classify(err)
}
func (s *Store) CreateNamespace(ctx context.Context, input domain.Namespace) (domain.Namespace, error) {
err := s.pool.QueryRow(ctx, `
INSERT INTO namespaces(app_id, name, format, description) VALUES($1, $2, $3, $4)
RETURNING id, app_id, name, format, description`, input.AppID, input.Name, input.Type, input.Desc,
).Scan(&input.ID, &input.AppID, &input.Name, &input.Type, &input.Desc)
if err != nil {
return domain.Namespace{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "create", "namespace", input.ID, input)
return input, nil
}
func (s *Store) UpdateNamespace(ctx context.Context, id int64, input domain.Namespace) (domain.Namespace, error) {
err := s.pool.QueryRow(ctx, `
UPDATE namespaces SET format=$2, description=$3 WHERE id=$1
RETURNING id, app_id, name, format, description`, id, input.Type, input.Desc,
).Scan(&input.ID, &input.AppID, &input.Name, &input.Type, &input.Desc)
if err != nil {
return domain.Namespace{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "update", "namespace", id, input)
return input, nil
}
func (s *Store) DeleteNamespace(ctx context.Context, id int64, actor string) error {
return s.deleteAndAudit(ctx, "namespaces", "namespace", id, actor)
}
func (s *Store) ListConfigItems(ctx context.Context, appID, namespaceID, envID int64) ([]domain.ConfigItem, error) {
rows, err := s.pool.Query(ctx, `
SELECT ci.id, n.app_id, ci.namespace_id, ci.env_id, ci.key, ci.value, ci.released_value,
ci.pending_delete, ci.comment, ci.updated_by, ci.updated_at
FROM config_items ci JOIN namespaces n ON n.id=ci.namespace_id
WHERE n.app_id=$1 AND ci.namespace_id=$2 AND ci.env_id=$3 ORDER BY ci.key`, appID, namespaceID, envID)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.ConfigItem, 0)
for rows.Next() {
item, err := scanConfig(rows)
if err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) GetConfigItem(ctx context.Context, id int64) (domain.ConfigItem, error) {
row := s.pool.QueryRow(ctx, `
SELECT ci.id, n.app_id, ci.namespace_id, ci.env_id, ci.key, ci.value, ci.released_value,
ci.pending_delete, ci.comment, ci.updated_by, ci.updated_at
FROM config_items ci JOIN namespaces n ON n.id=ci.namespace_id WHERE ci.id=$1`, id)
item, err := scanConfig(row)
return item, classify(err)
}
func (s *Store) CreateConfigItem(ctx context.Context, input domain.ConfigItem) (domain.ConfigItem, error) {
err := s.pool.QueryRow(ctx, `
INSERT INTO config_items(namespace_id, env_id, key, value, comment, updated_by)
SELECT n.id, $3, $4, $5, $6, $7 FROM namespaces n WHERE n.id=$2 AND n.app_id=$1
RETURNING id, namespace_id, env_id, key, value, released_value, pending_delete, comment, updated_by, updated_at`,
input.AppID, input.NamespaceID, input.EnvironmentID, input.Key, input.Value, input.Comment, actor(input.UpdatedBy),
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.Key, &input.Value, &input.ReleasedValue, &input.PendingDelete, &input.Comment, &input.UpdatedBy, &input.UpdatedAt)
if err != nil {
return domain.ConfigItem{}, classify(err)
}
s.audit(ctx, input.UpdatedBy, "create", "config", input.ID, input)
return input, nil
}
func (s *Store) UpdateConfigItem(ctx context.Context, id int64, input domain.ConfigItem) (domain.ConfigItem, error) {
err := s.pool.QueryRow(ctx, `
UPDATE config_items SET value=$2, comment=$3, updated_by=$4, updated_at=now(), pending_delete=false WHERE id=$1
RETURNING id, namespace_id, env_id, key, value, released_value, pending_delete, comment, updated_by, updated_at`,
id, input.Value, input.Comment, actor(input.UpdatedBy),
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.Key, &input.Value, &input.ReleasedValue, &input.PendingDelete, &input.Comment, &input.UpdatedBy, &input.UpdatedAt)
if err != nil {
return domain.ConfigItem{}, classify(err)
}
if err := s.pool.QueryRow(ctx, `SELECT app_id FROM namespaces WHERE id=$1`, input.NamespaceID).Scan(&input.AppID); err != nil {
return domain.ConfigItem{}, err
}
s.audit(ctx, input.UpdatedBy, "update", "config", id, input)
return input, nil
}
func (s *Store) SetConfigItemPendingDelete(ctx context.Context, id int64, pending bool, updatedBy string) (domain.ConfigItem, error) {
tx, err := s.pool.Begin(ctx)
if err != nil {
return domain.ConfigItem{}, err
}
defer tx.Rollback(ctx) //nolint:errcheck
var released *string
if err := tx.QueryRow(ctx, `SELECT released_value FROM config_items WHERE id=$1 FOR UPDATE`, id).Scan(&released); err != nil {
return domain.ConfigItem{}, classify(err)
}
if released == nil && pending {
if _, err := tx.Exec(ctx, `DELETE FROM config_items WHERE id=$1`, id); err != nil {
return domain.ConfigItem{}, err
}
if err := insertAudit(ctx, tx, actor(updatedBy), "delete", "config", id, map[string]bool{"draftOnly": true}); err != nil {
return domain.ConfigItem{}, err
}
if err := tx.Commit(ctx); err != nil {
return domain.ConfigItem{}, err
}
return domain.ConfigItem{ID: id}, nil
}
var item domain.ConfigItem
err = tx.QueryRow(ctx, `
UPDATE config_items SET pending_delete=$2, updated_by=$3, updated_at=now() WHERE id=$1
RETURNING id, namespace_id, env_id, key, value, released_value, pending_delete, comment, updated_by, updated_at`,
id, pending, actor(updatedBy),
).Scan(&item.ID, &item.NamespaceID, &item.EnvironmentID, &item.Key, &item.Value, &item.ReleasedValue, &item.PendingDelete, &item.Comment, &item.UpdatedBy, &item.UpdatedAt)
if err != nil {
return domain.ConfigItem{}, classify(err)
}
if err := tx.QueryRow(ctx, `SELECT app_id FROM namespaces WHERE id=$1`, item.NamespaceID).Scan(&item.AppID); err != nil {
return domain.ConfigItem{}, err
}
if err := insertAudit(ctx, tx, item.UpdatedBy, "update", "config", id, map[string]bool{"pendingDelete": pending}); err != nil {
return domain.ConfigItem{}, err
}
if err := tx.Commit(ctx); err != nil {
return domain.ConfigItem{}, err
}
return item, nil
}
func (s *Store) Publish(ctx context.Context, request domain.PublishRequest) (domain.Release, error) {
tx, err := s.pool.BeginTx(ctx, pgx.TxOptions{IsoLevel: pgx.ReadCommitted})
if err != nil {
return domain.Release{}, err
}
defer tx.Rollback(ctx) //nolint:errcheck
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock($1)`, advisoryKey(request.NamespaceID, request.EnvironmentID)); err != nil {
return domain.Release{}, err
}
appCode, namespaceName, envCode, err := loadScope(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID)
if err != nil {
return domain.Release{}, classify(err)
}
items, err := listConfigTx(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID, true)
if err != nil {
return domain.Release{}, err
}
snapshot := make(map[string]string)
added, modified, removed := 0, 0, 0
for _, item := range items {
if item.PendingDelete {
removed++
continue
}
snapshot[item.Key] = item.Value
if item.ReleasedValue == nil {
added++
} else if *item.ReleasedValue != item.Value {
modified++
}
}
if added+modified+removed == 0 {
return domain.Release{}, storepkg.ErrNoPendingChange
}
version, err := nextVersion(ctx, tx, request.NamespaceID, request.EnvironmentID)
if err != nil {
return domain.Release{}, err
}
release, err := insertRelease(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID, version, snapshot, added, modified, removed, request.Comment, actor(request.Operator))
if err != nil {
return domain.Release{}, err
}
key := fmt.Sprintf("/config/%s/%s/%s", strings.ToLower(envCode), appCode, namespaceName)
if _, err := tx.Exec(ctx, `INSERT INTO release_outbox(release_id, etcd_key, payload) VALUES($1, $2, $3)`, release.ID, key, snapshot); err != nil {
return domain.Release{}, err
}
if _, err := tx.Exec(ctx, `DELETE FROM config_items WHERE namespace_id=$1 AND env_id=$2 AND pending_delete=true`, request.NamespaceID, request.EnvironmentID); err != nil {
return domain.Release{}, err
}
if _, err := tx.Exec(ctx, `UPDATE config_items SET released_value=value, pending_delete=false WHERE namespace_id=$1 AND env_id=$2`, request.NamespaceID, request.EnvironmentID); err != nil {
return domain.Release{}, err
}
if err := insertAudit(ctx, tx, release.Operator, "publish", "release", release.ID, release); err != nil {
return domain.Release{}, err
}
if err := tx.Commit(ctx); err != nil {
return domain.Release{}, err
}
return release, nil
}
func (s *Store) Rollback(ctx context.Context, request domain.RollbackRequest) (domain.Release, error) {
tx, err := s.pool.Begin(ctx)
if err != nil {
return domain.Release{}, err
}
defer tx.Rollback(ctx) //nolint:errcheck
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock($1)`, advisoryKey(request.NamespaceID, request.EnvironmentID)); err != nil {
return domain.Release{}, err
}
appCode, namespaceName, envCode, err := loadScope(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID)
if err != nil {
return domain.Release{}, classify(err)
}
var targetSnapshot, currentSnapshot map[string]string
var targetVersion int
err = tx.QueryRow(ctx, `SELECT version, snapshot FROM releases WHERE namespace_id=$1 AND env_id=$2 AND version=$3`, request.NamespaceID, request.EnvironmentID, request.TargetVersion).Scan(&targetVersion, &targetSnapshot)
if err != nil {
return domain.Release{}, classify(err)
}
var latestVersion int
err = tx.QueryRow(ctx, `SELECT version, snapshot FROM releases WHERE namespace_id=$1 AND env_id=$2 ORDER BY version DESC LIMIT 1`, request.NamespaceID, request.EnvironmentID).Scan(&latestVersion, &currentSnapshot)
if err != nil {
return domain.Release{}, classify(err)
}
if latestVersion == targetVersion {
return domain.Release{}, storepkg.ErrInvalidRollback
}
added, modified, removed := diff(currentSnapshot, targetSnapshot)
if _, err := tx.Exec(ctx, `DELETE FROM config_items WHERE namespace_id=$1 AND env_id=$2`, request.NamespaceID, request.EnvironmentID); err != nil {
return domain.Release{}, err
}
for key, value := range targetSnapshot {
if _, err := tx.Exec(ctx, `INSERT INTO config_items(namespace_id, env_id, key, value, released_value, updated_by) VALUES($1,$2,$3,$4,$4,$5)`, request.NamespaceID, request.EnvironmentID, key, value, actor(request.Operator)); err != nil {
return domain.Release{}, err
}
}
version := latestVersion + 1
comment := fmt.Sprintf("回滚至版本 v%d", targetVersion)
release, err := insertRelease(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID, version, targetSnapshot, added, modified, removed, comment, actor(request.Operator))
if err != nil {
return domain.Release{}, err
}
key := fmt.Sprintf("/config/%s/%s/%s", strings.ToLower(envCode), appCode, namespaceName)
if _, err := tx.Exec(ctx, `INSERT INTO release_outbox(release_id, etcd_key, payload) VALUES($1,$2,$3)`, release.ID, key, targetSnapshot); err != nil {
return domain.Release{}, err
}
if err := insertAudit(ctx, tx, release.Operator, "rollback", "release", release.ID, map[string]int{"targetVersion": targetVersion, "newVersion": version}); err != nil {
return domain.Release{}, err
}
if err := tx.Commit(ctx); err != nil {
return domain.Release{}, err
}
return release, nil
}
func (s *Store) ListReleases(ctx context.Context, appID, namespaceID, envID int64) ([]domain.Release, error) {
rows, err := s.pool.Query(ctx, `
SELECT r.id, n.app_id, r.namespace_id, r.env_id, r.version, r.snapshot, r.diff_added, r.diff_modified,
r.diff_removed, r.comment, r.operator, r.etcd_revision, r.status, r.created_at
FROM releases r JOIN namespaces n ON n.id=r.namespace_id
WHERE n.app_id=$1 AND r.namespace_id=$2 AND r.env_id=$3 ORDER BY r.version DESC`, appID, namespaceID, envID)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.Release, 0)
for rows.Next() {
item, err := scanRelease(rows)
if err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) GetRelease(ctx context.Context, id int64) (domain.Release, error) {
row := s.pool.QueryRow(ctx, `
SELECT r.id, n.app_id, r.namespace_id, r.env_id, r.version, r.snapshot, r.diff_added, r.diff_modified,
r.diff_removed, r.comment, r.operator, r.etcd_revision, r.status, r.created_at
FROM releases r JOIN namespaces n ON n.id=r.namespace_id WHERE r.id=$1`, id)
item, err := scanRelease(row)
return item, classify(err)
}
func (s *Store) ListAuditLogs(ctx context.Context, limit int) ([]domain.AuditLog, error) {
if limit <= 0 || limit > 500 {
limit = 100
}
rows, err := s.pool.Query(ctx, `SELECT id, actor, action, target_type, target_id, detail, created_at FROM audit_logs ORDER BY created_at DESC LIMIT $1`, limit)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.AuditLog, 0)
for rows.Next() {
var item domain.AuditLog
if err := rows.Scan(&item.ID, &item.Actor, &item.Action, &item.TargetType, &item.TargetID, &item.Detail, &item.CreatedAt); err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) EnsureBootstrapAdmin(ctx context.Context, username, passwordHash, displayName string) error {
_, err := s.pool.Exec(ctx, `
INSERT INTO users(username, password_hash, display_name, is_admin)
VALUES($1,$2,$3,true) ON CONFLICT(username) DO NOTHING`, username, passwordHash, displayName)
return classify(err)
}
func (s *Store) FindUserByUsername(ctx context.Context, username string) (domain.UserCredential, error) {
var item domain.UserCredential
err := s.pool.QueryRow(ctx, `
SELECT id, username, password_hash, display_name, is_admin, disabled, created_at
FROM users WHERE lower(username)=lower($1)`, username).
Scan(&item.ID, &item.Username, &item.PasswordHash, &item.DisplayName, &item.IsAdmin, &item.Disabled, &item.CreatedAt)
return item, classify(err)
}
func (s *Store) ListUsers(ctx context.Context) ([]domain.User, error) {
rows, err := s.pool.Query(ctx, `SELECT id, username, display_name, is_admin, disabled, created_at FROM users ORDER BY username`)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.User, 0)
for rows.Next() {
var item domain.User
if err := rows.Scan(&item.ID, &item.Username, &item.DisplayName, &item.IsAdmin, &item.Disabled, &item.CreatedAt); err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) CreateUser(ctx context.Context, input domain.User, passwordHash string) (domain.User, error) {
err := s.pool.QueryRow(ctx, `
INSERT INTO users(username, password_hash, display_name, is_admin, disabled)
VALUES($1,$2,$3,$4,$5)
RETURNING id, username, display_name, is_admin, disabled, created_at`,
input.Username, passwordHash, input.DisplayName, input.IsAdmin, input.Disabled,
).Scan(&input.ID, &input.Username, &input.DisplayName, &input.IsAdmin, &input.Disabled, &input.CreatedAt)
if err != nil {
return domain.User{}, classify(err)
}
s.audit(ctx, actor(input.UpdatedBy), "create", "user", input.ID, input)
return input, nil
}
func (s *Store) SetUserAppRole(ctx context.Context, input domain.UserAppRole) error {
command, err := s.pool.Exec(ctx, `
INSERT INTO user_app_roles(user_id, app_id, role_id)
SELECT $1, $2, r.id FROM roles r WHERE r.name=$3
ON CONFLICT(user_id, app_id) DO UPDATE SET role_id=EXCLUDED.role_id`, input.UserID, input.AppID, input.Role)
if err != nil {
return classify(err)
}
if command.RowsAffected() == 0 {
return storepkg.ErrNotFound
}
s.audit(ctx, actor(input.UpdatedBy), "update", "user_app_role", input.UserID, input)
return nil
}
func (s *Store) DeleteUserAppRole(ctx context.Context, userID, appID int64, updatedBy string) error {
command, err := s.pool.Exec(ctx, `DELETE FROM user_app_roles WHERE user_id=$1 AND app_id=$2`, userID, appID)
if err != nil {
return err
}
if command.RowsAffected() == 0 {
return storepkg.ErrNotFound
}
s.audit(ctx, actor(updatedBy), "delete", "user_app_role", userID, map[string]int64{"appId": appID})
return nil
}
func (s *Store) GetUserAppRole(ctx context.Context, userID, appID int64) (string, error) {
var role string
err := s.pool.QueryRow(ctx, `
SELECT r.name FROM user_app_roles uar JOIN roles r ON r.id=uar.role_id
WHERE uar.user_id=$1 AND uar.app_id=$2`, userID, appID).Scan(&role)
return role, classify(err)
}
func (s *Store) ListUserAppRoles(ctx context.Context, userID int64) ([]domain.UserAppRole, error) {
rows, err := s.pool.Query(ctx, `
SELECT uar.user_id, uar.app_id, r.name
FROM user_app_roles uar JOIN roles r ON r.id=uar.role_id
WHERE uar.user_id=$1 ORDER BY uar.app_id`, userID)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.UserAppRole, 0)
for rows.Next() {
var item domain.UserAppRole
if err := rows.Scan(&item.UserID, &item.AppID, &item.Role); err != nil {
return nil, err
}
result = append(result, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
if len(result) == 0 {
var exists bool
if err := s.pool.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM users WHERE id=$1)`, userID).Scan(&exists); err != nil {
return nil, err
}
if !exists {
return nil, storepkg.ErrNotFound
}
}
return result, nil
}
func (s *Store) ResolveScope(ctx context.Context, envCode, appCode, namespaceName string) (int64, int64, int64, error) {
var appID, namespaceID, envID int64
err := s.pool.QueryRow(ctx, `
SELECT a.id, n.id, e.id
FROM applications a JOIN namespaces n ON n.app_id=a.id CROSS JOIN environments e
WHERE a.app_code=$1 AND n.name=$2 AND lower(e.env_code)=lower($3)`, appCode, namespaceName, envCode).
Scan(&appID, &namespaceID, &envID)
return appID, namespaceID, envID, classify(err)
}
func (s *Store) ListGrayRules(ctx context.Context, appID, namespaceID, envID int64) ([]domain.GrayRule, error) {
rows, err := s.pool.Query(ctx, `
SELECT g.id, n.app_id, g.namespace_id, g.env_id, g.rule_type, g.rule_value, g.overrides,
g.enabled, g.priority, g.description, g.created_at
FROM gray_rules g JOIN namespaces n ON n.id=g.namespace_id
WHERE n.app_id=$1 AND g.namespace_id=$2 AND g.env_id=$3
ORDER BY g.priority, g.id`, appID, namespaceID, envID)
if err != nil {
return nil, err
}
defer rows.Close()
result := make([]domain.GrayRule, 0)
for rows.Next() {
item, err := scanGrayRule(rows)
if err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) GetGrayRule(ctx context.Context, id int64) (domain.GrayRule, error) {
row := s.pool.QueryRow(ctx, `
SELECT g.id, n.app_id, g.namespace_id, g.env_id, g.rule_type, g.rule_value, g.overrides,
g.enabled, g.priority, g.description, g.created_at
FROM gray_rules g JOIN namespaces n ON n.id=g.namespace_id WHERE g.id=$1`, id)
item, err := scanGrayRule(row)
return item, classify(err)
}
func (s *Store) CreateGrayRule(ctx context.Context, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
err := s.pool.QueryRow(ctx, `
INSERT INTO gray_rules(namespace_id, env_id, rule_type, rule_value, overrides, enabled, priority, description)
SELECT n.id, $3, $4, $5, $6, $7, $8, $9 FROM namespaces n
WHERE n.id=$2 AND n.app_id=$1
RETURNING id, namespace_id, env_id, rule_type, rule_value, overrides, enabled, priority, description, created_at`,
input.AppID, input.NamespaceID, input.EnvironmentID, input.RuleType, input.RuleValue, input.Overrides, input.Enabled, input.Priority, input.Desc,
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.RuleType, &input.RuleValue, &input.Overrides, &input.Enabled, &input.Priority, &input.Desc, &input.CreatedAt)
if err != nil {
return domain.GrayRule{}, classify(err)
}
s.audit(ctx, actor(updatedBy), "create", "gray_rule", input.ID, input)
return input, nil
}
func (s *Store) UpdateGrayRule(ctx context.Context, id int64, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
err := s.pool.QueryRow(ctx, `
UPDATE gray_rules SET rule_type=$2, rule_value=$3, overrides=$4, enabled=$5, priority=$6, description=$7
WHERE id=$1
RETURNING id, namespace_id, env_id, rule_type, rule_value, overrides, enabled, priority, description, created_at`,
id, input.RuleType, input.RuleValue, input.Overrides, input.Enabled, input.Priority, input.Desc,
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.RuleType, &input.RuleValue, &input.Overrides, &input.Enabled, &input.Priority, &input.Desc, &input.CreatedAt)
if err != nil {
return domain.GrayRule{}, classify(err)
}
if err := s.pool.QueryRow(ctx, `SELECT app_id FROM namespaces WHERE id=$1`, input.NamespaceID).Scan(&input.AppID); err != nil {
return domain.GrayRule{}, classify(err)
}
s.audit(ctx, actor(updatedBy), "update", "gray_rule", id, input)
return input, nil
}
func (s *Store) DeleteGrayRule(ctx context.Context, id int64, updatedBy string) error {
command, err := s.pool.Exec(ctx, `DELETE FROM gray_rules WHERE id=$1`, id)
if err != nil {
return err
}
if command.RowsAffected() == 0 {
return storepkg.ErrNotFound
}
s.audit(ctx, actor(updatedBy), "delete", "gray_rule", id, nil)
return nil
}
func (s *Store) ClaimOutbox(ctx context.Context, limit int) ([]domain.OutboxEntry, error) {
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx) //nolint:errcheck
rows, err := tx.Query(ctx, `
SELECT id, release_id, etcd_key, payload, retry_count
FROM release_outbox
WHERE (status='pending' AND next_attempt_at <= now())
OR (status='processing' AND locked_at < now() - interval '60 seconds')
ORDER BY id FOR UPDATE SKIP LOCKED LIMIT $1`, limit)
if err != nil {
return nil, err
}
entries := make([]domain.OutboxEntry, 0)
ids := make([]int64, 0)
for rows.Next() {
var entry domain.OutboxEntry
if err := rows.Scan(&entry.ID, &entry.ReleaseID, &entry.EtcdKey, &entry.Payload, &entry.RetryCount); err != nil {
rows.Close()
return nil, err
}
entries, ids = append(entries, entry), append(ids, entry.ID)
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
for _, id := range ids {
if _, err := tx.Exec(ctx, `UPDATE release_outbox SET status='processing', locked_at=now() WHERE id=$1`, id); err != nil {
return nil, err
}
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return entries, nil
}
func (s *Store) MarkOutboxDone(ctx context.Context, id, releaseID, revision int64) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer tx.Rollback(ctx) //nolint:errcheck
if _, err := tx.Exec(ctx, `UPDATE release_outbox SET status='done', locked_at=NULL WHERE id=$1`, id); err != nil {
return err
}
if _, err := tx.Exec(ctx, `UPDATE releases SET status='applied', etcd_revision=$2 WHERE id=$1`, releaseID, revision); err != nil {
return err
}
return tx.Commit(ctx)
}
func (s *Store) MarkOutboxFailed(ctx context.Context, id, releaseID int64, message string, maxRetry int) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer tx.Rollback(ctx) //nolint:errcheck
var status string
err = tx.QueryRow(ctx, `
UPDATE release_outbox
SET retry_count=retry_count+1,
status=CASE WHEN retry_count+1 >= $3 THEN 'failed' ELSE 'pending' END,
last_error=$2,
next_attempt_at=now() + make_interval(secs => power(2, LEAST(retry_count+1, 6))::int),
locked_at=NULL
WHERE id=$1 RETURNING status`, id, message, maxRetry).Scan(&status)
if err != nil {
return classify(err)
}
if status == "failed" {
if _, err := tx.Exec(ctx, `UPDATE releases SET status='failed' WHERE id=$1`, releaseID); err != nil {
return err
}
}
return tx.Commit(ctx)
}
func (s *Store) OutboxStats(ctx context.Context) (domain.OutboxStats, error) {
var result domain.OutboxStats
err := s.pool.QueryRow(ctx, `
SELECT COUNT(*) FILTER (WHERE status='pending'),
COUNT(*) FILTER (WHERE status='processing'),
COUNT(*) FILTER (WHERE status='failed')
FROM release_outbox`).Scan(&result.Pending, &result.Processing, &result.Failed)
return result, err
}
type rowScanner interface {
Scan(...any) error
}
func scanConfig(row rowScanner) (domain.ConfigItem, error) {
var item domain.ConfigItem
err := row.Scan(&item.ID, &item.AppID, &item.NamespaceID, &item.EnvironmentID, &item.Key, &item.Value, &item.ReleasedValue, &item.PendingDelete, &item.Comment, &item.UpdatedBy, &item.UpdatedAt)
return item, err
}
func scanRelease(row rowScanner) (domain.Release, error) {
var item domain.Release
err := row.Scan(&item.ID, &item.AppID, &item.NamespaceID, &item.EnvironmentID, &item.Version, &item.Snapshot, &item.Added, &item.Modified, &item.Removed, &item.Comment, &item.Operator, &item.EtcdRevision, &item.Status, &item.Time)
return item, err
}
func scanGrayRule(row rowScanner) (domain.GrayRule, error) {
var item domain.GrayRule
err := row.Scan(&item.ID, &item.AppID, &item.NamespaceID, &item.EnvironmentID, &item.RuleType, &item.RuleValue, &item.Overrides, &item.Enabled, &item.Priority, &item.Desc, &item.CreatedAt)
return item, err
}
func listConfigTx(ctx context.Context, tx pgx.Tx, appID, namespaceID, envID int64, lock bool) ([]domain.ConfigItem, error) {
query := `
SELECT ci.id, n.app_id, ci.namespace_id, ci.env_id, ci.key, ci.value, ci.released_value,
ci.pending_delete, ci.comment, ci.updated_by, ci.updated_at
FROM config_items ci JOIN namespaces n ON n.id=ci.namespace_id
WHERE n.app_id=$1 AND ci.namespace_id=$2 AND ci.env_id=$3 ORDER BY ci.key`
if lock {
query += ` FOR UPDATE OF ci`
}
rows, err := tx.Query(ctx, query, appID, namespaceID, envID)
if err != nil {
return nil, err
}
defer rows.Close()
items := make([]domain.ConfigItem, 0)
for rows.Next() {
item, err := scanConfig(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
func loadScope(ctx context.Context, tx pgx.Tx, appID, namespaceID, envID int64) (string, string, string, error) {
var appCode, namespaceName, envCode string
err := tx.QueryRow(ctx, `
SELECT a.app_code, n.name, e.env_code
FROM applications a JOIN namespaces n ON n.app_id=a.id CROSS JOIN environments e
WHERE a.id=$1 AND n.id=$2 AND e.id=$3`, appID, namespaceID, envID).Scan(&appCode, &namespaceName, &envCode)
return appCode, namespaceName, envCode, err
}
func nextVersion(ctx context.Context, tx pgx.Tx, namespaceID, envID int64) (int, error) {
var version int
err := tx.QueryRow(ctx, `SELECT COALESCE(MAX(version),0)+1 FROM releases WHERE namespace_id=$1 AND env_id=$2`, namespaceID, envID).Scan(&version)
return version, err
}
func insertRelease(ctx context.Context, tx pgx.Tx, appID, namespaceID, envID int64, version int, snapshot map[string]string, added, modified, removed int, comment, operator string) (domain.Release, error) {
item := domain.Release{AppID: appID, NamespaceID: namespaceID, EnvironmentID: envID, Version: version, Snapshot: snapshot, Added: added, Modified: modified, Removed: removed, Comment: comment, Operator: operator}
err := tx.QueryRow(ctx, `
INSERT INTO releases(namespace_id, env_id, version, snapshot, diff_added, diff_modified, diff_removed, comment, operator)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9)
RETURNING id, status, created_at`, namespaceID, envID, version, snapshot, added, modified, removed, comment, operator,
).Scan(&item.ID, &item.Status, &item.Time)
return item, err
}
func (s *Store) deleteAndAudit(ctx context.Context, table, targetType string, id int64, updatedBy string) error {
if table != "applications" && table != "environments" && table != "namespaces" {
return errors.New("unsupported delete target")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer tx.Rollback(ctx) //nolint:errcheck
command, err := tx.Exec(ctx, fmt.Sprintf("DELETE FROM %s WHERE id=$1", table), id) // #nosec G201 -- table is allow-listed above.
if err != nil {
return classify(err)
}
if command.RowsAffected() == 0 {
return storepkg.ErrNotFound
}
if err := insertAudit(ctx, tx, actor(updatedBy), "delete", targetType, id, nil); err != nil {
return err
}
return tx.Commit(ctx)
}
func (s *Store) audit(ctx context.Context, updatedBy, action, targetType string, targetID int64, detail any) {
_, _ = s.pool.Exec(ctx, `INSERT INTO audit_logs(actor, action, target_type, target_id, detail) VALUES($1,$2,$3,$4,$5)`, actor(updatedBy), action, targetType, targetID, toJSON(detail))
}
func insertAudit(ctx context.Context, tx pgx.Tx, updatedBy, action, targetType string, targetID int64, detail any) error {
_, err := tx.Exec(ctx, `INSERT INTO audit_logs(actor, action, target_type, target_id, detail) VALUES($1,$2,$3,$4,$5)`, actor(updatedBy), action, targetType, targetID, toJSON(detail))
return err
}
func toJSON(value any) json.RawMessage {
payload, _ := json.Marshal(value)
return payload
}
func classify(err error) error {
if err == nil {
return nil
}
if errors.Is(err, pgx.ErrNoRows) {
return storepkg.ErrNotFound
}
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) {
switch pgErr.Code {
case "23505":
return storepkg.ErrConflict
case "23503", "23514", "23502":
return fmt.Errorf("invalid data: %s", pgErr.Message)
}
}
return err
}
func actor(value string) string {
if strings.TrimSpace(value) == "" {
return "admin"
}
return strings.TrimSpace(value)
}
func advisoryKey(namespaceID, envID int64) int64 {
return namespaceID*1_000_000_000 + envID
}
func diff(current, target map[string]string) (added, modified, removed int) {
for key, value := range target {
old, exists := current[key]
if !exists {
added++
} else if old != value {
modified++
}
}
for key := range current {
if _, exists := target[key]; !exists {
removed++
}
}
return
}
var _ = time.Second

69
internal/store/store.go Normal file
View File

@@ -0,0 +1,69 @@
package store
import (
"context"
"errors"
"github.com/longpeng/configcenter/internal/domain"
)
var (
ErrNotFound = errors.New("resource not found")
ErrConflict = errors.New("resource already exists")
ErrNoPendingChange = errors.New("no pending configuration changes")
ErrInvalidRollback = errors.New("invalid rollback target")
)
type Store interface {
Close()
Ping(context.Context) error
ListApplications(context.Context) ([]domain.Application, error)
CreateApplication(context.Context, domain.Application) (domain.Application, error)
UpdateApplication(context.Context, int64, domain.Application) (domain.Application, error)
DeleteApplication(context.Context, int64, string) error
ListEnvironments(context.Context) ([]domain.Environment, error)
CreateEnvironment(context.Context, domain.Environment) (domain.Environment, error)
UpdateEnvironment(context.Context, int64, domain.Environment) (domain.Environment, error)
DeleteEnvironment(context.Context, int64, string) error
ListNamespaces(context.Context, *int64) ([]domain.Namespace, error)
GetNamespace(context.Context, int64) (domain.Namespace, error)
CreateNamespace(context.Context, domain.Namespace) (domain.Namespace, error)
UpdateNamespace(context.Context, int64, domain.Namespace) (domain.Namespace, error)
DeleteNamespace(context.Context, int64, string) error
ListConfigItems(context.Context, int64, int64, int64) ([]domain.ConfigItem, error)
GetConfigItem(context.Context, int64) (domain.ConfigItem, error)
CreateConfigItem(context.Context, domain.ConfigItem) (domain.ConfigItem, error)
UpdateConfigItem(context.Context, int64, domain.ConfigItem) (domain.ConfigItem, error)
SetConfigItemPendingDelete(context.Context, int64, bool, string) (domain.ConfigItem, error)
Publish(context.Context, domain.PublishRequest) (domain.Release, error)
Rollback(context.Context, domain.RollbackRequest) (domain.Release, error)
ListReleases(context.Context, int64, int64, int64) ([]domain.Release, error)
GetRelease(context.Context, int64) (domain.Release, error)
ListAuditLogs(context.Context, int) ([]domain.AuditLog, error)
EnsureBootstrapAdmin(context.Context, string, string, string) error
FindUserByUsername(context.Context, string) (domain.UserCredential, error)
ListUsers(context.Context) ([]domain.User, error)
CreateUser(context.Context, domain.User, string) (domain.User, error)
SetUserAppRole(context.Context, domain.UserAppRole) error
DeleteUserAppRole(context.Context, int64, int64, string) error
GetUserAppRole(context.Context, int64, int64) (string, error)
ListUserAppRoles(context.Context, int64) ([]domain.UserAppRole, error)
ResolveScope(context.Context, string, string, string) (int64, int64, int64, error)
ListGrayRules(context.Context, int64, int64, int64) ([]domain.GrayRule, error)
GetGrayRule(context.Context, int64) (domain.GrayRule, error)
CreateGrayRule(context.Context, domain.GrayRule, string) (domain.GrayRule, error)
UpdateGrayRule(context.Context, int64, domain.GrayRule, string) (domain.GrayRule, error)
DeleteGrayRule(context.Context, int64, string) error
ClaimOutbox(context.Context, int) ([]domain.OutboxEntry, error)
MarkOutboxDone(context.Context, int64, int64, int64) error
MarkOutboxFailed(context.Context, int64, int64, string, int) error
OutboxStats(context.Context) (domain.OutboxStats, error)
}

82
internal/watch/hub.go Normal file
View File

@@ -0,0 +1,82 @@
package watch
import (
"context"
"sync"
"github.com/longpeng/configcenter/internal/domain"
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
)
type Hub struct {
ctx context.Context
runtime runtimepkg.Store
mu sync.Mutex
keys map[string]*keyWatch
}
type keyWatch struct {
cancel context.CancelFunc
nextID int64
subs map[int64]chan domain.ConfigEvent
}
func New(ctx context.Context, runtimeStore runtimepkg.Store) *Hub {
return &Hub{ctx: ctx, runtime: runtimeStore, keys: make(map[string]*keyWatch)}
}
func (h *Hub) Subscribe(key string) (<-chan domain.ConfigEvent, func()) {
h.mu.Lock()
watch, exists := h.keys[key]
if !exists {
ctx, cancel := context.WithCancel(h.ctx)
watch = &keyWatch{cancel: cancel, subs: make(map[int64]chan domain.ConfigEvent)}
h.keys[key] = watch
go h.run(ctx, key, watch)
}
watch.nextID++
id := watch.nextID
channel := make(chan domain.ConfigEvent, 16)
watch.subs[id] = channel
h.mu.Unlock()
var once sync.Once
return channel, func() {
once.Do(func() {
h.mu.Lock()
current, ok := h.keys[key]
if ok {
if ch, found := current.subs[id]; found {
delete(current.subs, id)
close(ch)
}
if len(current.subs) == 0 {
current.cancel()
delete(h.keys, key)
}
}
h.mu.Unlock()
})
}
}
func (h *Hub) run(ctx context.Context, key string, target *keyWatch) {
for result := range h.runtime.Watch(ctx, key, 0) {
if result.Err != nil {
return
}
h.mu.Lock()
current, exists := h.keys[key]
if !exists || current != target {
h.mu.Unlock()
return
}
for _, channel := range current.subs {
select {
case channel <- result.Event:
default:
}
}
h.mu.Unlock()
}
}

324
pkg/sdk/go/client.go Normal file
View File

@@ -0,0 +1,324 @@
// Package configsdk provides a concurrency-safe Config Center client with
// full-snapshot synchronization and an optional local-file fallback cache.
package configsdk
import (
"bufio"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"time"
)
type Options struct {
BaseURL string
Env string
App string
Token string
IP string
Instance string
CacheFile string
HTTPClient *http.Client
}
type Client struct {
baseURL string
env string
app string
token string
ip string
instance string
cacheFile string
http *http.Client
mu sync.RWMutex
cache map[string]map[string]string
revisions map[string]int64
}
type apiEnvelope struct {
Data json.RawMessage `json:"data"`
}
type runtimeConfig struct {
Items map[string]string `json:"items"`
Revision int64 `json:"revision"`
ReleaseVersion int `json:"releaseVersion"`
}
type configEvent struct {
Type string `json:"type"`
Items map[string]string `json:"items"`
Revision int64 `json:"revision"`
}
type diskCache struct {
Namespaces map[string]map[string]string `json:"namespaces"`
Revisions map[string]int64 `json:"revisions"`
}
func New(options Options) (*Client, error) {
if strings.TrimSpace(options.BaseURL) == "" || strings.TrimSpace(options.Env) == "" || strings.TrimSpace(options.App) == "" {
return nil, errors.New("base URL, environment and application are required")
}
httpClient := options.HTTPClient
if httpClient == nil {
httpClient = &http.Client{Timeout: 15 * time.Second}
}
client := &Client{
baseURL: strings.TrimRight(options.BaseURL, "/"),
env: strings.TrimSpace(options.Env),
app: strings.TrimSpace(options.App),
token: strings.TrimSpace(options.Token),
ip: strings.TrimSpace(options.IP),
instance: strings.TrimSpace(options.Instance),
cacheFile: options.CacheFile,
http: httpClient,
cache: make(map[string]map[string]string),
revisions: make(map[string]int64),
}
if options.CacheFile != "" {
if err := client.loadDiskCache(); err != nil && !errors.Is(err, os.ErrNotExist) {
return nil, fmt.Errorf("load config cache: %w", err)
}
}
return client, nil
}
func (c *Client) GetString(namespace, key, fallback string) string {
c.mu.RLock()
defer c.mu.RUnlock()
if items, ok := c.cache[namespace]; ok {
if value, exists := items[key]; exists {
return value
}
}
return fallback
}
func (c *Client) GetInt(namespace, key string, fallback int) int {
value := c.GetString(namespace, key, "")
parsed, err := strconv.Atoi(value)
if err != nil {
return fallback
}
return parsed
}
func (c *Client) GetBool(namespace, key string, fallback bool) bool {
value := c.GetString(namespace, key, "")
parsed, err := strconv.ParseBool(value)
if err != nil {
return fallback
}
return parsed
}
func (c *Client) Snapshot(namespace string) map[string]string {
c.mu.RLock()
defer c.mu.RUnlock()
return clone(c.cache[namespace])
}
func (c *Client) Load(ctx context.Context, namespace string) error {
endpoint := c.endpoint("/v1/config", namespace)
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
if err != nil {
return err
}
c.authorize(request)
response, err := c.http.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return responseError(response)
}
var envelope apiEnvelope
if err := json.NewDecoder(response.Body).Decode(&envelope); err != nil {
return err
}
var current runtimeConfig
if err := json.Unmarshal(envelope.Data, &current); err != nil {
return err
}
return c.apply(namespace, current.Items, current.Revision)
}
// WatchAndSync blocks until ctx is canceled. It reconnects with exponential
// backoff and replaces the namespace cache only with complete snapshots.
func (c *Client) WatchAndSync(ctx context.Context, namespace string) error {
backoff := time.Second
for {
err := c.watchOnce(ctx, namespace)
if ctx.Err() != nil {
return nil
}
if err == nil {
backoff = time.Second
}
timer := time.NewTimer(backoff)
select {
case <-ctx.Done():
timer.Stop()
return nil
case <-timer.C:
}
if backoff < 30*time.Second {
backoff *= 2
}
}
}
func (c *Client) watchOnce(ctx context.Context, namespace string) error {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, c.endpoint("/v1/watch", namespace), nil)
if err != nil {
return err
}
request.Header.Set("Accept", "text/event-stream")
c.authorize(request)
response, err := c.http.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return responseError(response)
}
scanner := bufio.NewScanner(response.Body)
scanner.Buffer(make([]byte, 64*1024), 10*1024*1024)
var data strings.Builder
for scanner.Scan() {
line := scanner.Text()
if line == "" {
if data.Len() > 0 {
var event configEvent
if err := json.Unmarshal([]byte(data.String()), &event); err != nil {
return err
}
if err := c.apply(namespace, event.Items, event.Revision); err != nil {
return err
}
data.Reset()
}
continue
}
if strings.HasPrefix(line, "data:") {
if data.Len() > 0 {
data.WriteByte('\n')
}
data.WriteString(strings.TrimSpace(strings.TrimPrefix(line, "data:")))
}
}
if err := scanner.Err(); err != nil {
return err
}
return io.EOF
}
func (c *Client) endpoint(path, namespace string) string {
query := url.Values{"env": {c.env}, "app": {c.app}, "namespace": {namespace}}
if c.ip != "" {
query.Set("ip", c.ip)
}
if c.instance != "" {
query.Set("instance", c.instance)
}
return c.baseURL + path + "?" + query.Encode()
}
func (c *Client) authorize(request *http.Request) {
if c.token != "" {
request.Header.Set("Authorization", "Bearer "+c.token)
}
}
func (c *Client) apply(namespace string, items map[string]string, revision int64) error {
c.mu.Lock()
if revision > 0 && revision < c.revisions[namespace] {
c.mu.Unlock()
return nil
}
c.cache[namespace] = clone(items)
c.revisions[namespace] = revision
err := c.saveDiskCacheLocked()
c.mu.Unlock()
return err
}
func (c *Client) loadDiskCache() error {
payload, err := os.ReadFile(c.cacheFile)
if err != nil {
return err
}
var cached diskCache
if err := json.Unmarshal(payload, &cached); err != nil {
return err
}
if cached.Namespaces != nil {
c.cache = cached.Namespaces
}
if cached.Revisions != nil {
c.revisions = cached.Revisions
}
return nil
}
func (c *Client) saveDiskCacheLocked() error {
if c.cacheFile == "" {
return nil
}
payload, err := json.MarshalIndent(diskCache{Namespaces: c.cache, Revisions: c.revisions}, "", " ")
if err != nil {
return err
}
directory := filepath.Dir(c.cacheFile)
if err := os.MkdirAll(directory, 0o700); err != nil {
return err
}
temporary, err := os.CreateTemp(directory, ".configcenter-cache-*")
if err != nil {
return err
}
temporaryName := temporary.Name()
defer os.Remove(temporaryName) //nolint:errcheck
if err := temporary.Chmod(0o600); err != nil {
temporary.Close()
return err
}
if _, err := temporary.Write(payload); err != nil {
temporary.Close()
return err
}
if err := temporary.Sync(); err != nil {
temporary.Close()
return err
}
if err := temporary.Close(); err != nil {
return err
}
return os.Rename(temporaryName, c.cacheFile)
}
func responseError(response *http.Response) error {
payload, _ := io.ReadAll(io.LimitReader(response.Body, 64*1024))
return fmt.Errorf("config center returned %s: %s", response.Status, strings.TrimSpace(string(payload)))
}
func clone(input map[string]string) map[string]string {
result := make(map[string]string, len(input))
for key, value := range input {
result[key] = value
}
return result
}

42
pkg/sdk/go/client_test.go Normal file
View File

@@ -0,0 +1,42 @@
package configsdk
import (
"context"
"io"
"net/http"
"strings"
"testing"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (fn roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) { return fn(request) }
func TestLoadSendsAuthenticationAndGrayTarget(t *testing.T) {
transport := roundTripFunc(func(r *http.Request) (*http.Response, error) {
if got := r.Header.Get("Authorization"); got != "Bearer sdk-token" {
t.Errorf("unexpected authorization header %q", got)
}
query := r.URL.Query()
if query.Get("env") != "PROD" || query.Get("app") != "orders" || query.Get("namespace") != "application" || query.Get("ip") != "10.0.0.8" || query.Get("instance") != "orders-3" {
t.Errorf("unexpected query: %s", r.URL.RawQuery)
}
return &http.Response{
StatusCode: http.StatusOK,
Header: http.Header{"Content-Type": []string{"application/json"}},
Body: io.NopCloser(strings.NewReader(`{"data":{"items":{"feature.gray":"on"},"revision":12,"releaseVersion":3,"grayRuleIds":[7]}}`)),
Request: r,
}, nil
})
client, err := New(Options{BaseURL: "https://config.test", Env: "PROD", App: "orders", Token: "sdk-token", IP: "10.0.0.8", Instance: "orders-3", HTTPClient: &http.Client{Transport: transport}})
if err != nil {
t.Fatal(err)
}
if err := client.Load(context.Background(), "application"); err != nil {
t.Fatal(err)
}
if value := client.GetString("application", "feature.gray", "off"); value != "on" {
t.Fatalf("unexpected synchronized value %q", value)
}
}

View File

@@ -0,0 +1,106 @@
#!/usr/bin/env sh
set -eu
api_base="${CONFIGCENTER_API_BASE:-http://127.0.0.1:8080}"
app_code="config-smoke-$(date +%s)-$$"
app_id=""
json_data_field() {
field="$1"
python3 -c 'import json,sys; payload=json.load(sys.stdin)["data"]; print(payload[sys.argv[1]])' "$field"
}
cleanup() {
if [ -n "$app_id" ]; then
curl -fsS -X DELETE -H 'X-User: integration-smoke' "$api_base/v1/applications/$app_id" >/dev/null || true
fi
}
trap cleanup EXIT INT TERM
curl -fsS "$api_base/health/ready" >/dev/null
app_response="$(curl -fsS -X POST \
-H 'Content-Type: application/json' \
-H 'X-User: integration-smoke' \
-d "{\"code\":\"$app_code\",\"name\":\"Integration Smoke\"}" \
"$api_base/v1/applications")"
app_id="$(printf '%s' "$app_response" | json_data_field id)"
env_id="$(curl -fsS "$api_base/v1/environments" | python3 -c '
import json,sys
for item in json.load(sys.stdin)["data"]:
if item["code"] == "DEV":
print(item["id"])
break
else:
raise SystemExit("DEV environment not found")
')"
namespace_response="$(curl -fsS -X POST \
-H 'Content-Type: application/json' \
-H 'X-User: integration-smoke' \
-d "{\"appId\":$app_id,\"name\":\"application\",\"type\":\"properties\"}" \
"$api_base/v1/namespaces")"
namespace_id="$(printf '%s' "$namespace_response" | json_data_field id)"
curl -fsS -X POST \
-H 'Content-Type: application/json' \
-H 'X-User: integration-smoke' \
-d "{\"appId\":$app_id,\"nsId\":$namespace_id,\"envId\":$env_id,\"key\":\"smoke.enabled\",\"value\":\"true\"}" \
"$api_base/v1/config-items" >/dev/null
release_response="$(curl -fsS -X POST \
-H 'Content-Type: application/json' \
-H 'X-User: integration-smoke' \
-d "{\"appId\":$app_id,\"nsId\":$namespace_id,\"envId\":$env_id,\"comment\":\"integration smoke\"}" \
"$api_base/v1/publish")"
release_id="$(printf '%s' "$release_response" | json_data_field id)"
attempt=0
status="pending"
while [ "$attempt" -lt 50 ]; do
release_response="$(curl -fsS "$api_base/v1/releases/$release_id")"
status="$(printf '%s' "$release_response" | json_data_field status)"
if [ "$status" = "applied" ]; then
break
fi
if [ "$status" = "failed" ]; then
printf 'release failed: %s\n' "$release_response" >&2
exit 1
fi
attempt=$((attempt + 1))
sleep 0.2
done
if [ "$status" != "applied" ]; then
printf 'release did not become applied (last status: %s)\n' "$status" >&2
exit 1
fi
runtime_response="$(curl -fsS "$api_base/v1/config?env=DEV&app=$app_code&namespace=application")"
printf '%s' "$runtime_response" | python3 -c '
import json,sys
data=json.load(sys.stdin)["data"]
assert data["items"]["smoke.enabled"] == "true", data
assert data["releaseVersion"] == 1, data
assert data["revision"] > 0, data
'
gray_response="$(curl -fsS -X POST \
-H 'Content-Type: application/json' \
-H 'X-User: integration-smoke' \
-d "{\"appId\":$app_id,\"nsId\":$namespace_id,\"envId\":$env_id,\"ruleType\":\"instance\",\"ruleValue\":{\"instances\":[\"smoke-1\"]},\"overrides\":{\"smoke.enabled\":\"gray\"},\"enabled\":true,\"priority\":10,\"desc\":\"smoke rule\"}" \
"$api_base/v1/gray-rules")"
gray_id="$(printf '%s' "$gray_response" | json_data_field id)"
gray_runtime_response="$(curl -fsS "$api_base/v1/config?env=DEV&app=$app_code&namespace=application&instance=smoke-1")"
printf '%s' "$gray_runtime_response" | python3 -c '
import json,sys
data=json.load(sys.stdin)["data"]
assert data["items"]["smoke.enabled"] == "gray", data
assert len(data["grayRuleIds"]) == 1, data
'
curl -fsS "$api_base/metrics" | grep -q '^configcenter_gray_rule_matches_total 1$'
printf 'integration smoke passed: release=%s gray_rule=%s status=%s\n' "$release_id" "$gray_id" "$status"

View File

@@ -0,0 +1,3 @@
from .client import ConfigClient
__all__ = ["ConfigClient"]

View File

@@ -0,0 +1,129 @@
"""Thread-safe Config Center client using only Python's standard library."""
from __future__ import annotations
import json
import os
import tempfile
import threading
import time
import urllib.parse
import urllib.request
from pathlib import Path
from typing import Any
class ConfigClient:
def __init__(
self,
base_url: str,
env: str,
app: str,
cache_file: str | None = None,
*,
token: str | None = None,
ip: str | None = None,
instance: str | None = None,
):
if not base_url or not env or not app:
raise ValueError("base_url, env and app are required")
self._base_url = base_url.rstrip("/")
self._env = env
self._app = app
self._token = token or ""
self._ip = ip or ""
self._instance = instance or ""
self._cache_file = Path(cache_file) if cache_file else None
self._cache: dict[str, dict[str, str]] = {}
self._revisions: dict[str, int] = {}
self._lock = threading.RLock()
self._load_disk_cache()
def get(self, namespace: str, key: str, default: Any = None) -> Any:
with self._lock:
return self._cache.get(namespace, {}).get(key, default)
def snapshot(self, namespace: str) -> dict[str, str]:
with self._lock:
return dict(self._cache.get(namespace, {}))
def load(self, namespace: str, timeout: float = 10.0) -> None:
with urllib.request.urlopen(self._request("/v1/config", namespace), timeout=timeout) as response:
payload = json.load(response)["data"]
self._apply(namespace, payload.get("items", {}), int(payload.get("revision", 0)))
def start_background_watch(self, namespace: str) -> threading.Thread:
thread = threading.Thread(target=self.watch_and_sync, args=(namespace,), daemon=True)
thread.start()
return thread
def watch_and_sync(self, namespace: str) -> None:
backoff = 1.0
while True:
try:
request = self._request("/v1/watch", namespace, {"Accept": "text/event-stream"})
with urllib.request.urlopen(request, timeout=None) as response:
backoff = 1.0
data_lines: list[str] = []
for raw_line in response:
line = raw_line.decode("utf-8").rstrip("\r\n")
if not line:
if data_lines:
event = json.loads("\n".join(data_lines))
self._apply(namespace, event.get("items", {}), int(event.get("revision", 0)))
data_lines.clear()
continue
if line.startswith("data:"):
data_lines.append(line[5:].strip())
except Exception:
time.sleep(backoff)
backoff = min(backoff * 2, 30.0)
def _url(self, path: str, namespace: str) -> str:
values = {"env": self._env, "app": self._app, "namespace": namespace}
if self._ip:
values["ip"] = self._ip
if self._instance:
values["instance"] = self._instance
query = urllib.parse.urlencode(values)
return f"{self._base_url}{path}?{query}"
def _request(
self, path: str, namespace: str, headers: dict[str, str] | None = None
) -> urllib.request.Request:
request_headers = dict(headers or {})
if self._token:
request_headers["Authorization"] = f"Bearer {self._token}"
return urllib.request.Request(self._url(path, namespace), headers=request_headers)
def _apply(self, namespace: str, items: dict[str, str], revision: int) -> None:
with self._lock:
if revision and revision < self._revisions.get(namespace, 0):
return
self._cache[namespace] = dict(items)
self._revisions[namespace] = revision
self._save_disk_cache()
def _load_disk_cache(self) -> None:
if not self._cache_file or not self._cache_file.exists():
return
payload = json.loads(self._cache_file.read_text(encoding="utf-8"))
self._cache = payload.get("namespaces", {})
self._revisions = payload.get("revisions", {})
def _save_disk_cache(self) -> None:
if not self._cache_file:
return
self._cache_file.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
payload = {"namespaces": self._cache, "revisions": self._revisions}
descriptor, temporary = tempfile.mkstemp(prefix=".configcenter-cache-", dir=self._cache_file.parent)
try:
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(payload, handle, ensure_ascii=False, indent=2)
handle.flush()
os.fsync(handle.fileno())
os.chmod(temporary, 0o600)
os.replace(temporary, self._cache_file)
finally:
if os.path.exists(temporary):
os.unlink(temporary)

13
sdk/python/pyproject.toml Normal file
View File

@@ -0,0 +1,13 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "configcenter-client"
version = "0.1.0"
description = "Config Center Python client with SSE watch and local cache fallback"
requires-python = ">=3.10"
[tool.setuptools.packages.find]
where = ["."]
include = ["configcenter*"]

12
web/Dockerfile Normal file
View File

@@ -0,0 +1,12 @@
FROM node:24-alpine AS build
WORKDIR /src
COPY web/package.json web/package-lock.json ./web/
RUN cd web && npm ci
COPY ConfigCenter.jsx ./ConfigCenter.jsx
COPY web ./web
RUN cd web && npm run build
FROM nginx:1.27-alpine
COPY web/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/web/dist /usr/share/nginx/html
EXPOSE 80

13
web/index.html Normal file
View File

@@ -0,0 +1,13 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="theme-color" content="#0f172a" />
<title>Config Center</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.jsx"></script>
</body>
</html>

24
web/nginx.conf Normal file
View File

@@ -0,0 +1,24 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
location /v1/ {
proxy_pass http://server:8080;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_read_timeout 1h;
}
location /health/ {
proxy_pass http://server:8080;
}
location / {
try_files $uri $uri/ /index.html;
}
}

1848
web/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

22
web/package.json Normal file
View File

@@ -0,0 +1,22 @@
{
"name": "configcenter-web",
"private": true,
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"dependencies": {
"@vitejs/plugin-react": "latest",
"autoprefixer": "latest",
"lucide-react": "latest",
"postcss": "latest",
"react": "latest",
"react-dom": "latest",
"tailwindcss": "^3.4.17",
"vite": "latest"
},
"devDependencies": {}
}

6
web/postcss.config.js Normal file
View File

@@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};

196
web/src/api.js Normal file
View File

@@ -0,0 +1,196 @@
const API_ROOT = (import.meta.env.VITE_API_BASE_URL || '').replace(/\/$/, '');
const TOKEN_KEY = 'configcenter.accessToken';
let unauthorizedHandler = null;
function accessToken() {
return window.sessionStorage.getItem(TOKEN_KEY) || '';
}
class ApiError extends Error {
constructor(message, status, code) {
super(message);
this.name = 'ApiError';
this.status = status;
this.code = code;
}
}
function normalizeId(value) {
return value == null ? value : String(value);
}
function localTime(value, dateOnly = false) {
if (!value) return '';
const date = new Date(value);
if (Number.isNaN(date.getTime())) return value;
if (dateOnly) return date.toLocaleDateString('zh-CN');
return date.toLocaleString('zh-CN', { hour12: false });
}
function app(item) {
return { ...item, id: normalizeId(item.id), createdAt: localTime(item.createdAt, true), updatedAt: localTime(item.updatedAt) };
}
function env(item) {
return { ...item, id: normalizeId(item.id) };
}
function namespace(item) {
return { ...item, id: normalizeId(item.id), appId: normalizeId(item.appId) };
}
function config(item) {
return {
...item,
id: normalizeId(item.id),
appId: normalizeId(item.appId),
nsId: normalizeId(item.nsId),
envId: normalizeId(item.envId),
updatedAt: localTime(item.updatedAt),
};
}
function release(item) {
return {
...item,
id: normalizeId(item.id),
appId: normalizeId(item.appId),
nsId: normalizeId(item.nsId),
envId: normalizeId(item.envId),
time: localTime(item.time),
};
}
function grayRule(item) {
return {
...item,
id: normalizeId(item.id),
appId: normalizeId(item.appId),
nsId: normalizeId(item.nsId),
envId: normalizeId(item.envId),
createdAt: localTime(item.createdAt),
};
}
async function request(path, options = {}) {
const token = accessToken();
const response = await fetch(`${API_ROOT}${path}`, {
...options,
headers: {
'Content-Type': 'application/json',
'X-User': 'admin',
...(token ? { Authorization: `Bearer ${token}` } : {}),
...options.headers,
},
});
if (!response.ok) {
let message = `请求失败 (${response.status})`;
let code = 'http_error';
try {
const payload = await response.json();
message = payload.error?.message || message;
code = payload.error?.code || code;
} catch {
// Keep the HTTP status based fallback.
}
const error = new ApiError(message, response.status, code);
if (response.status === 401 && path !== '/v1/auth/login') {
window.sessionStorage.removeItem(TOKEN_KEY);
unauthorizedHandler?.();
}
throw error;
}
if (response.status === 204) return null;
const payload = await response.json();
return payload.data ?? payload;
}
function json(method, body) {
return { method, body: JSON.stringify(body) };
}
function scopeQuery(appId, nsId, envId) {
return new URLSearchParams({ appId, nsId, envId }).toString();
}
export const api = {
setToken(token) {
if (token) window.sessionStorage.setItem(TOKEN_KEY, token);
else window.sessionStorage.removeItem(TOKEN_KEY);
},
onUnauthorized(handler) {
unauthorizedHandler = handler;
return () => { if (unauthorizedHandler === handler) unauthorizedHandler = null; };
},
hasToken: () => Boolean(accessToken()),
login: (username, password) => request('/v1/auth/login', json('POST', { username, password })),
me: () => request('/v1/me'),
async bootstrap() {
const [apps, environments, namespaces, identity] = await Promise.all([
request('/v1/applications'),
request('/v1/environments'),
request('/v1/namespaces'),
request('/v1/me'),
]);
return {
apps: apps.map(app),
environments: environments.map(env),
namespaces: namespaces.map(namespace),
identity,
};
},
createApplication: (input) => request('/v1/applications', json('POST', input)).then(app),
updateApplication: (id, input) => request(`/v1/applications/${id}`, json('PUT', input)).then(app),
deleteApplication: (id) => request(`/v1/applications/${id}`, { method: 'DELETE' }),
createEnvironment: (input) => request('/v1/environments', json('POST', input)).then(env),
updateEnvironment: (id, input) => request(`/v1/environments/${id}`, json('PUT', input)).then(env),
deleteEnvironment: (id) => request(`/v1/environments/${id}`, { method: 'DELETE' }),
createNamespace: (input) => request('/v1/namespaces', json('POST', { ...input, appId: Number(input.appId) })).then(namespace),
updateNamespace: (id, input) => request(`/v1/namespaces/${id}`, json('PUT', input)).then(namespace),
deleteNamespace: (id) => request(`/v1/namespaces/${id}`, { method: 'DELETE' }),
async listConfigItems(appId, nsId, envId) {
const data = await request(`/v1/config-items?${scopeQuery(appId, nsId, envId)}`);
return data.map(config);
},
createConfigItem: (input) => request('/v1/config-items', json('POST', {
...input,
appId: Number(input.appId),
nsId: Number(input.nsId),
envId: Number(input.envId),
})).then(config),
updateConfigItem: (id, input) => request(`/v1/config-items/${id}`, json('PUT', input)).then(config),
deleteConfigItem: (id) => request(`/v1/config-items/${id}`, { method: 'DELETE' }).then(config),
restoreConfigItem: (id) => request(`/v1/config-items/${id}/restore`, { method: 'POST' }).then(config),
async listReleases(appId, nsId, envId) {
const data = await request(`/v1/releases?${scopeQuery(appId, nsId, envId)}`);
return data.map(release);
},
publish: (input) => request('/v1/publish', json('POST', {
appId: Number(input.appId), nsId: Number(input.nsId), envId: Number(input.envId), comment: input.comment,
})).then(release),
rollback: (input) => request('/v1/rollback', json('POST', {
appId: Number(input.appId), nsId: Number(input.nsId), envId: Number(input.envId), targetVersion: input.targetVersion,
})).then(release),
getRelease: (id) => request(`/v1/releases/${id}`).then(release),
async listGrayRules(appId, nsId, envId) {
const data = await request(`/v1/gray-rules?${scopeQuery(appId, nsId, envId)}`);
return data.map(grayRule);
},
createGrayRule: (input) => request('/v1/gray-rules', json('POST', {
...input, appId: Number(input.appId), nsId: Number(input.nsId), envId: Number(input.envId),
})).then(grayRule),
updateGrayRule: (id, input) => request(`/v1/gray-rules/${id}`, json('PUT', input)).then(grayRule),
deleteGrayRule: (id) => request(`/v1/gray-rules/${id}`, { method: 'DELETE' }),
listUsers: () => request('/v1/users'),
createUser: (input) => request('/v1/users', json('POST', input)),
listUserRoles: (id) => request(`/v1/users/${id}/roles`),
setUserRole: (userId, appId, role) => request(`/v1/users/${userId}/roles/${appId}`, json('PUT', { role })),
deleteUserRole: (userId, appId) => request(`/v1/users/${userId}/roles/${appId}`, { method: 'DELETE' }),
};

22
web/src/index.css Normal file
View File

@@ -0,0 +1,22 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
html,
body,
#root {
height: 100%;
margin: 0;
}
body {
min-width: 720px;
background: #f8fafc;
}
button,
input,
select,
textarea {
font: inherit;
}

10
web/src/main.jsx Normal file
View File

@@ -0,0 +1,10 @@
import React from 'react';
import ReactDOM from 'react-dom/client';
import ConfigCenter from '../../ConfigCenter.jsx';
import './index.css';
ReactDOM.createRoot(document.getElementById('root')).render(
<React.StrictMode>
<ConfigCenter />
</React.StrictMode>,
);

6
web/tailwind.config.js Normal file
View File

@@ -0,0 +1,6 @@
/** @type {import('tailwindcss').Config} */
export default {
content: ['./index.html', './src/**/*.{js,jsx}', '../ConfigCenter.jsx'],
theme: { extend: {} },
plugins: [],
};

21
web/vite.config.js Normal file
View File

@@ -0,0 +1,21 @@
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { fileURLToPath, URL } from 'node:url';
export default defineConfig({
plugins: [react()],
resolve: {
alias: {
react: fileURLToPath(new URL('./node_modules/react', import.meta.url)),
'lucide-react': fileURLToPath(new URL('./node_modules/lucide-react', import.meta.url)),
},
},
server: {
port: 5173,
proxy: {
'/v1': 'http://localhost:8080',
'/health': 'http://localhost:8080',
},
fs: { allow: ['..'] },
},
});