Project: vilbert/skills Commit: 8a4abdcac7413aa51e91901feda4d59c7f35a8e2 Message: Split backend-dev into focused documentation files with SOLID principles Major changes: - Split mon --- AGENTS.md --- @@ -6,6 +6,7 @@ OpenCode skill collection for reusable project templates and specialized develop - Each subdirectory = one skill domain - `SKILL.md` in each directory contains comprehensive usage instructions +- Detailed documentation split into focused files ## Adding New Skills @@ -17,10 +18,24 @@ OpenCode skill collection for reusable project templates and specialized develop | Directory | Description | |-----------|-------------| | `backend-dev/` | Go hexagonal architecture template with Keycloak JWT auth | +| `orchestrator/` | End-to-end development orchestration | ## backend-dev Skill -Go backend service template using hexagonal (ports & adapters) architecture. +Go backend service template using hexagonal (ports & adapters) architecture with 12-factor compliance. + +### Documentation Structure + +| File | Contents | +|------|----------| +| [SKILL.md](backend-dev/SKILL.md) | Entry point, quick start, commands | +| [ARCHITECTURE.md](backend-dev/ARCHITECTURE.md) | Hexagonal patterns, DDD, SOLID principles | +| [12-FACTOR.md](backend-dev/12-FACTOR.md) | 12-factor methodology | +| [CONFIGURATION.md](backend-dev/CONFIGURATION.md) | OpenBao secrets, env loading | +| [AUTHENTICATION.md](backend-dev/AUTHENTICATION.md) | Keycloak JWT integration | +| [DATABASE.md](backend-dev/DATABASE.md) | PostgreSQL, migrations, caching | +| [DEPLOYMENT.md](backend-dev/DEPLOYMENT.md) | Docker, CI/CD, admin CLI | +| [CODING-STANDARDS.md](backend-dev/CODING-STANDARDS.md) | REST, SOLID, error handling, testing | ### Tech Stack - Gin web framework @@ -28,11 +43,13 @@ Go backend service template using hexagonal (ports & adapters) architecture. - Prometheus metrics - OpenTelemetry tracing - Keycloak JWT authentication (keyfunc/v3) +- OpenBao v2 for secrets (token-based, KV v2) ### Key Environment Variables ```env KEYCLOAK_JWKS_URL=https://auth.pharos.id/realms/production/protocol/openid-connect/certs KEYCLOAK_CLIENT_ID=exodus +PERSISTENCE_DSN=postgres://user:pass@localhost:5432/dbname?sslmode=disable ``` ### Keycloak JWT Claims Mapping @@ -43,16 +60,23 @@ KEYCLOAK_CLIENT_ID=exodus | `email` | Email (non-unique) | | `resource_access.{client_id}.roles` | Authorization roles | -### Authentication Flow -1. Backend acts as OAuth2 Resource Server (validates Bearer tokens) -2. JWT validated locally via Keycloak JWKS (no client secret) -3. User auto-created/updated on each authentication -4. Roles extracted from `resource_access.{client_id}.roles` +### Design Principles +- **SOLID**: SRP (one usecase per operation), DIP (depend on ports) +- **Hexagonal**: Core domain has no external dependencies +- **12-Factor**: Environment-based config, stateless processes +- **Error Handling**: NEVER skip error checking ### Database - `users` table with `nip` as primary key - `cache.entries` unlogged table for caching +### Commands +```bash +./messaging-be # Run server (default) +./messaging-be admin migrate up # Run migrations +./messaging-be admin user list # List users +``` + ## Skill Documentation Conventions - Use Go-style code blocks with line numbers for large sections --- backend-dev/12-FACTOR.md --- @@ -0,0 +1,395 @@ +# 12-Factor Compliance + +- [Table of Contents](#table-of-contents) +- [Overview](#overview) +- [Factor I: Codebase](#factor-i-codebase) +- [Factor II: Dependencies](#factor-ii-dependencies) +- [Factor III: Config](#factor-iii-config) +- [Factor IV: Backing Services](#factor-iv-backing-services) +- [Factor V: Build, Release, Run](#factor-v-build-release-run) +- [Factor VI: Processes](#factor-vi-processes) +- [Factor VII: Port Binding](#factor-vii-port-binding) +- [Factor VIII: Concurrency](#factor-viii-concurrency) +- [Factor IX: Disposability](#factor-ix-disposability) +- [Factor X: Dev/Prod Parity](#factor-x-devprod-parity) +- [Factor XI: Logs](#factor-xi-logs) +- [Factor XII: Admin Processes](#factor-xii-admin-processes) + +--- + +## Table of Contents + +1. [Codebase](#factor-i-codebase) - One codebase per app +2. [Dependencies](#factor-ii-dependencies) - Explicit declaration +3. [Config](#factor-iii-config) - Environment-based configuration +4. [Backing Services](#factor-iv-backing-services) - Treat as attached resources +5. [Build, Release, Run](#factor-v-build-release-run) - Strict separation +6. [Processes](#factor-vi-processes) - Stateless and share-nothing +7. [Port Binding](#factor-vii-port-binding) - Self-contained service +8. [Concurrency](#factor-viii-concurrency) - Scale via process model +9. [Disposability](#factor-ix-disposability) - Fast startup, graceful shutdown +10. [Dev/Prod Parity](#factor-x-devprod-parity) - Keep environments similar +11. [Logs](#factor-xi-logs) - Treat logs as event streams +12. [Admin Processes](#factor-xii-admin-processes) - One-off admin tasks + +--- + +## Overview + +This template follows [12-Factor App](https://12factor.net/) methodology for production-ready deployments. + +--- + +## Factor I: Codebase + +One codebase tracked in version control, multiple deploys. + +**Practices:** +- Git repository with main branch +- Environment-specific configs via environment variables +- Same codebase deploys to local, staging, production + +**Git Workflow:** +```bash +git clone https://gitlab.example.com/backend.git +cd backend +git checkout -b feature/new-feature +# ... development ... +git commit -m "feat: add new feature" +git push origin feature/new-feature +# Create merge request to main +``` + +--- + +## Factor II: Dependencies + +Explicitly declare and isolate dependencies. + +**go.mod:** +```go +require ( + github.com/gin-gonic/gin v1.12.0 + github.com/jackc/pgx/v5 v5.5.0 + github.com/golang-migrate/migrate/v4 v4.19.0 +) +``` + +**Vendor directory (optional isolation):** +```bash +go mod vendor +``` + +**Key Principle:** Never rely on system-wide packages. + +--- + +## Factor III: Config + +Store config in environment variables. + +**Configuration Loading:** +```go +func Load(ctx context.Context) (*Config, error) { + env := os.Getenv("APP_ENV") + if env == "" { + env = "local" + } + + switch env { + case "local": + dotenv.Load() + return loadFromEnv() + default: + return loadFromOpenBao(ctx) + } +} +``` + +**Environment Variables:** +```env +APP_ENV=production +PERSISTENCE_DSN=postgres://user:pass@host:5432/dbname +KEYCLOAK_JWKS_URL=https://auth.example.com/realms/production/certs +``` + +See [CONFIGURATION.md](CONFIGURATION.md) for OpenBao integration. + +--- + +## Factor IV: Backing Services + +Treat backing services as attached resources. + +**Connection via config:** +```go +dsn := cfg.Persistence.DSN // PostgreSQL +jwksURL := cfg.Keycloak.JWKSURL // Keycloak +tempoEndpoint := cfg.Tempo.Endpoint // OpenTelemetry +``` + +**Health checks:** +```go +r.GET("/healthz", func(c *gin.Context) { + checks := map[string]bool{ + "database": pingDB(persistencePool), + "cache": pingCache(cachePool), + } + + allHealthy := true + for _, ok := range checks { + if !ok { + allHealthy = false + break + } + } + + if allHealthy { + c.JSON(200, gin.H{"status": "ok"}) + } else { + c.JSON(503, gin.H{"status": "degraded", "checks": checks}) + } +}) +``` + +--- + +## Factor V: Build, Release, Run + +Strict separation of build, release, and run stages. + +**GitLab CI Pipeline:** +```yaml +stages: + - build + - test + - release + - deploy + +build: + stage: build + script: + - go build -ldflags "-X main.version=$CI_COMMIT_SHA" -o bin/server ./cmd/server + +test: + stage: test + services: + - postgres:15 + script: + - go test -race -coverprofile=coverage.out ./... + coverage: '/total:\s+\(statements\)\s+(\d+\.\d+)%/' + +release: + stage: release + script: + - docker build -t $IMAGE_NAME:$CI_COMMIT_SHA . + - docker push $IMAGE_NAME:$CI_COMMIT_SHA + rules: + - main + +deploy: + stage: deploy + script: + - kubectl set image deployment/server server=$IMAGE_NAME:$CI_COMMIT_SHA + environment: + name: production + rules: + - main + when: manual +``` + +--- + +## Factor VI: Processes + +Stateless processes with no shared state. + +**Share-nothing architecture:** +```go +// Good: User sessions in PostgreSQL, not memory +// Cache in PostgreSQL unlogged tables, not process memory +// File uploads to object storage, not local disk + +type userService struct { + userRepo outbound.UserRepository // PostgreSQL + cache outbound.CacheRepository // Cache layer +} +``` + +**Bad:** Storing user data in global variables. + +--- + +## Factor VII: Port Binding + +Self-contained HTTP service. + +```go +// cmd/server/run.go +func Run(args []string) { + cfg, err := config.Load(context.Background()) + if err != nil { + fmt.Fprintf(os.Stderr, "Config error: %v\n", err) + os.Exit(1) + } + + port := cfg.App.Port + if !strings.HasPrefix(port, ":") { + port = ":" + port + } + + log.Printf("Starting server on %s", port) + if err := http.ListenAndServe(port, r.Handler()); err != nil { + log.Fatal(err) + } +} +``` + +--- + +## Factor VIII: Concurrency + +Scale via process model. + +**Worker pool pattern:** +```go +type WorkerPool struct { + workers int + jobs chan Job + wg sync.WaitGroup +} + +func (wp *WorkerPool) Start(ctx context.Context) { + for i := 0; i < wp.workers; i++ { + wp.wg.Add(1) + go wp.worker(ctx, i) + } +} + +func (wp *WorkerPool) Submit(job Job) { + wp.jobs <- job +} + +func (wp *WorkerPool) worker(ctx context.Context, id int) { + defer wp.wg.Done() + for { + select { + case <-ctx.Done(): + return + case job := <-wp.jobs: + job.Execute() + } + } +} +``` + +--- + +## Factor IX: Disposability + +Fast startup, graceful shutdown. + +**Graceful shutdown:** +```go +func gracefulShutdown(sig os.Signal, server *http.Server) { + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + + log.Printf("Received %s, shutting down gracefully...", sig) + if err := server.Shutdown(ctx); err != nil { + log.Fatalf("Server shutdown failed: %v", err) + } +} + +// Signal handling +sig := make(chan os.Signal, 1) +signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM) +go func() { + <-sig + gracefulShutdown(<-sig, server) +}() +``` + +--- + +## Factor X: Dev/Prod Parity + +Use Docker Compose for local development. + +**docker-compose.yml:** +```yaml +version: '3.8' + +services: + app: + build: . + env_file: .env.local + ports: + - "8080:8080" + environment: + PERSISTENCE_DSN: ${PERSISTENCE_DSN} + KEYCLOAK_JWKS_URL: ${KEYCLOAK_JWKS_URL} + +# Dev provides their own PostgreSQL and Keycloak +# Use same PostgreSQL version locally as in production +``` + +**Key Principle:** Same PostgreSQL version in all environments. + +--- + +## Factor XI: Logs + +Structured logging to stdout only. + +```go +// Use slog - Go 1.21+ standard library +log := slog.New(slog.NewJSONHandler(os.Stdout, nil)) + +log.Info("server started", + "port", 8080, + "env", "production", + "version", buildinfo.Version, +) +``` + +**Key Principles:** +- Log to stdout/stderr only +- No log files in container +- Aggregate via external services (Loki, CloudWatch) + +--- + +## Factor XII: Admin Processes + +One-off admin tasks as CLI commands. + +See [DEPLOYMENT.md](DEPLOYMENT.md#admin-cli) for detailed admin commands. + +**Commands:** +```bash +./messaging-be admin migrate up # Run migrations +./messaging-be admin migrate down 1 # Rollback +./messaging-be admin user list # List users +./messaging-be admin cache clear # Clear cache +./messaging-be admin health # Health check +``` + +--- + +## Summary Table + +| Factor | Practice | +|--------|----------| +| I | Git workflow, one repo per app | +| II | go.mod with explicit versions | +| III | Environment variables, no hardcoded config | +| IV | PostgreSQL, Keycloak as resources | +| V | CI/CD pipeline with stages | +| VI | Stateless, share-nothing | +| VII | Self-contained HTTP service | +| VIII | Worker pools for concurrency | +| IX | Graceful shutdown, crash-only | +| X | Docker Compose for dev parity | +| XI | Structured logs to stdout | +| XII | CLI admin commands | --- backend-dev/ARCHITECTURE.md --- @@ -0,0 +1,601 @@ +# Architecture + +- [Table of Contents](#table-of-contents) +- [Hexagonal Architecture](#hexagonal-architecture) +- [SOLID Principles](#solid-principles) +- [Domain Layer](#domain-layer) +- [Ports](#ports) +- [Services/Usecases](#servicesusecases) +- [Adapters](#adapters) +- [Provider Pattern](#provider-pattern) +- [Directory Structure](#directory-structure) + +--- + +## Table of Contents + +1. [Hexagonal Architecture](#hexagonal-architecture) +2. [SOLID Principles](#solid-principles) +3. [Domain Layer](#domain-layer) +4. [Ports](#ports) +5. [Services/Usecases](#servicesusecases) +6. [Adapters](#adapters) +7. [Provider Pattern](#provider-pattern) +8. [Directory Structure](#directory-structure) + +--- + +## Hexagonal Architecture + +``` + ┌─────────────────────────────────────┐ + │ Adapters │ + │ ┌─────────────┐ ┌──────────────┐ │ + │ │ Inbound │ │ Outbound │ │ + │ │ (Driving) │ │ (Driven) │ │ + │ │ │ │ │ │ + │ │ HTTP/REST │ │ PostgreSQL │ │ + │ │ gRPC │ │ Cache │ │ + │ │ CLI │ │ Keycloak │ │ + │ └──────┬──────┘ └──────┬───────┘ │ + └─────────┼─────────────────┼──────────┘ + │ │ + ▼ ▼ + ┌─────────────────────────────────────┐ + │ Ports │ + │ ┌─────────────┐ ┌──────────────┐ │ + │ │ Inbound │ │ Outbound │ │ + │ │ (Usecase) │ │ (Repository) │ │ + │ └─────────────┘ └──────────────┘ │ + └─────────────────┬───────────────────┘ + │ + ▼ + ┌─────────────────────────────────────┐ + │ Core/Domain │ + │ │ + │ ┌───────────┐ ┌───────────────┐ │ + │ │ Entity │ │ Value Object │ │ + │ └───────────┘ └───────────────┘ │ + │ │ + │ ┌─────────────────────────────────┐ │ + │ │ Business Logic │ │ + │ │ (Usecase Implementation) │ │ + │ └─────────────────────────────────┘ │ + └─────────────────────────────────────┘ +``` + +### Key Principles + +- **Independence**: Core domain has no external dependencies +- **Ports**: Interfaces define boundaries +- **Adapters**: Implementations outside the core +- **Testability**: Core can be tested without external dependencies + +--- + +## SOLID Principles + +### Single Responsibility (SRP) + +Each component has one reason to change. + +**Good:** +```go +type UserService interface { + GetByNIP(ctx context.Context, nip string) (*User, error) + Create(ctx context.Context, user *User) error +} +``` + +**Bad:** +```go +type AdminService interface { + GetUser() + SendEmail() // Wrong: Not user-related + ProcessPayment() // Wrong: Different domain + GenerateReport() // Wrong: Reporting concern +} +``` + +### Dependency Inversion (DIP) + +High-level modules depend on abstractions, not concrete implementations. + +**Good:** +```go +type userService struct { + userRepo outbound.UserRepository +} + +func NewUserService(repo outbound.UserRepository) UserService { + return &userService{userRepo: repo} +} +``` + +**Bad:** +```go +type userService struct { + db *sql.DB +} + +func NewUserService() *userService { + return &userService{ + db: connectPostgres(), + } +} +``` + +**Test vs Production (LSP in action):** +```go +mockRepo := &mockUserRepository{} +pgRepo := persistence.NewUserRepository(pool) + +svc := usecase.NewUserService(mockRepo) // Works +svc := usecase.NewUserService(pgRepo) // Works - both satisfy interface +``` + +--- + +## Domain Layer + +### Entities + +Pure business entities with no external dependencies. + +```go +package domain + +import "time" + +type User struct { + NIP string `db:"nip" json:"nip"` + Name string `db:"name" json:"name"` + Email string `db:"email" json:"email"` + IsActive bool `db:"is_active" json:"is_active"` + LastLoginAt time.Time `db:"last_login_at" json:"last_login_at"` + CreatedAt time.Time `db:"created_at" json:"created_at"` + UpdatedAt time.Time `db:"updated_at" json:"updated_at"` +} +``` + +### Value Objects + +```go +package valueobject + +type NIP string + +func (n NIP) Validate() error { + if len(n) == 0 { + return errors.New("nip is required") + } + if len(n) > 20 { + return errors.New("nip must not exceed 20 characters") + } + return nil +} + +type Email string + +func (e Email) Validate() error { + if len(e) == 0 { + return nil + } + if !strings.Contains(string(e), "@") { + return errors.New("invalid email format") + } + return nil +} +``` + +### Domain Events + +```go +package event + +type EventType string + +const ( + UserCreated EventType = "user.created" + UserUpdated EventType = "user.updated" + UserDeactivated EventType = "user.deactivated" +) + +type DomainEvent interface { + EventType() EventType + Timestamp() time.Time +} + +type UserCreatedEvent struct { + NIP string + Name string + Email string + timestamp time.Time +} + +func (e *UserCreatedEvent) EventType() EventType { + return UserCreated +} + +func (e *UserCreatedEvent) Timestamp() time.Time { + return e.timestamp +} +``` + +--- + +## Ports + +### Inbound Ports (Usecase Interfaces) + +```go +package inbound + +import ( + "context" + "myapp/internal/core/domain" +) + +type UserService interface { + GetByNIP(ctx context.Context, nip string) (*domain.User, error) + Create(ctx context.Context, user *domain.User) error + List(ctx context.Context) ([]domain.User, error) +} + +type AuthService interface { + Authenticate(ctx context.Context, token string) (*domain.User, error) + HasRole(user *domain.User, role string) bool +} +``` + +### Outbound Ports (Repository Interfaces) + +```go +package outbound + +import ( + "context" + "myapp/internal/core/domain" +) + +type UserRepository interface { + GetByNIP(ctx context.Context, nip string) (*domain.User, error) + Create(ctx context.Context, user *domain.User) error + Update(ctx context.Context, user *domain.User) error + List(ctx context.Context) ([]domain.User, error) +} + +type CacheRepository interface { + Get(ctx context.Context, key string) ([]byte, error) + Set(ctx context.Context, key string, value []byte, ttl time.Duration) error + Delete(ctx context.Context, key string) error + Exists(ctx context.Context, key string) (bool, error) + Clear(ctx context.Context) error +} + +type EventPublisher interface { + Publish(ctx context.Context, event event.DomainEvent) error +} +``` + +--- + +## Services/Usecases + +Implement inbound ports, depend on outbound ports. + +```go +package usecase + +import ( + "context" + "encoding/json" + "time" + + "myapp/internal/core/domain" + "myapp/internal/core/port/inbound" + "myapp/internal/core/port/outbound" + + "github.com/pkg/errors" + "go.opentelemetry.io/otel" + "go.opentelemetry.io/otel/attribute" +) + +var tracer = otel.Tracer("usecase") + +type userService struct { + userRepo outbound.UserRepository + cache outbound.CacheRepository +} + +func NewUserService( + userRepo outbound.UserRepository, + cache outbound.CacheRepository, +) inbound.UserService { + return &userService{ + userRepo: userRepo, + cache: cache, + } +} + +func (s *userService) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { + ctx, span := tracer.Start(ctx, "UserService.GetByNIP", + attribute.String("user.nip", nip), + ) + defer span.End() + + cacheKey := "user:" + nip + cached, err := s.cache.Get(ctx, cacheKey) + if err == nil && cached != nil { + span.AddEvent("cache_hit") + var user domain.User + if err := json.Unmarshal(cached, &user); err == nil { + return &user, nil + } + } + + span.AddEvent("cache_miss") + user, err := s.userRepo.GetByNIP(ctx, nip) + if err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to get user") + } + + if user != nil { + if data, err := json.Marshal(user); err == nil { + _ = s.cache.Set(ctx, cacheKey, data, 5*time.Minute) + } + } + + return user, nil +} +``` + +--- + +## Adapters + +### Inbound Adapter (HTTP Handler) + +```go +package v1 + +import ( + "net/http" + + "myapp/internal/core/port/inbound" + "myapp/pkg/response" + + "github.com/gin-gonic/gin" + "github.com/pkg/errors" +) + +type UserHandler struct { + svc inbound.UserService +} + +func NewUserHandler(svc inbound.UserService) *UserHandler { + return &UserHandler{svc: svc} +} + +func (h *UserHandler) GetByNIP(c *gin.Context) { + resp := response.Response{} + defer resp.Render(c) + + nip := c.Param("nip") + + user, err := h.svc.GetByNIP(c.Request.Context(), nip) + if err != nil { + resp.SetError(err, http.StatusInternalServerError) + return + } + if user == nil { + resp.SetError(errors.New("user not found"), http.StatusNotFound) + return + } + + resp.Data = user + resp.StatusCode = http.StatusOK +} + +func (h *UserHandler) RegisterRoutes(r *gin.RouterGroup) { + users := r.Group("/users") + { + users.GET("/:nip", h.GetByNIP) + users.POST("/", h.Create) + users.GET("/", h.List) + } +} +``` + +### Outbound Adapter (PostgreSQL Repository) + +```go +package persistence + +import ( + "context" + "encoding/json" + + "myapp/internal/core/domain" + "myapp/internal/core/port/outbound" + + "github.com/jackc/pgx/v5" + "github.com/pkg/errors" + "go.opentelemetry.io/otel/attribute" +) + +type userRepository struct { + pool *Pool +} + +func NewUserRepository(pool *Pool) outbound.UserRepository { + return &userRepository{pool: pool} +} + +func (r *userRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { + ctx, span := tracer.Start(ctx, "UserRepository.GetByNIP") + defer span.End() + + var user domain.User + err := r.pool.QueryRow(ctx, + `SELECT nip, name, email, is_active, last_login_at, created_at, updated_at + FROM users WHERE nip = $1`, + nip, + ).Scan(&user.NIP, &user.Name, &user.Email, &user.IsActive, + &user.LastLoginAt, &user.CreatedAt, &user.UpdatedAt) + + if err == pgx.ErrNoRows { + span.AddEvent("user_not_found") + return nil, nil + } + if err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to get user") + } + + span.SetAttributes(attribute.String("user.nip", user.NIP)) + return &user, nil +} + +func (r *userRepository) Create(ctx context.Context, user *domain.User) error { + ctx, span := tracer.Start(ctx, "UserRepository.Create") + defer span.End() + + err := r.pool.QueryRow(ctx, + `INSERT INTO users (nip, name, email, is_active, last_login_at) + VALUES ($1, $2, $3, $4, $5) + RETURNING created_at, updated_at`, + user.NIP, user.Name, user.Email, user.IsActive, user.LastLoginAt, + ).Scan(&user.CreatedAt, &user.UpdatedAt) + + if err != nil { + span.RecordError(err) + return errors.Wrap(err, "failed to create user") + } + + return nil +} +``` + +--- + +## Provider Pattern + +Dependency wiring via explicit Provider struct. + +```go +package provider + +import ( + "context" + "fmt" + + "myapp/config" + "myapp/internal/adapter/outbound/cache" + "myapp/internal/adapter/outbound/keycloak" + "myapp/internal/adapter/outbound/persistence" + "myapp/internal/core/port/inbound" + "myapp/internal/core/usecase" +) + +type Provider struct { + Config *config.Config + Logger *logger.Logger + Metrics *metrics.Prometheus + Tracer *tracing.TracerProvider + PersistencePool *persistence.Pool + CachePool *persistence.Pool + UserRepo outbound.UserRepository + CacheRepo outbound.CacheRepository + Keycloak *keycloak.Provider + AuthSvc inbound.AuthService + UserSvc inbound.UserService +} + +func NewProvider(ctx context.Context, cfg *config.Config) (*Provider, error) { + p := &Provider{Config: cfg} + + if err := p.initLogger(); err != nil { + return nil, fmt.Errorf("logger: %w", err) + } + if err := p.initMetrics(); err != nil { + return nil, fmt.Errorf("metrics: %w", err) + } + if err := p.initTracer(ctx); err != nil { + return nil, fmt.Errorf("tracer: %w", err) + } + if err := p.initPersistence(ctx); err != nil { + return nil, fmt.Errorf("persistence: %w", err) + } + if err := p.initCache(ctx); err != nil { + return nil, fmt.Errorf("cache: %w", err) + } + if err := p.initKeycloak(ctx); err != nil { + return nil, fmt.Errorf("keycloak: %w", err) + } + + p.UserRepo = persistence.NewUserRepository(p.PersistencePool) + p.CacheRepo = cache.NewPostgresCacheRepository(p.CachePool) + p.AuthSvc = usecase.NewAuthService(p.Keycloak, p.UserRepo) + p.UserSvc = usecase.NewUserService(p.UserRepo, p.CacheRepo) + + return p, nil +} + +func (p *Provider) Close(ctx context.Context) { + if p.PersistencePool != nil { + p.PersistencePool.Close() + } + if p.CachePool != nil { + p.CachePool.Close() + } + if p.Tracer != nil { + p.Tracer.Shutdown(ctx) + } +} +``` + +--- + +## Directory Structure + +``` +. +├── cmd/ +│ ├── server/ +│ │ └── run.go +│ └── admin/ +│ └── run.go +├── config/ +│ ├── config.go +│ ├── model.go +│ └── loader.go +├── internal/ +│ ├── core/ +│ │ ├── domain/ +│ │ │ ├── user.go +│ │ │ ├── event/ +│ │ │ └── valueobject/ +│ │ ├── port/ +│ │ │ ├── inbound/ +│ │ │ └── outbound/ +│ │ └── usecase/ +│ ├── adapter/ +│ │ ├── inbound/ +│ │ │ └── rest/v1/ +│ │ └── outbound/ +│ │ ├── persistence/ +│ │ ├── cache/ +│ │ └── keycloak/ +│ └── provider/ +├── middlewares/ +├── pkg/ +│ ├── logger/ +│ ├── metrics/ +│ ├── response/ +│ ├── tracing/ +│ └── buildinfo/ +├── db/migrations/ +└── deployments/ +``` --- backend-dev/AUTHENTICATION.md --- @@ -0,0 +1,418 @@ +# Authentication + +- [Table of Contents](#table-of-contents) +- [Overview](#overview) +- [Architecture](#architecture) +- [Keycloak Integration](#keycloak-integration) +- [JWT Claims](#jwt-claims) +- [Token Validation](#token-validation) +- [Auth Middleware](#auth-middleware) +- [Role-Based Authorization](#role-based-authorization) + +--- + +## Table of Contents + +1. [Overview](#overview) +2. [Architecture](#architecture) +3. [Keycloak Integration](#keycloak-integration) +4. [JWT Claims](#jwt-claims) +5. [Token Validation](#token-validation) +6. [Auth Middleware](#auth-middleware) +7. [Role-Based Authorization](#role-based-authorization) + +--- + +## Overview + +The backend acts as an **OAuth2 Resource Server** - it validates Bearer tokens from the UI/browser without being an OIDC client. Token validation is done locally using JWKS fetched from Keycloak. + +**Key Features:** +- No client secret needed +- Local JWT validation via JWKS +- Automatic user creation/update on login +- Roles extracted from `resource_access` + +--- + +## Architecture + +``` +┌─────────┐ Bearer Token ┌──────────────┐ JWKS ┌────────────┐ +│ UI │ ───────────────────► │ Backend │ ◄─────────── │ Keycloak │ +│ Browser │ │ Resource Srv │ (cached) │ JWKS │ +└─────────┘ └──────────────┘ └────────────┘ + │ + ▼ + ┌──────────────┐ + │ PostgreSQL │ + │ (users) │ + └──────────────┘ +``` + +--- + +## Keycloak Integration + +### Dependencies + +```go +github.com/MicahParks/keyfunc/v3 v3.0.0 +github.com/golang-jwt/jwt/v5 v5.2.0 +``` + +### Keycloak Provider + +```go +package keycloak + +import ( + "context" + "fmt" + "time" + + "github.com/MicahParks/keyfunc/v3" + "github.com/golang-jwt/jwt/v5" +) + +type Provider struct { + jwks *keyfunc.JWKS + clientID string +} + +type TokenClaims struct { + jwt.RegisteredClaims + + NIP string `json:"nip"` + Name string `json:"name"` + Email string `json:"email"` + + ResourceAccess map[string]struct { + Roles []string `json:"roles"` + } `json:"resource_access"` +} + +func NewProvider(ctx context.Context, jwksURL, clientID string) (*Provider, error) { + jwks, err := keyfunc.Get(jwksURL, keyfunc.Options{ + RefreshInterval: time.Hour, + RefreshRateLimit: time.Minute * 5, + }) + if err != nil { + return nil, fmt.Errorf("failed to fetch JWKS: %w", err) + } + + return &Provider{ + jwks: jwks, + clientID: clientID, + }, nil +} + +func (p *Provider) VerifyToken(tokenString string) (*TokenClaims, error) { + token, err := jwt.ParseWithClaims(tokenString, &TokenClaims{}, p.jwks.Keyfunc) + if err != nil { + return nil, fmt.Errorf("invalid token: %w", err) + } + + if !token.Valid { + return nil, fmt.Errorf("token is not valid") + } + + claims, ok := token.Claims.(*TokenClaims) + if !ok { + return nil, fmt.Errorf("failed to extract claims") + } + + return claims, nil +} + +func (p *Provider) ExtractRoles(claims *TokenClaims) []string { + if client, ok := claims.ResourceAccess[p.clientID]; ok { + return client.Roles + } + return nil +} +``` + +--- + +## JWT Claims + +### Expected JWT Structure + +```json +{ + "exp": 1776127146, + "iat": 1775867946, + "iss": "https://auth.pharos.id/realms/production", + "sub": "4a098117-e595-411b-b8fc-4328fc14b0ba", + "nip": "p021050", + "name": "DR. CITRA ANGGREINI SEMBIRING", + "email": "ctasbr@gmail.com", + "resource_access": { + "exodus": { + "roles": [ + "exodus.business-support", + "exodus.default", + "exodus.user" + ] + } + } +} +``` + +### Claims Mapping + +| JWT Claim | Field | Description | +|-----------|-------|-------------| +| `nip` | `NIP` | Primary key (e.g., "p021050") | +| `name` | `Name` | Full name | +| `email` | `Email` | Email address | +| `resource_access.{client_id}.roles` | `Roles` | Authorization roles | + +--- + +## Token Validation + +### Auth Service + +```go +package usecase + +import ( + "context" + "time" + + "myapp/internal/core/domain" + "myapp/internal/core/port/inbound" + "myapp/internal/core/port/outbound" + "myapp/pkg/keycloak" + + "github.com/pkg/errors" + "go.opentelemetry.io/otel" + "go.opentelemetry.io/otel/attribute" +) + +var tracer = otel.Tracer("usecase") + +type authService struct { + keycloak *keycloak.Provider + userRepo outbound.UserRepository +} + +func NewAuthService(keycloak *keycloak.Provider, userRepo outbound.UserRepository) inbound.AuthService { + return &authService{ + keycloak: keycloak, + userRepo: userRepo, + } +} + +func (s *authService) Authenticate(ctx context.Context, token string) (*domain.User, error) { + ctx, span := tracer.Start(ctx, "AuthService.Authenticate") + defer span.End() + + claims, err := s.keycloak.VerifyToken(token) + if err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "token validation failed") + } + + if claims.NIP == "" { + return nil, errors.New("nip claim is required") + } + + user := &domain.User{ + NIP: claims.NIP, + Name: claims.Name, + Email: claims.Email, + Roles: s.keycloak.ExtractRoles(claims), + } + + if err := s.syncUser(ctx, user); err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to sync user") + } + + existing, err := s.userRepo.GetByNIP(ctx, user.NIP) + if err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to get user") + } + + if existing == nil { + return nil, errors.New("user not found") + } + + if !existing.IsActive { + return nil, errors.New("user is deactivated") + } + + user.IsActive = existing.IsActive + user.LastLoginAt = existing.LastLoginAt + user.CreatedAt = existing.CreatedAt + user.UpdatedAt = existing.UpdatedAt + + span.SetAttributes(attribute.StringSlice("user.roles", user.Roles)) + + return user, nil +} + +func (s *authService) syncUser(ctx context.Context, user *domain.User) error { + existing, err := s.userRepo.GetByNIP(ctx, user.NIP) + if err != nil { + return err + } + + now := time.Now() + + if existing == nil { + user.IsActive = true + user.LastLoginAt = now + user.CreatedAt = now + user.UpdatedAt = now + return s.userRepo.Create(ctx, user) + } + + existing.Name = user.Name + existing.Email = user.Email + existing.LastLoginAt = now + existing.UpdatedAt = now + + return s.userRepo.Update(ctx, existing) +} + +func (s *authService) HasRole(user *domain.User, role string) bool { + for _, r := range user.Roles { + if r == role { + return true + } + } + return false +} + +func (s *authService) HasAnyRole(user *domain.User, roles ...string) bool { + for _, role := range roles { + if s.HasRole(user, role) { + return true + } + } + return false +} +``` + +--- + +## Auth Middleware + +```go +package middlewares + +import ( + "net/http" + "strings" + + "myapp/internal/core/domain" + "myapp/internal/core/port/inbound" + + "github.com/gin-gonic/gin" +) + +const UserContextKey = "authenticated_user" + +func AuthMiddleware(authSvc inbound.AuthService) gin.HandlerFunc { + return func(c *gin.Context) { + authHeader := c.GetHeader("Authorization") + if authHeader == "" { + c.AbortWithStatusJSON(http.StatusUnauthorized, + gin.H{"error": "missing authorization header"}) + return + } + + parts := strings.SplitN(authHeader, " ", 2) + if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") { + c.AbortWithStatusJSON(http.StatusUnauthorized, + gin.H{"error": "invalid authorization format"}) + return + } + + user, err := authSvc.Authenticate(c.Request.Context(), parts[1]) + if err != nil { + c.AbortWithStatusJSON(http.StatusUnauthorized, + gin.H{"error": err.Error()}) + return + } + + c.Set(UserContextKey, user) + c.Next() + } +} + +func RequireRole(authSvc inbound.AuthService, roles ...string) gin.HandlerFunc { + return func(c *gin.Context) { + user := GetUserFromContext(c) + if user == nil { + c.AbortWithStatusJSON(http.StatusUnauthorized, + gin.H{"error": "unauthenticated"}) + return + } + + for _, role := range roles { + if authSvc.HasRole(user, role) { + c.Next() + return + } + } + + c.AbortWithStatusJSON(http.StatusForbidden, + gin.H{"error": "insufficient permissions"}) + } +} + +func GetUserFromContext(c *gin.Context) *domain.User { + if user, exists := c.Get(UserContextKey); exists { + if u, ok := user.(*domain.User); ok { + return u + } + } + return nil +} +``` + +--- + +## Role-Based Authorization + +### Usage Examples + +```go +// All authenticated users +v1 := r.Group("/v1") +v1.Use(middlewares.AuthMiddleware(authSvc)) + +// Admin-only routes +admin := v1.Group("/admin") +admin.Use(middlewares.RequireRole(authSvc, "team-trade.admin")) + +// Multiple allowed roles (any match grants access) +project := v1.Group("/projects") +project.Use(middlewares.RequireRole(authSvc, "project-manager", "owner")) + +// Handler accessing authenticated user +func (h *UserHandler) GetProfile(c *gin.Context) { + user := middlewares.GetUserFromContext(c) + + c.JSON(200, gin.H{ + "user": user, + "roles": user.Roles, + }) +} +``` + +### Role Configuration + +Configure which client roles to extract via `KEYCLOAK_CLIENT_ID`: + +| Config | Roles Extracted | +|--------|-----------------| +| `KEYCLOAK_CLIENT_ID=exodus` | `["exodus.business-support", "exodus.default", "exodus.user"]` | +| `KEYCLOAK_CLIENT_ID=exodus-admin` | `["team-trade.admin"]` | +| `KEYCLOAK_CLIENT_ID=portal` | `["portal.user"]` | --- backend-dev/CODING-STANDARDS.md --- @@ -0,0 +1,441 @@ +# Coding Standards + +- [Table of Contents](#table-of-contents) +- [REST API Standards](#rest-api-standards) +- [Naming Conventions](#naming-conventions) +- [SOLID Guidelines](#solid-guidelines) +- [Critical Rules](#critical-rules) +- [Import Grouping](#import-grouping) +- [Error Handling](#error-handling) +- [Testing Patterns](#testing-patterns) + +--- + +## Table of Contents + +1. [REST API Standards](#rest-api-standards) +2. [Naming Conventions](#naming-conventions) +3. [SOLID Guidelines](#solid-guidelines) +4. [Critical Rules](#critical-rules) +5. [Import Grouping](#import-grouping) +6. [Error Handling](#error-handling) +7. [Testing Patterns](#testing-patterns) + +--- + +## REST API Standards + +### URL Naming + +| Pattern | Correct | Incorrect | +|---------|---------|-----------| +| Resources | `/users` | `/getUsers` | +| Nested | `/users/123/orders` | `/getUserOrders` | +| Actions | `/users/123/activate` | `/activateUser` | +| Plural | Always plural | Singular | + +### HTTP Methods + +| Method | Usage | Response | +|--------|-------|----------| +| GET | Retrieve resource(s) | 200 + body | +| POST | Create new resource | 201 + body | +| PUT | Full replace | 200/204 + body | +| PATCH | Partial update | 200/204 + body | +| DELETE | Remove resource | 204 No Content | + +### Status Codes + +| Code | Usage | +|------|-------| +| 200 | GET success, PUT/PATCH success | +| 201 | POST create success | +| 204 | DELETE success, no body | +| 400 | Malformed request, validation error | +| 401 | Authentication required | +| 403 | Authenticated but forbidden | +| 404 | Resource not found | +| 409 | Conflict (duplicate, state violation) | +| 422 | Semantic validation error | +| 500 | Internal server error | + +### Response Envelope + +```go +type Response struct { + Data interface{} `json:"data,omitempty"` + Error string `json:"error,omitempty"` + Meta *Meta `json:"meta,omitempty"` +} + +type Meta struct { + Page int `json:"page,omitempty"` + PerPage int `json:"per_page,omitempty"` + Total int `json:"total,omitempty"` + NextPage int `json:"next_page,omitempty"` +} +``` + +### Request/Response Examples + +**Create User (POST /users):** + +Request: +```json +{ + "name": "John Doe", + "email": "john@example.com" +} +``` + +Response (201): +```json +{ + "data": { + "nip": "p021050", + "name": "John Doe", + "email": "john@example.com", + "is_active": true, + "created_at": "2024-01-15T10:00:00Z" + } +} +``` + +**List Users (GET /users):** + +Response (200): +```json +{ + "data": [ + {"nip": "p021050", "name": "John Doe", ...}, + {"nip": "p021051", "name": "Jane Doe", ...} + ], + "meta": { + "page": 1, + "per_page": 20, + "total": 2 + } +} +``` + +**Validation Error (400):** + +```json +{ + "error": "validation failed", + "meta": { + "fields": { + "email": "invalid email format" + } + } +} +``` + +--- + +## Naming Conventions + +| Element | Convention | Example | +|---------|------------|---------| +| Packages | lowercase, single word | domain, usecase, persistence | +| Structs | PascalCase | UserService, ProjectHandler | +| Interfaces | PascalCase | UserRepository, CacheRepository | +| Functions | PascalCase (exported), camelCase (unexported) | NewUserService, getByNIP | +| Variables | camelCase | projectID, isActive | +| Constants | PascalCase | MaxRetries, DefaultTimeout | +| DB columns | snake_case | project_id, created_at | +| JSON fields | snake_case | project_id, channel_type | +| Env vars | UPPER_SNAKE | PERSISTENCE_DSN, LOG_LEVEL | + +--- + +## SOLID Guidelines + +### Keep Interfaces Small (ISP) + +**Good:** +```go +type UserReader interface { + GetByNIP(ctx context.Context, nip string) (*User, error) +} +``` + +**Bad:** +```go +type UserManager interface { + CreateUser() + UpdateUser() + DeleteUser() + ListUsers() + SearchUsers() + ValidateUser() + ActivateUser() + ExportUsers() + ImportUsers() + // ... 10 more methods +} +``` + +### No Global State + +**Bad:** +```go +var db *sql.DB + +func GetUser() { + db.Query(...) // Hidden dependency +} +``` + +**Good:** +```go +type userRepository struct { + pool *pgxpool.Pool +} + +func (r *userRepository) GetUser() { + r.pool.Query(...) +} +``` + +### Constructor Injection + +**Good:** +```go +func NewProvider(cfg *Config) (*Provider, error) { + pool, err := persistence.NewPool(...) + if err != nil { + return nil, err + } + return &Provider{pool: pool}, nil +} +``` + +**Bad:** +```go +func NewProvider() *Provider { + return &Provider{ + db: connectPostgres(), // Hidden, untestable + } +} +``` + +--- + +## Critical Rules + +### NEVER Skip Error Checking + +**Bad:** +```go +// Silent failure - user created but we don't know if it worked +userRepo.Create(ctx, user) + +// Ignored error - database might be down +rows, _ := db.Query(ctx, "SELECT * FROM users") + +// Deferred close without error check +defer file.Close() +``` + +**Good:** +```go +// Always check errors immediately +if err := userRepo.Create(ctx, user); err != nil { + return fmt.Errorf("failed to create user: %w", err) +} + +// Handle or return +rows, err := db.Query(ctx, "SELECT * FROM users") +if err != nil { + return nil, fmt.Errorf("query failed: %w", err) +} + +// Named return for deferred error check +func processFile() (err error) { + f, err := os.Open("file.txt") + if err != nil { + return err + } + defer func() { + if cerr := f.Close(); cerr != nil && err == nil { + err = cerr + } + }() + // ... process + return nil +} +``` + +**Rule:** If you see `_` for error ignoring, it must have a comment explaining why it's safe. + +--- + +## Import Grouping + +Group imports with blank line between groups: + +```go +import ( + // Standard library + "context" + "encoding/json" + "fmt" + "time" + + // Third-party packages + "github.com/gin-gonic/gin" + "github.com/jackc/pgx/v5" + "github.com/pkg/errors" + + // Internal packages + "myapp/internal/core/domain" + "myapp/internal/core/port/inbound" + "myapp/internal/core/port/outbound" +) +``` + +--- + +## Error Handling + +### Error Wrapping + +Use `github.com/pkg/errors` for error wrapping: + +```go +func (r *userRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { + var user domain.User + err := r.pool.QueryRow(ctx, + "SELECT nip, name FROM users WHERE nip = $1", nip, + ).Scan(&user.NIP, &user.Name) + + if errors.Is(err, pgx.ErrNoRows) { + return nil, nil // Not found = nil, nil + } + if err != nil { + return nil, errors.Wrap(err, "failed to get user") + } + return &user, nil +} +``` + +### Response Error Pattern + +```go +func (h *UserHandler) GetByNIP(c *gin.Context) { + resp := response.Response{} + defer resp.Render(c) + + nip := c.Param("nip") + + user, err := h.svc.GetByNIP(c.Request.Context(), nip) + if err != nil { + resp.SetError(err, http.StatusInternalServerError) + return + } + if user == nil { + resp.SetError(errors.New("user not found"), http.StatusNotFound) + return + } + + resp.Data = user + resp.StatusCode = http.StatusOK +} +``` + +--- + +## Testing Patterns + +### Unit Test with Mocks + +```go +package usecase + +import ( + "context" + "testing" + + "myapp/internal/core/domain" +) + +type mockUserRepository struct { + getByNIPFunc func(ctx context.Context, nip string) (*domain.User, error) + createFunc func(ctx context.Context, user *domain.User) error +} + +func (m *mockUserRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { + if m.getByNIPFunc != nil { + return m.getByNIPFunc(ctx, nip) + } + return nil, nil +} + +func (m *mockUserRepository) Create(ctx context.Context, user *domain.User) error { + if m.createFunc != nil { + return m.createFunc(ctx, user) + } + return nil +} + +func TestUserService_GetByNIP(t *testing.T) { + tests := []struct { + name string + nip string + setupMock func(*mockUserRepository) + expectError bool + }{ + { + name: "user found", + nip: "12345", + setupMock: func(mr *mockUserRepository) { + mr.getByNIPFunc = func(ctx context.Context, nip string) (*domain.User, error) { + return &domain.User{NIP: "12345", Name: "Test User"}, nil + } + }, + expectError: false, + }, + { + name: "user not found", + nip: "99999", + setupMock: func(mr *mockUserRepository) { + mr.getByNIPFunc = func(ctx context.Context, nip string) (*domain.User, error) { + return nil, nil + } + }, + expectError: false, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + userRepo := &mockUserRepository{} + tt.setupMock(userRepo) + + svc := NewUserService(userRepo, nil) + user, err := svc.GetByNIP(context.Background(), tt.nip) + + if tt.expectError && err == nil { + t.Error("expected error, got nil") + } + if !tt.expectError && err != nil { + t.Errorf("unexpected error: %v", err) + } + if user != nil && user.NIP != tt.nip { + t.Errorf("expected nip %s, got %s", tt.nip, user.NIP) + } + }) + } +} +``` + +### Test Naming + +``` +Test{UnitOfWork}_{Scenario}_{ExpectedBehavior} + +TestUserService_GetByNIP_UserFound_ReturnsUser +TestUserService_GetByNIP_UserNotFound_ReturnsNil +TestUserService_GetByNIP_DatabaseError_ReturnsError +``` --- backend-dev/CONFIGURATION.md --- @@ -0,0 +1,336 @@ +# Configuration + +- [Table of Contents](#table-of-contents) +- [Configuration Strategy](#configuration-strategy) +- [Local Development](#local-development) +- [OpenBao Integration](#openbao-integration) +- [Config Loader](#config-loader) +- [Provider Pattern](#provider-pattern) +- [Environment Variables](#environment-variables) + +--- + +## Table of Contents + +1. [Configuration Strategy](#configuration-strategy) +2. [Local Development](#local-development) +3. [OpenBao Integration](#openbao-integration) +4. [Config Loader](#config-loader) +5. [Provider Pattern](#provider-pattern) +6. [Environment Variables](#environment-variables) + +--- + +## Configuration Strategy + +``` +APP_ENV determines config source: + +┌─────────────────────────────────────────────────────────┐ +│ APP_ENV=value │ +├──────────────┬──────────────────────────────────────────┤ +│ "local" │ "staging" │ +│ │ │ +│ .env file │ OpenBao v2 │ +│ (golobby) │ (KV secrets) │ +│ │ │ +│ Manual │ Automated │ +│ editing │ secret injection │ +└─────────────┴──────────────────────────────────────────┘ +``` + +**Rules:** +- Never commit secrets to version control +- Local dev always uses `.env` file +- Staging/production uses OpenBao v2 +- Config struct validates required fields + +--- + +## Local Development + +### .env File + +Create `.env` in project root: + +```env +# Application +APP_ENV=local +APP_MODE=release +APP_PORT=8080 +APP_ALLOWED_ORIGINS=* + +# Database +PERSISTENCE_DSN=postgres://user:pass@localhost:5432/messaging?sslmode=disable +PERSISTENCE_MAX_CONNS=25 +PERSISTENCE_MIN_CONNS=5 + +# Cache +CACHE_DSN=postgres://user:pass@localhost:5432/messaging?sslmode=disable&search_path=cache +CACHE_MAX_CONNS=10 +CACHE_MIN_CONNS=2 +CACHE_DEFAULT_TTL=5m +CACHE_CLEANUP_INTERVAL=10m + +# Logging +LOG_LEVEL=info +LOG_FORMAT=json +LOG_MASK_FIELDS=password,token,api_key,secret,authorization + +# Tracing +TEMPO_ENABLED=false +TEMPO_ENDPOINT=tempo:4318 + +# Keycloak (JWT Authentication) +KEYCLOAK_JWKS_URL=https://auth.pharos.id/realms/production/protocol/openid-connect/certs +KEYCLOAK_CLIENT_ID=exodus + +# OpenBao (not used in local, but defined for completeness) +OPENBAO_ADDR=https://openbao.pharos.id +OPENBAO_TOKEN= +OPENBAO_MOUNT_PATH=secret +OPENBAO_SECRET_PATH=backend/local +``` + +### .env.example + +Share `.env.example` (without secrets) for other developers: + +```bash +cp .env .env.example +# Remove all secret values +``` + +--- + +## OpenBao Integration + +### OpenBao v2 KV + +Secrets stored at: `{mount_path}/data/{secret_path}` + +**Example:** +``` +secret/data/backend/production +├── PERSISTENCE_DSN=postgres://... +├── KEYCLOAK_JWKS_URL=https://... +├── KEYCLOAK_CLIENT_ID=exodus +└── ... other secrets +``` + +### Config Model + +```go +type OpenBaoConfig struct { + Addr string `env:"OPENBAO_ADDR" required:"true"` + Token string `env:"OPENBAO_TOKEN" required:"true"` + MountPath string `env:"OPENBAO_MOUNT_PATH" default:"secret"` + SecretPath string `env:"OPENBAO_SECRET_PATH" required:"true"` +} + +type Config struct { + App AppConfig + Persistence PersistenceConfig + Cache CacheConfig + Log LogConfig + Tempo TempoConfig + Keycloak KeycloakConfig + OpenBao OpenBaoConfig +} +``` + +--- + +## Config Loader + +```go +package config + +import ( + "context" + "fmt" + "os" + "time" + + "github.com/golobby/dotenv" + "github.com/golobby/env/v2" + "github.com/hashicorp/vault/v2/api" +) + +func Load(ctx context.Context) (*Config, error) { + env := os.Getenv("APP_ENV") + if env == "" { + env = "local" + } + + if env == "local" { + dotenv.Load() + return loadFromEnv() + } + + return loadFromOpenBao(ctx) +} + +func loadFromEnv() (*Config, error) { + var cfg Config + if err := env.Parse(&cfg); err != nil { + return nil, fmt.Errorf("failed to parse config from env: %w", err) + } + return &cfg, nil +} + +func loadFromOpenBao(ctx context.Context) (*Config, error) { + var cfg Config + if err := env.Parse(&cfg.OpenBao); err != nil { + return nil, fmt.Errorf("failed to parse OpenBao config: %w", err) + } + + client, err := api.NewClient(&api.Config{ + Address: cfg.OpenBao.Addr, + }) + if err != nil { + return nil, fmt.Errorf("failed to create OpenBao client: %w", err) + } + client.SetToken(cfg.OpenBao.Token) + + secret, err := client.KVv2(cfg.OpenBao.MountPath).Get(ctx, cfg.OpenBao.SecretPath) + if err != nil { + return nil, fmt.Errorf("failed to get secret from OpenBao: %w", err) + } + + data, ok := secret.Data["data"].(map[string]interface{}) + if !ok { + return nil, fmt.Errorf("invalid secret data format") + } + + // Apply secrets to config + cfg.Persistence.DSN = getString(data, "PERSISTENCE_DSN", cfg.Persistence.DSN) + cfg.Keycloak.JWKSURL = getString(data, "KEYCLOAK_JWKS_URL", cfg.Keycloak.JWKSURL) + cfg.Keycloak.ClientID = getString(data, "KEYCLOAK_CLIENT_ID", cfg.Keycloak.ClientID) + // ... apply other secrets + + return &cfg, nil +} + +func getString(data map[string]interface{}, key, fallback string) string { + if v, ok := data[key].(string); ok && v != "" { + return v + } + return fallback +} +``` + +--- + +## Provider Pattern + +Dependency wiring via explicit Provider struct. + +```go +package provider + +import ( + "context" + "fmt" + + "myapp/config" + "myapp/internal/adapter/outbound/cache" + "myapp/internal/adapter/outbound/keycloak" + "myapp/internal/adapter/outbound/persistence" + "myapp/internal/core/port/inbound" + "myapp/internal/core/usecase" + "myapp/pkg/logger" + "myapp/pkg/metrics" + "myapp/pkg/tracing" +) + +type Provider struct { + Config *config.Config + Logger *logger.Logger + Metrics *metrics.Prometheus + Tracer *tracing.TracerProvider + PersistencePool *persistence.Pool + CachePool *persistence.Pool + UserRepo outbound.UserRepository + CacheRepo outbound.CacheRepository + Keycloak *keycloak.Provider + AuthSvc inbound.AuthService + UserSvc inbound.UserService +} + +func NewProvider(ctx context.Context, cfg *config.Config) (*Provider, error) { + p := &Provider{Config: cfg} + + if err := p.initLogger(); err != nil { + return nil, fmt.Errorf("logger: %w", err) + } + if err := p.initMetrics(); err != nil { + return nil, fmt.Errorf("metrics: %w", err) + } + if err := p.initTracer(ctx); err != nil { + return nil, fmt.Errorf("tracer: %w", err) + } + if err := p.initPersistence(ctx); err != nil { + return nil, fmt.Errorf("persistence: %w", err) + } + if err := p.initCache(ctx); err != nil { + return nil, fmt.Errorf("cache: %w", err) + } + if err := p.initKeycloak(ctx); err != nil { + return nil, fmt.Errorf("keycloak: %w", err) + } + + p.UserRepo = persistence.NewUserRepository(p.PersistencePool) + p.CacheRepo = cache.NewPostgresCacheRepository(p.CachePool) + p.AuthSvc = usecase.NewAuthService(p.Keycloak, p.UserRepo) + p.UserSvc = usecase.NewUserService(p.UserRepo, p.CacheRepo) + + return p, nil +} + +func (p *Provider) Close(ctx context.Context) { + if p.PersistencePool != nil { + p.PersistencePool.Close() + } + if p.CachePool != nil { + p.CachePool.Close() + } + if p.Tracer != nil { + p.Tracer.Shutdown(ctx) + } +} +``` + +--- + +## Environment Variables + +### Required Variables + +| Variable | Description | Local Default | +|----------|-------------|---------------| +| `APP_ENV` | Environment name | `local` | +| `PERSISTENCE_DSN` | PostgreSQL connection string | Required | +| `KEYCLOAK_JWKS_URL` | Keycloak JWKS endpoint | Required | +| `KEYCLOAK_CLIENT_ID` | Keycloak client ID | Required | + +### Optional Variables + +| Variable | Description | Default | +|----------|-------------|---------| +| `APP_PORT` | HTTP server port | `8080` | +| `APP_MODE` | Gin mode (`release`, `debug`) | `release` | +| `APP_ALLOWED_ORIGINS` | CORS origins (comma-separated) | `*` | +| `LOG_LEVEL` | Log level (`debug`, `info`, `warn`, `error`) | `info` | +| `LOG_FORMAT` | Log format (`json`, `text`) | `json` | +| `TEMPO_ENABLED` | Enable tracing | `false` | +| `TEMPO_ENDPOINT` | Tempo OTLP endpoint | `tempo:4318` | + +### OpenBao Variables (Non-Local) + +| Variable | Description | +|----------|-------------| +| `OPENBAO_ADDR` | OpenBao server address | +| `OPENBAO_TOKEN` | OpenBao token | +| `OPENBAO_MOUNT_PATH` | KV v2 mount path | +| `OPENBAO_SECRET_PATH` | Secret path within mount | --- backend-dev/DATABASE.md --- @@ -0,0 +1,552 @@ +# Database + +- [Table of Contents](#table-of-contents) +- [Schema Design](#schema-design) +- [Migrations](#migrations) +- [Repository Pattern](#repository-pattern) +- [Caching Layer](#caching-layer) +- [Health Checks](#health-checks) +- [Connection Pooling](#connection-pooling) + +--- + +## Table of Contents + +1. [Schema Design](#schema-design) +2. [Migrations](#migrations) +3. [Repository Pattern](#repository-pattern) +4. [Caching Layer](#caching-layer) +5. [Health Checks](#health-checks) +6. [Connection Pooling](#connection-pooling) + +--- + +## Schema Design + +### Users Table + +```sql +CREATE TABLE IF NOT EXISTS users ( + nip VARCHAR(20) PRIMARY KEY, + name VARCHAR(255) NOT NULL, + email VARCHAR(255), + is_active BOOLEAN DEFAULT true, + last_login_at TIMESTAMP WITH TIME ZONE, + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() +); + +CREATE INDEX idx_users_email ON users(email); +CREATE INDEX idx_users_is_active ON users(is_active); +``` + +### Migration File + +```sql +-- db/migrations/persistence/0001_initial_schema.up.sql +CREATE TABLE IF NOT EXISTS users ( + nip VARCHAR(20) PRIMARY KEY, + name VARCHAR(255) NOT NULL, + email VARCHAR(255), + is_active BOOLEAN DEFAULT true, + last_login_at TIMESTAMP WITH TIME ZONE, + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), + updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() +); + +CREATE INDEX idx_users_email ON users(email); +CREATE INDEX idx_users_is_active ON users(is_active); + +-- db/migrations/persistence/0001_initial_schema.down.sql +DROP TABLE IF EXISTS users; +``` + +--- + +## Migrations + +### Using golang-migrate + +```bash +# Install CLI +go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest + +# Run migrations +migrate -database "$PERSISTENCE_DSN" -path db/migrations/persistence up + +# Rollback +migrate -database "$PERSISTENCE_DSN" -path db/migrations/persistence down 1 + +# Check status +migrate -database "$PERSISTENCE_DSN" -path db/migrations/persistence version +``` + +### Programmatic Migrations (Admin CLI) + +```go +package admin + +import ( + "fmt" + "strconv" + + "github.com/golang-migrate/migrate/v4" + _ "github.com/golang-migrate/migrate/v4/database/postgres" + _ "github.com/golang-migrate/migrate/v4/source/file" + "github.com/urfave/cli/v2" +) + +func migrateCommand(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "migrate", + Usage: "Database migrations", + Subcommands: []*cli.Command{ + { + Name: "up", + Usage: "Apply all pending migrations", + Action: func(c *cli.Context) error { + m, err := migrate.New( + "file://db/migrations/persistence", + cfg.Persistence.DSN, + ) + if err != nil { + return fmt.Errorf("failed to create migrator: %w", err) + } + if err := m.Up(); err != nil && err != migrate.ErrNoChange { + return fmt.Errorf("migration failed: %w", err) + } + fmt.Println("Migrations applied successfully") + return nil + }, + }, + { + Name: "down", + Usage: "Rollback migrations", + Action: func(c *cli.Context) error { + if cfg.App.Env == "production" { + return fmt.Errorf("migrate down is disabled in production environment") + } + + steps := 1 + if c.Args().Len() > 0 { + steps, _ = strconv.Atoi(c.Args().First()) + } + + m, err := migrate.New( + "file://db/migrations/persistence", + cfg.Persistence.DSN, + ) + if err != nil { + return fmt.Errorf("failed to create migrator: %w", err) + } + if err := m.Steps(-steps); err != nil && err != migrate.ErrNoChange { + return fmt.Errorf("rollback failed: %w", err) + } + fmt.Printf("Rolled back %d migration(s)\n", steps) + return nil + }, + }, + { + Name: "status", + Usage: "Show migration status", + Action: func(c *cli.Context) error { + m, err := migrate.New( + "file://db/migrations/persistence", + cfg.Persistence.DSN, + ) + if err != nil { + return fmt.Errorf("failed to create migrator: %w", err) + } + version, dirty, err := m.Version() + if err != nil { + if err == migrate.ErrNilVersion { + fmt.Println("No migrations applied") + return nil + } + return fmt.Errorf("failed to get version: %w", err) + } + fmt.Printf("Version: %d, Dirty: %v\n", version, dirty) + return nil + }, + }, + }, + } +} +``` + +--- + +## Repository Pattern + +### Pool Initialization + +```go +package persistence + +import ( + "context" + "fmt" + "time" + + "github.com/jackc/pgx/v5/pgxpool" +) + +type Pool struct { + *pgxpool.Pool +} + +func NewPool(ctx context.Context, dsn string, maxConns, minConns int) (*Pool, error) { + config, err := pgxpool.ParseConfig(dsn) + if err != nil { + return nil, fmt.Errorf("failed to parse config: %w", err) + } + + config.MaxConns = int32(maxConns) + config.MinConns = int32(minConns) + config.MaxConnLifetime = time.Hour + config.MaxConnIdleTime = 30 * time.Minute + config.HealthCheckPeriod = time.Minute + + pool, err := pgxpool.NewWithConfig(ctx, config) + if err != nil { + return nil, fmt.Errorf("failed to create pool: %w", err) + } + + if err := pool.Ping(ctx); err != nil { + return nil, fmt.Errorf("failed to ping pool: %w", err) + } + + return &Pool{Pool: pool}, nil +} + +func (p *Pool) Close() { + p.Pool.Close() +} +``` + +### User Repository Implementation + +```go +package persistence + +import ( + "context" + + "myapp/internal/core/domain" + "myapp/internal/core/port/outbound" + + "github.com/jackc/pgx/v5" + "github.com/pkg/errors" + "go.opentelemetry.io/otel/attribute" +) + +type userRepository struct { + pool *Pool +} + +func NewUserRepository(pool *Pool) outbound.UserRepository { + return &userRepository{pool: pool} +} + +func (r *userRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { + ctx, span := tracer.Start(ctx, "UserRepository.GetByNIP") + defer span.End() + + var user domain.User + err := r.pool.QueryRow(ctx, + `SELECT nip, name, email, is_active, last_login_at, created_at, updated_at + FROM users WHERE nip = $1`, + nip, + ).Scan(&user.NIP, &user.Name, &user.Email, &user.IsActive, + &user.LastLoginAt, &user.CreatedAt, &user.UpdatedAt) + + if err == pgx.ErrNoRows { + span.AddEvent("user_not_found") + return nil, nil + } + if err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to get user") + } + + span.SetAttributes(attribute.String("user.nip", user.NIP)) + return &user, nil +} + +func (r *userRepository) Create(ctx context.Context, user *domain.User) error { + ctx, span := tracer.Start(ctx, "UserRepository.Create") + defer span.End() + + err := r.pool.QueryRow(ctx, + `INSERT INTO users (nip, name, email, is_active, last_login_at) + VALUES ($1, $2, $3, $4, $5) + RETURNING created_at, updated_at`, + user.NIP, user.Name, user.Email, user.IsActive, user.LastLoginAt, + ).Scan(&user.CreatedAt, &user.UpdatedAt) + + if err != nil { + span.RecordError(err) + return errors.Wrap(err, "failed to create user") + } + + return nil +} + +func (r *userRepository) Update(ctx context.Context, user *domain.User) error { + ctx, span := tracer.Start(ctx, "UserRepository.Update") + defer span.End() + + _, err := r.pool.Exec(ctx, + `UPDATE users SET name = $1, email = $2, is_active = $3, + last_login_at = $4, updated_at = NOW() + WHERE nip = $5`, + user.Name, user.Email, user.IsActive, user.LastLoginAt, user.NIP, + ) + if err != nil { + span.RecordError(err) + return errors.Wrap(err, "failed to update user") + } + + return nil +} + +func (r *userRepository) List(ctx context.Context) ([]domain.User, error) { + ctx, span := tracer.Start(ctx, "UserRepository.List") + defer span.End() + + rows, err := r.pool.Query(ctx, + `SELECT nip, name, email, is_active, last_login_at, created_at, updated_at + FROM users ORDER BY created_at DESC`, + ) + if err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to list users") + } + defer rows.Close() + + var users []domain.User + for rows.Next() { + var u domain.User + if err := rows.Scan(&u.NIP, &u.Name, &u.Email, &u.IsActive, + &u.LastLoginAt, &u.CreatedAt, &u.UpdatedAt); err != nil { + span.RecordError(err) + return nil, errors.Wrap(err, "failed to scan user") + } + users = append(users, u) + } + + span.SetAttributes(attribute.Int("users.count", len(users))) + return users, nil +} +``` + +--- + +## Caching Layer + +### Cache Schema + +```sql +-- db/migrations/cache/0001_cache_schema.up.sql +CREATE SCHEMA IF NOT EXISTS cache; + +CREATE UNLOGGED TABLE cache.entries ( + key TEXT PRIMARY KEY, + value BYTEA NOT NULL, + expires_at TIMESTAMP WITH TIME ZONE, + created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() +); + +CREATE INDEX idx_cache_expires ON cache.entries(expires_at) + WHERE expires_at IS NOT NULL; + +-- db/migrations/cache/0001_cache_schema.down.sql +DROP TABLE IF EXISTS cache.entries; +DROP SCHEMA IF EXISTS cache; +``` + +### Cache Repository + +```go +package cache + +import ( + "context" + "time" + + "myapp/internal/core/port/outbound" + + "github.com/jackc/pgx/v5" + "github.com/pkg/errors" +) + +type PostgresCacheRepository struct { + pool *Pool +} + +func NewPostgresCacheRepository(pool *Pool) outbound.CacheRepository { + return &PostgresCacheRepository{pool: pool} +} + +func (r *PostgresCacheRepository) Get(ctx context.Context, key string) ([]byte, error) { + var value []byte + var expiresAt time.Time + + err := r.pool.QueryRow(ctx, + `SELECT value, expires_at FROM cache.entries + WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())`, + key, + ).Scan(&value, &expiresAt) + + if err == pgx.ErrNoRows { + return nil, nil + } + if err != nil { + return nil, errors.Wrap(err, "failed to get cache") + } + + return value, nil +} + +func (r *PostgresCacheRepository) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error { + expiresAt := time.Now().Add(ttl) + + _, err := r.pool.Exec(ctx, + `INSERT INTO cache.entries (key, value, expires_at) + VALUES ($1, $2, $3) + ON CONFLICT (key) DO UPDATE SET value = $2, expires_at = $3`, + key, value, expiresAt, + ) + if err != nil { + return errors.Wrap(err, "failed to set cache") + } + + return nil +} + +func (r *PostgresCacheRepository) Delete(ctx context.Context, key string) error { + _, err := r.pool.Exec(ctx, `DELETE FROM cache.entries WHERE key = $1`, key) + if err != nil { + return errors.Wrap(err, "failed to delete cache") + } + return nil +} + +func (r *PostgresCacheRepository) Exists(ctx context.Context, key string) (bool, error) { + var exists bool + err := r.pool.QueryRow(ctx, + `SELECT EXISTS( + SELECT 1 FROM cache.entries + WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW()) + )`, + key, + ).Scan(&exists) + if err != nil { + return false, errors.Wrap(err, "failed to check cache existence") + } + return exists, nil +} + +func (r *PostgresCacheRepository) Clear(ctx context.Context) error { + _, err := r.pool.Exec(ctx, `TRUNCATE cache.entries`) + if err != nil { + return errors.Wrap(err, "failed to clear cache") + } + return nil +} +``` + +### Why Unlogged Tables? + +- No Write-Ahead Log (WAL) overhead = faster writes +- Data survives crashes but not unclean shutdowns +- Acceptable for cache (rebuilds from source on restart) + +--- + +## Health Checks + +### Database Health Check + +```go +func checkDatabase(pool *persistence.Pool) bool { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + return pool.Ping(ctx) == nil +} + +func checkCache(pool *persistence.Pool) bool { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + return pool.Ping(ctx) == nil +} +``` + +### Health Endpoint + +```go +r.GET("/healthz", func(c *gin.Context) { + checks := map[string]bool{ + "database": checkDatabase(provider.PersistencePool), + "cache": checkCache(provider.CachePool), + } + + allHealthy := true + for _, ok := range checks { + if !ok { + allHealthy = false + break + } + } + + response := gin.H{ + "status": "ok", + "version": buildinfo.Version, + "commit": buildinfo.Commit, + "checks": checks, + } + + if allHealthy { + c.JSON(200, response) + } else { + c.JSON(503, response) + } +}) +``` + +--- + +## Connection Pooling + +### Pool Configuration + +```go +config.MaxConns = 25 // Maximum connections +config.MinConns = 5 // Minimum connections +config.MaxConnLifetime = time.Hour // Connection lifetime +config.MaxConnIdleTime = 30 * time.Minute // Idle timeout +config.HealthCheckPeriod = time.Minute // Health check interval +``` + +### Monitoring + +```go +import "github.com/prometheus/client_golang/prometheus" + +func RecordDBMetrics(pool *pgxpool.Pool) { + go func() { + for { + stats := pool.Stats() + prometheus.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "db_pool_connections", + Help: "Number of connections in the pool", + }, + []string{"state"}, + ).WithLabelValues("active").Set(float64(stats.TotalConns)) + + time.Sleep(15 * time.Second) + } + }() +} +``` --- backend-dev/DEPLOYMENT.md --- @@ -0,0 +1,588 @@ +# Deployment + +- [Table of Contents](#table-of-contents) +- [Application Bootstrapper](#application-bootstrapper) +- [Admin CLI](#admin-cli) +- [Health Endpoint](#health-endpoint) +- [Docker](#docker) +- [Docker Compose](#docker-compose) +- [GitLab CI/CD](#gitlab-cicd) +- [Operations](#operations) + +--- + +## Table of Contents + +1. [Application Bootstrapper](#application-bootstrapper) +2. [Admin CLI](#admin-cli) +3. [Health Endpoint](#health-endpoint) +4. [Docker](#docker) +5. [Docker Compose](#docker-compose) +6. [GitLab CI/CD](#gitlab-cicd) +7. [Operations](#operations) + +--- + +## Application Bootstrapper + +Single binary with subcommand dispatch. + +### main.go + +```go +package main + +import ( + "fmt" + "os" + + "myapp/cmd/admin" + "myapp/cmd/server" +) + +func main() { + if len(os.Args) < 2 { + server.Run([]string{}) + return + } + + switch os.Args[1] { + case "server": + server.Run(os.Args[2:]) + case "admin": + admin.Run(os.Args[2:]) + case "version", "-v", "--version": + printVersion() + case "--help", "-h": + printHelp() + default: + fmt.Printf("Unknown command: %s\n", os.Args[1]) + printHelp() + os.Exit(1) + } +} + +func printVersion() { + fmt.Printf("%s version %s (commit: %s)\n", + buildinfo.Name, + buildinfo.Version, + buildinfo.Commit, + ) +} + +func printHelp() { + fmt.Println("Usage: messaging-be [command] [args...]") + fmt.Println() + fmt.Println("Commands:") + fmt.Println(" server Run HTTP server (default)") + fmt.Println(" admin [cmd] Run administrative commands") + fmt.Println(" version Show version information") + fmt.Println(" --help Show this help") +} +``` + +### cmd/server/run.go + +```go +package server + +import ( + "context" + "flag" + "fmt" + "os" + + "myapp/config" + "myapp/internal/provider" + "myapp/pkg/logger" +) + +func Run(args []string) { + fs := flag.NewFlagSet("server", flag.ExitOnError) + port := fs.String("port", "", "Server port (overrides config)") + + if err := fs.Parse(args); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } + + ctx := context.Background() + cfg, err := config.Load(ctx) + if err != nil { + fmt.Fprintf(os.Stderr, "Failed to load config: %v\n", err) + os.Exit(1) + } + + if *port != "" { + cfg.App.Port = *port + } + + pvd, err := provider.NewProvider(ctx, cfg) + if err != nil { + fmt.Fprintf(os.Stderr, "Failed to initialize: %v\n", err) + os.Exit(1) + } + defer pvd.Close(ctx) + + log := logger.Default() + log.Info("Starting server", "port", cfg.App.Port) + + if err := runServer(pvd, cfg); err != nil { + log.Error("Server error", "error", err) + os.Exit(1) + } +} +``` + +### cmd/admin/run.go + +```go +package admin + +import ( + "context" + "fmt" + "os" + "strconv" + + "github.com/golang-migrate/migrate/v4" + _ "github.com/golang-migrate/migrate/v4/database/postgres" + _ "github.com/golang-migrate/migrate/v4/source/file" + "github.com/urfave/cli/v2" + "myapp/config" + "myapp/internal/provider" +) + +func Run(args []string) { + ctx := context.Background() + + cfg, err := config.Load(ctx) + if err != nil { + fmt.Fprintf(os.Stderr, "Config error: %v\n", err) + os.Exit(1) + } + + pvd, err := provider.NewProvider(ctx, cfg) + if err != nil { + fmt.Fprintf(os.Stderr, "Provider error: %v\n", err) + os.Exit(1) + } + defer pvd.Close(ctx) + + app := &cli.App{ + Name: "admin", + Usage: "Administrative commands", + Commands: []*cli.Command{ + migrateCommand(cfg), + userCommand(pvd), + cacheCommand(pvd), + healthCommand(pvd), + }, + } + + if err := app.Run(args); err != nil { + fmt.Fprintf(os.Stderr, "Error: %v\n", err) + os.Exit(1) + } +} + +func migrateCommand(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "migrate", + Usage: "Database migrations", + Subcommands: []*cli.Command{ + { + Name: "up", + Usage: "Apply all pending migrations", + Action: func(c *cli.Context) error { + m, err := migrate.New( + "file://db/migrations/persistence", + cfg.Persistence.DSN, + ) + if err != nil { + return fmt.Errorf("failed to create migrator: %w", err) + } + if err := m.Up(); err != nil && err != migrate.ErrNoChange { + return fmt.Errorf("migration failed: %w", err) + } + fmt.Println("Migrations applied successfully") + return nil + }, + }, + { + Name: "down", + Usage: "Rollback migrations", + Action: func(c *cli.Context) error { + if cfg.App.Env == "production" { + return fmt.Errorf("migrate down is disabled in production environment") + } + + steps := 1 + if c.Args().Len() > 0 { + steps, _ = strconv.Atoi(c.Args().First()) + } + + m, err := migrate.New( + "file://db/migrations/persistence", + cfg.Persistence.DSN, + ) + if err != nil { + return fmt.Errorf("failed to create migrator: %w", err) + } + if err := m.Steps(-steps); err != nil && err != migrate.ErrNoChange { + return fmt.Errorf("rollback failed: %w", err) + } + fmt.Printf("Rolled back %d migration(s)\n", steps) + return nil + }, + }, + { + Name: "status", + Usage: "Show migration status", + Action: func(c *cli.Context) error { + m, err := migrate.New( + "file://db/migrations/persistence", + cfg.Persistence.DSN, + ) + if err != nil { + return fmt.Errorf("failed to create migrator: %w", err) + } + version, dirty, err := m.Version() + if err != nil { + if err == migrate.ErrNilVersion { + fmt.Println("No migrations applied") + return nil + } + return fmt.Errorf("failed to get version: %w", err) + } + fmt.Printf("Version: %d, Dirty: %v\n", version, dirty) + return nil + }, + }, + }, + } +} +``` + +--- + +## Admin CLI + +### Available Commands + +```bash +# Database migrations +./messaging-be admin migrate up # Apply all pending +./messaging-be admin migrate down 1 # Rollback 1 migration +./messaging-be admin migrate status # Check status + +# User management +./messaging-be admin user list # List all users +./messaging-be admin user info # Show user details +./messaging-be admin user deactivate # Deactivate user +./messaging-be admin user activate # Reactivate user + +# Cache management +./messaging-be admin cache clear # Clear all cache +./messaging-be admin cache stats # Show cache stats + +# Health checks +./messaging-be admin health # Check all services +``` + +### Production Safety + +`migrate down` is disabled in production: + +```go +if cfg.App.Env == "production" { + return fmt.Errorf("migrate down is disabled in production environment") +} +``` + +--- + +## Health Endpoint + +### Endpoint: GET /healthz + +Returns 200 if healthy, 503 if degraded: + +```json +{ + "status": "ok", + "version": "1.0.0", + "commit": "abc123", + "checks": { + "database": true, + "cache": true + } +} +``` + +### Implementation + +```go +r.GET("/healthz", func(c *gin.Context) { + checks := map[string]bool{ + "database": checkDatabase(provider.PersistencePool), + "cache": checkCache(provider.CachePool), + } + + allHealthy := true + for _, ok := range checks { + if !ok { + allHealthy = false + break + } + } + + status := "ok" + if !allHealthy { + status = "degraded" + } + + response := gin.H{ + "status": status, + "version": buildinfo.Version, + "commit": buildinfo.Commit, + "checks": checks, + } + + if allHealthy { + c.JSON(200, response) + } else { + c.JSON(503, response) + } +}) +``` + +--- + +## Docker + +### Dockerfile (Multi-stage) + +```dockerfile +FROM golang:1.21-alpine AS builder + +WORKDIR /app +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . +RUN CGO_ENABLED=0 GOOS=linux go build \ + -ldflags "-s -w \ + -X myapp/pkg/buildinfo.Version=$VERSION \ + -X myapp/pkg/buildinfo.Commit=$COMMIT" \ + -o messaging-be . + +FROM alpine:3.19 + +RUN apk --no-cache add ca-certificates tzdata + +WORKDIR /app +COPY --from=builder /app/messaging-be . +COPY --from=builder /app/db/migrations ./db/migrations + +EXPOSE 8080 + +CMD ["./messaging-be"] +``` + +### Build Args + +| Arg | Description | +|-----|-------------| +| `VERSION` | Version string (e.g., `1.0.0`) | +| `COMMIT` | Git commit SHA | + +--- + +## Docker Compose + +### docker-compose.yml + +```yaml +version: '3.8' + +services: + app: + build: .. + env_file: ../.env.local + ports: + - "8080:8080" + environment: + PERSISTENCE_DSN: ${PERSISTENCE_DSN} + KEYCLOAK_JWKS_URL: ${KEYCLOAK_JWKS_URL} + KEYCLOAK_CLIENT_ID: ${KEYCLOAK_CLIENT_ID} +``` + +### .env.local (for local dev) + +```env +APP_ENV=local +APP_PORT=8080 + +PERSISTENCE_DSN=postgres://user:pass@host:5432/messaging?sslmode=disable +KEYCLOAK_JWKS_URL=https://auth.pharos.id/realms/production/protocol/openid-connect/certs +KEYCLOAK_CLIENT_ID=exodus +``` + +--- + +## GitLab CI/CD + +### .gitlab-ci.yml + +```yaml +stages: + - build + - test + - release + - deploy + +variables: + IMAGE_NAME: $CI_REGISTRY_IMAGE/messaging-be + +build: + stage: build + image: golang:1.21-alpine + before_script: + - apk add git make + script: + - make tidy + - make build + artifacts: + paths: + - bin/ + expire_in: 1 day + +test: + stage: test + image: golang:1.21-alpine + services: + - postgres:15-alpine + variables: + POSTGRES_DB: test + POSTGRES_USER: test + POSTGRES_PASSWORD: test + PERSISTENCE_DSN: postgres://test:test@postgres:5432/test?sslmode=disable + before_script: + - apk add git make + script: + - make migrate + - go test -race -coverprofile=coverage.out ./... + - go tool cover -func=coverage.out + coverage: '/total:\s+\(statements\)\s+(\d+\.\d+)%/' + +release: + stage: release + image: docker:24.0-cli + services: + - docker:24.0-dind + script: + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + - docker build --build-arg VERSION=$CI_COMMIT_SHA --build-arg COMMIT=$CI_COMMIT_SHORT_SHA -t $IMAGE_NAME:$CI_COMMIT_SHA -t $IMAGE_NAME:latest . + - docker push $IMAGE_NAME:$CI_COMMIT_SHA + - docker push $IMAGE_NAME:latest + rules: + - main + +deploy-production: + stage: deploy + image: bitnami/kubectl:latest + environment: + name: production + url: https://api.example.com + script: + - kubectl set image deployment/messaging-be server=$IMAGE_NAME:$CI_COMMIT_SHA + - kubectl rollout status deployment/messaging-be + rules: + - main + when: manual +``` + +--- + +## Operations + +### Build + +```bash +make build +``` + +### Run Server + +```bash +./bin/messaging-be +./bin/messaging-be server +./bin/messaging-be server -port=8081 +``` + +### Run Admin Commands + +```bash +./bin/messaging-be admin migrate up +./bin/messaging-be admin user list +./bin/messaging-be admin health +``` + +### Docker Operations + +```bash +# Build and run +docker-compose up -d + +# Run migrations +docker-compose exec app /app/messaging-be admin migrate up + +# Check logs +docker-compose logs -f app + +# Health check +curl http://localhost:8080/healthz +``` + +### Version + +```bash +./bin/messaging-be version +# Output: messaging-be version 1.0.0 (commit: abc123) +``` + +### Makefile + +```makefile +BINARY_NAME=messaging-be +VERSION=$(shell git describe --tags --always --dirty 2>/dev/null || echo "dev") +COMMIT=$(shell git rev-parse --short HEAD 2>/dev/null || echo "unknown") + +.PHONY: build clean run server admin migrate + +build: +\tgo build -ldflags "-s -w -X myapp/pkg/buildinfo.Version=$(VERSION) -X myapp/pkg/buildinfo.Commit=$(COMMIT)" -o bin/$(BINARY_NAME) . + +clean: +\trm -rf bin/ + +run: build +\t./bin/$(BINARY_NAME) + +server: build +\t./bin/$(BINARY_NAME) server + +admin: build +\t./bin/$(BINARY_NAME) admin + +migrate: build +\t./bin/$(BINARY_NAME) admin migrate up + +version: build +\t./bin/$(BINARY_NAME) version + +tidy: +\tgo mod tidy +\tgo mod verify +``` --- backend-dev/SKILL.md --- @@ -1,6 +1,6 @@ # Go Hexagonal Architecture - Production Ready -Go backend service template using hexagonal (ports & adapters) architecture with Gin, PostgreSQL/pgx, Prometheus, OpenTelemetry, and enterprise-grade observability. +Production-ready Go backend template with hexagonal architecture, 12-factor compliance, and Keycloak OIDC integration. ## When to Use @@ -12,3743 +12,200 @@ Go backend service template using hexagonal (ports & adapters) architecture with --- -## Project Structure - -``` -. -├── cmd/ -│ ├── server/ # Application entry point -│ │ └── main.go # Server initialization and wiring -│ ├── migrate/ # Database migration command -│ │ └── main.go -│ └── admin/ # Admin CLI (Factor 12) -│ └── main.go -├── config/ # Configuration models and loading -│ ├── config.go # Config initialization -│ ├── model.go # Config struct definitions -│ └── loader.go # OpenBao/env loader -├── internal/ -│ ├── core/ # Business logic (hexagon center) -│ │ ├── domain/ # Domain entities and value objects -│ │ │ ├── user.go -│ │ │ └── event/ # Domain events (DDD) -│ │ ├── port/ -│ │ │ ├── inbound/ # Service interfaces (usecases) -│ │ │ └── outbound/ # Repository/provider interfaces -│ │ └── usecase/ # Business logic implementations -│ ├── adapter/ # External adapters -│ │ ├── inbound/ # Driving adapters (HTTP handlers) -│ │ │ └── rest/v1/ -│ │ └── outbound/ # Driven adapters (DB, external APIs) -│ │ ├── persistence/ # Main data store (pgx/pgxpool) -│ │ ├── cache/ # Cache layer (pgx, unlogged tables) -│ │ ├── keycloak/ # Keycloak JWT provider -│ │ └── openbao/ # OpenBao v2 client -│ └── provider/ # Dependency provider (Option B) -│ └── provider.go -├── middlewares/ # HTTP middleware -│ ├── logging.go # Request/response logging -│ ├── metrics.go # Prometheus metrics -│ ├── tracing.go # OpenTelemetry tracing -│ └── auth.go # JWT authentication -├── pkg/ # Shared packages -│ ├── logger/ # Structured logging (slog) -│ │ ├── logger.go -│ │ ├── context.go -│ │ └── mask.go -│ ├── metrics/ # Prometheus metrics -│ │ └── prometheus.go -│ ├── tracing/ # OpenTelemetry/Tempo -│ │ ├── otel.go -│ │ └── sampler.go -│ ├── response/ # HTTP response utilities -│ └── grace/ # Graceful shutdown -├── db/migrations/ # Database migrations -│ ├── persistence/ # Main schema migrations -│ └── cache/ # Cache schema migrations -├── deployments/ # Deployment configs (Factor 5, 10) -│ ├── Dockerfile -│ ├── docker-compose.yml -│ └── .gitlab-ci.yml -└── main.go # Application bootstrap -``` - ---- - -## Core Libraries - -### Web Framework - -- **gin-gonic/gin** - HTTP web framework - - Routing, middleware, request binding - - JSON validation and rendering - -### Database - -- **jackc/pgx/v5** - PostgreSQL driver with connection pooling - - Native pgx for high performance - - pgxpool for connection management -- **golang-migrate/migrate** - Database migrations - -### Observability - -- **log/slog** - Structured logging (Go 1.21+ standard library) -- **prometheus/client_golang** - Prometheus metrics -- **go.opentelemetry.io/otel** - Distributed tracing - -### Configuration - -- **golobby/env/v2** - Environment variable loading -- **golobby/dotenv** - .env file support -- **hashicorp/vault** - OpenBao/OpenVault secrets client (KV v2) - -### Utilities - -- **pkg/errors** - Error wrapping with stack traces -- **google/uuid** - UUID generation - -### Authentication - -- **MicahParks/keyfunc/v3** - JWKS key fetching and JWT validation - - No client secret required for Resource Server pattern - - Automatic JWKS endpoint discovery and caching - - Supports RS256 signature validation -- **golang-jwt/jwt/v5** - JWT parsing and claims extraction - ---- - -## 12-Factor Compliance - -This template follows [12-Factor App](https://12factor.net/) methodology for production-ready deployments. - -### Factor 1: Codebase - -One codebase tracked in version control, multiple deploys: - -- Git repository with main branch -- Environment-specific configs via environment variables -- Same codebase deploys to local, staging, production - -### Factor 2: Dependencies - -Explicitly declare and isolate dependencies: - -```go -// go.mod -require ( - github.com/gin-gonic/gin v1.12.0 - github.com/jackc/pgx/v5 v5.5.0 -) -``` - -Never rely on system-wide packages. Use `go mod vendor` for full isolation. - -### Factor 3: Config - -Store config in environment variables (Factor 3). See [Configuration](#configuration) for details: - -- Local dev: `.env` file via golobby/dotenv -- Staging/prod: OpenBao v2 KV secrets -- No config hardcoded in code - -### Factor 4: Backing Services - -Treat backing services as attached resources: - -```go -// Connection strings via config, not hardcoded -dsn := cfg.Persistence.DSN // PostgreSQL -jwksURL := cfg.Keycloak.JWKSURL // Keycloak -tempoEndpoint := cfg.Tempo.Endpoint // OpenTelemetry -``` - -Health checks for all backing services: - -```go -// Health check endpoint -r.GET("/healthz", func(c *gin.Context) { - checks := map[string]bool{ - "database": pingDB(), - "cache": pingCache(), - } - for name, ok := range checks { - if !ok { - c.JSON(503, gin.H{"status": "unhealthy", "checks": checks}) - return - } - } - c.JSON(200, gin.H{"status": "ok"}) -}) -``` - -### Factor 5: Build, Release, Run - -Strict separation of build, release, and run stages: - -```yaml -# .gitlab-ci.yml -stages: - - build - - test - - release - - deploy - -build: - stage: build - script: - - go build -ldflags "-X main.version=$CI_COMMIT_SHA" -o bin/server ./cmd/server - -test: - stage: test - script: - - go test -race -coverprofile=coverage.out ./... - coverage: '/total:\s+\(statements\)\s+(\d+\.\d+)%/' - -release: - stage: release - script: - - docker build -t $IMAGE_NAME:$CI_COMMIT_SHA . - - docker push $IMAGE_NAME:$CI_COMMIT_SHA - rules: - - main - -deploy: - stage: deploy - script: - - kubectl set image deployment/server server=$IMAGE_NAME:$CI_COMMIT_SHA - environment: - name: production - rules: - - main -``` - -### Factor 6: Processes - -Stateless processes with no shared state: - -```go -// Share-nothing architecture -// User sessions stored in PostgreSQL, not memory -// File uploads to object storage, not local disk -// Cache in PostgreSQL unlogged tables, not process memory -``` - -### Factor 7: Port Binding - -Self-contained HTTP service: - -```go -// cmd/server/main.go -port := cfg.App.Port -if !strings.HasPrefix(port, ":") { - port = ":" + port -} -log.Fatal(http.ListenAndServe(port, r.Handler())) -``` - -### Factor 8: Concurrency - -Scale via process model: - -```go -// Worker pool for background jobs -type WorkerPool struct { - workers int - jobs chan Job - wg sync.WaitGroup -} - -func (wp *WorkerPool) Start(ctx context.Context) { - for i := 0; i < wp.workers; i++ { - wp.wg.Add(1) - go wp.worker(ctx, i) - } -} - -func (wp *WorkerPool) Submit(job Job) { - wp.jobs <- job -} -``` - -### Factor 9: Disposability - -Fast startup and graceful shutdown (Factor 9): - -```go -// Graceful shutdown with SIGTERM -func gracefulShutdown(sig os.Signal, server *http.Server) { - ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) - defer cancel() - - log.Printf("Received %s, shutting down gracefully...", sig) - if err := server.Shutdown(ctx); err != nil { - log.Fatalf("Server shutdown failed: %v", err) - } -} -``` - -### Factor 10: Dev/Prod Parity - -Use Docker Compose for local development: - -```yaml -# deployments/docker-compose.yml -version: '3.8' -services: - app: - build: . - env_file: .env - depends_on: - postgres: - condition: service_healthy - keycloak: - condition: service_started - ports: - - "8080:8080" - - postgres: - image: postgres:15 - environment: - POSTGRES_DB: messaging - POSTGRES_USER: user - POSTGRES_PASSWORD: pass - healthcheck: - test: ["CMD-SHELL", "pg_isready -U user -d messaging"] - interval: 5s - timeout: 5s - retries: 5 - - keycloak: - image: quay.io/keycloak/keycloak:24.0 - command: start-dev - environment: - KEYCLOAK_ADMIN: admin - KEYCLOAK_ADMIN_PASSWORD: admin - ports: - - "8081:8080" -``` - -### Factor 11: Logs - -Structured logging to stdout only: - -```go -// Use slog - Go 1.21+ standard library -log := slog.New(slog.NewJSONHandler(os.Stdout, nil)) -log.Info("server started", "port", 8080, "env", "production") -``` - -No log files in container. Aggregate via external services (Loki, CloudWatch). - -### Factor 12: Admin Processes - -One-off admin tasks as CLI commands. See [Admin CLI](#admin-cli) section. - ---- - -## Architecture Patterns - -### 1. Domain (Core/Entities) - -Pure business entities with no external dependencies: - -```go -package domain - -import "time" - -type User struct { - NIP string `db:"nip" json:"nip"` - Name string `db:"name" json:"name"` - IsActive bool `db:"is_active" json:"is_active"` - CreatedAt time.Time `db:"created_at" json:"created_at"` -} -``` - -**Key Rules:** - -- No framework dependencies -- Use struct tags for DB (`db:"`) and JSON (`json:"`) mapping -- Simple structs with primitive types - -### 2. Ports (Interfaces) - -Define contracts for dependencies. - -**Inbound Ports (Usecases):** - -```go -package inbound - -import ( - "context" - "myapp/internal/core/domain" -) - -type UserService interface { - GetByNIP(ctx context.Context, nip string) (*domain.User, error) - Create(ctx context.Context, user *domain.User) error - List(ctx context.Context) ([]domain.User, error) -} -``` - -**Outbound Ports (Repositories):** - -```go -package outbound - -import ( - "context" - "myapp/internal/core/domain" -) - -type UserRepository interface { - GetByNIP(ctx context.Context, nip string) (*domain.User, error) - Create(ctx context.Context, user *domain.User) error - List(ctx context.Context) ([]domain.User, error) -} -``` - -**Outbound Port (Cache):** - -```go -package outbound - -import ( - "context" - "time" -) - -type CacheRepository interface { - Get(ctx context.Context, key string) ([]byte, error) - Set(ctx context.Context, key string, value []byte, ttl time.Duration) error - Delete(ctx context.Context, key string) error - Exists(ctx context.Context, key string) (bool, error) - Clear(ctx context.Context) error -} -``` - -### 3. Services/Usecases (Business Logic) - -Implement inbound ports, depend on outbound ports: - -```go -package usecase - -import ( - "context" - "myapp/internal/core/domain" - "myapp/internal/core/port/inbound" - "myapp/internal/core/port/outbound" - "github.com/pkg/errors" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" -) - -var tracer = otel.Tracer("usecase") - -type userService struct { - userRepo outbound.UserRepository - auditRepo outbound.AuditLogRepository - cache outbound.CacheRepository -} - -func NewUserService( - userRepo outbound.UserRepository, - auditRepo outbound.AuditLogRepository, - cache outbound.CacheRepository, -) inbound.UserService { - return &userService{ - userRepo: userRepo, - auditRepo: auditRepo, - cache: cache, - } -} - -func (s *userService) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { - ctx, span := tracer.Start(ctx, "UserService.GetByNIP", - trace.WithAttributes(attribute.String("user.nip", nip)), - ) - defer span.End() - - // Try cache first - cacheKey := "user:" + nip - cached, err := s.cache.Get(ctx, cacheKey) - if err == nil && cached != nil { - span.AddEvent("cache_hit") - var user domain.User - if err := json.Unmarshal(cached, &user); err == nil { - return &user, nil - } - } - - span.AddEvent("cache_miss") - user, err := s.userRepo.GetByNIP(ctx, nip) - if err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "failed to get user") - } - - // Cache the result - if user != nil { - if data, err := json.Marshal(user); err == nil { - _ = s.cache.Set(ctx, cacheKey, data, 5*time.Minute) - } - } - - return user, nil -} -``` - -**Key Rules:** - -- Always accept context.Context as first parameter -- Return concrete errors wrapped with errors.Wrap() -- Constructor receives interfaces, returns interfaces -- Add tracing spans for observability -- Implement cache-aside pattern - -### 4. Inbound Adapters (HTTP Handlers) - -Implement HTTP interface, call services: - -```go -package v1 - -import ( - "net/http" - "myapp/internal/core/port/inbound" - "myapp/pkg/response" - "github.com/gin-gonic/gin" - "github.com/pkg/errors" -) - -type UserHandler struct { - svc inbound.UserService -} - -func NewUserHandler(svc inbound.UserService) *UserHandler { - return &UserHandler{svc: svc} -} - -func (h *UserHandler) GetByNIP(c *gin.Context) { - resp := response.Response{} - defer resp.Render(c) - - nip := c.Param("nip") - - user, err := h.svc.GetByNIP(c.Request.Context(), nip) - if err != nil { - resp.SetError(err, http.StatusInternalServerError) - return - } - if user == nil { - resp.SetError(errors.New("user not found"), http.StatusNotFound) - return - } - - resp.Data = user - resp.StatusCode = http.StatusOK -} - -func (h *UserHandler) RegisterRoutes(r *gin.RouterGroup) { - users := r.Group("/users") - { - users.GET("/:nip", h.GetByNIP) - } -} -``` - -**Key Rules:** - -- Use defer resp.Render(c) pattern for consistent responses -- Parse params/JSON using Gin's binding -- Convert HTTP concerns (params, headers) to domain types -- Call service methods with context - -### 5. Persistence Layer (pgx/pgxpool) - -**Pool Initialization:** - -```go -package persistence - -import ( - "context" - "fmt" - "time" - - "github.com/jackc/pgx/v5/pgxpool" -) - -type Pool struct { - *pgxpool.Pool -} - -func NewPool(ctx context.Context, dsn string, maxConns, minConns int) (*Pool, error) { - config, err := pgxpool.ParseConfig(dsn) - if err != nil { - return nil, fmt.Errorf("failed to parse config: %w", err) - } - - config.MaxConns = int32(maxConns) - config.MinConns = int32(minConns) - config.MaxConnLifetime = time.Hour - config.MaxConnIdleTime = 30 * time.Minute - config.HealthCheckPeriod = time.Minute - - pool, err := pgxpool.NewWithConfig(ctx, config) - if err != nil { - return nil, fmt.Errorf("failed to create pool: %w", err) - } - - // Verify connection - if err := pool.Ping(ctx); err != nil { - return nil, fmt.Errorf("failed to ping pool: %w", err) - } - - return &Pool{Pool: pool}, nil -} - -func (p *Pool) Close() { - p.Pool.Close() -} -``` - -**Repository Implementation (Native pgx):** - -```go -package persistence - -import ( - "context" - "encoding/json" - "myapp/internal/core/domain" - "myapp/internal/core/port/outbound" - - "github.com/jackc/pgx/v5" - "github.com/pkg/errors" -) - -type userRepository struct { - pool *Pool -} - -func NewUserRepository(pool *Pool) outbound.UserRepository { - return &userRepository{pool: pool} -} - -func (r *userRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { - ctx, span := tracer.Start(ctx, "UserRepository.GetByNIP") - defer span.End() - - var user domain.User - err := r.pool.QueryRow(ctx, - `SELECT nip, name, is_active, created_at, updated_at - FROM users WHERE nip = $1`, - nip, - ).Scan(&user.NIP, &user.Name, &user.IsActive, &user.CreatedAt, &user.UpdatedAt) - - if err == pgx.ErrNoRows { - span.AddEvent("user_not_found") - return nil, nil - } - if err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "failed to get user") - } - - span.SetAttributes(attribute.String("user.nip", user.NIP)) - return &user, nil -} - -func (r *userRepository) Create(ctx context.Context, user *domain.User) error { - ctx, span := tracer.Start(ctx, "UserRepository.Create") - defer span.End() - - err := r.pool.QueryRow(ctx, - `INSERT INTO users (nip, name, is_active) - VALUES ($1, $2, $3) - RETURNING created_at, updated_at`, - user.NIP, user.Name, user.IsActive, - ).Scan(&user.CreatedAt, &user.UpdatedAt) - - if err != nil { - span.RecordError(err) - return errors.Wrap(err, "failed to create user") - } - - return nil -} - -func (r *userRepository) List(ctx context.Context) ([]domain.User, error) { - ctx, span := tracer.Start(ctx, "UserRepository.List") - defer span.End() - - rows, err := r.pool.Query(ctx, - `SELECT nip, name, is_active, created_at, updated_at - FROM users ORDER BY created_at DESC`, - ) - if err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "failed to list users") - } - defer rows.Close() - - var users []domain.User - for rows.Next() { - var u domain.User - if err := rows.Scan(&u.NIP, &u.Name, &u.IsActive, &u.CreatedAt, &u.UpdatedAt); err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "failed to scan user") - } - users = append(users, u) - } - - span.SetAttributes(attribute.Int("users.count", len(users))) - return users, nil -} -``` - -**Key Rules:** - -- Use native pgx for all database operations -- Always include tracing spans -- Use positional parameters ($1, $2) -- Check for pgx.ErrNoRows for not found - -### 6. Cache Layer (PostgreSQL Unlogged Tables) - -**Cache Repository Interface:** - -```go -package cache - -import ( - "context" - "time" - - "myapp/internal/core/port/outbound" -) - -type PostgresCacheRepository struct { - pool *Pool -} - -func NewPostgresCacheRepository(pool *Pool) outbound.CacheRepository { - return &PostgresCacheRepository{pool: pool} -} - -func (r *PostgresCacheRepository) Get(ctx context.Context, key string) ([]byte, error) { - var value []byte - var expiresAt time.Time - - err := r.pool.QueryRow(ctx, - `SELECT value, expires_at FROM cache.entries - WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW())`, - key, - ).Scan(&value, &expiresAt) - - if err == pgx.ErrNoRows { - return nil, nil - } - if err != nil { - return nil, errors.Wrap(err, "failed to get cache") - } - - return value, nil -} - -func (r *PostgresCacheRepository) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error { - expiresAt := time.Now().Add(ttl) - - _, err := r.pool.Exec(ctx, - `INSERT INTO cache.entries (key, value, expires_at) - VALUES ($1, $2, $3) - ON CONFLICT (key) DO UPDATE SET value = $2, expires_at = $3`, - key, value, expiresAt, - ) - if err != nil { - return errors.Wrap(err, "failed to set cache") - } - - return nil -} - -func (r *PostgresCacheRepository) Delete(ctx context.Context, key string) error { - _, err := r.pool.Exec(ctx, - `DELETE FROM cache.entries WHERE key = $1`, - key, - ) - if err != nil { - return errors.Wrap(err, "failed to delete cache") - } - return nil -} - -func (r *PostgresCacheRepository) Exists(ctx context.Context, key string) (bool, error) { - var exists bool - err := r.pool.QueryRow(ctx, - `SELECT EXISTS( - SELECT 1 FROM cache.entries - WHERE key = $1 AND (expires_at IS NULL OR expires_at > NOW()) - )`, - key, - ).Scan(&exists) - if err != nil { - return false, errors.Wrap(err, "failed to check cache existence") - } - return exists, nil -} - -func (r *PostgresCacheRepository) Clear(ctx context.Context) error { - _, err := r.pool.Exec(ctx, `TRUNCATE cache.entries`) - if err != nil { - return errors.Wrap(err, "failed to clear cache") - } - return nil -} -``` - -**HA-Safe Cleanup with Advisory Locks:** - -```go -package cache - -import ( - "context" - "time" - - "github.com/jackc/pgx/v5" -) - -func (r *PostgresCacheRepository) CleanupExpired(ctx context.Context) (int64, error) { - ctx, span := tracer.Start(ctx, "CacheRepository.CleanupExpired") - defer span.End() - - // Try to acquire advisory lock (HA-safe - only one instance runs cleanup) - var acquired bool - err := r.pool.QueryRow(ctx, - `SELECT pg_try_advisory_lock(hashtext('cache_cleanup'))`, - ).Scan(&acquired) - - if err != nil { - span.RecordError(err) - return 0, errors.Wrap(err, "failed to acquire cleanup lock") - } - - if !acquired { - span.AddEvent("cleanup_lock_not_acquired") - return 0, nil // Another instance is running cleanup - } - - // Ensure lock is released when done - defer func() { - _, _ = r.pool.Exec(ctx, - `SELECT pg_advisory_unlock(hashtext('cache_cleanup'))`, - ) - }() - - span.AddEvent("cleanup_lock_acquired") - - // Perform cleanup - result, err := r.pool.Exec(ctx, - `DELETE FROM cache.entries WHERE expires_at < NOW()`, - ) - if err != nil { - span.RecordError(err) - return 0, errors.Wrap(err, "failed to cleanup expired entries") - } - - deleted := result.RowsAffected() - span.SetAttributes(attribute.Int64("cache.cleanup.deleted", deleted)) - span.AddEvent("cleanup_completed") - - return deleted, nil -} - -// StartCleanupScheduler starts a background goroutine for periodic cleanup -func (r *PostgresCacheRepository) StartCleanupScheduler(ctx context.Context, interval time.Duration) { - go func() { - ticker := time.NewTicker(interval) - defer ticker.Stop() - - for { - select { - case <-ctx.Done(): - return - case <-ticker.C: - deleted, err := r.CleanupExpired(context.Background()) - if err != nil { - log.Printf("cache cleanup error: %v", err) - } else if deleted > 0 { - log.Printf("cache cleanup: deleted %d entries", deleted) - } - } - } - }() -} -``` - -**Cache Schema Migration:** - -```sql --- db/migrations/cache/0001_cache_schema.up.sql -CREATE SCHEMA IF NOT EXISTS cache; - -CREATE UNLOGGED TABLE cache.entries ( - key TEXT PRIMARY KEY, - value BYTEA NOT NULL, - expires_at TIMESTAMP WITH TIME ZONE, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() -); - -CREATE INDEX idx_cache_expires ON cache.entries(expires_at) - WHERE expires_at IS NOT NULL; - --- db/migrations/cache/0001_cache_schema.down.sql -DROP TABLE IF EXISTS cache.entries; -DROP SCHEMA IF EXISTS cache; -``` - -**Why Unlogged Tables?** - -- No Write-Ahead Log (WAL) overhead = faster writes -- Data survives crashes but not unclean shutdowns (acceptable for cache) -- Auto-recovery on restart (cache rebuilds from source) -- Same PostgreSQL instance = simpler operations - ---- - -## JWT Authentication with Keycloak - -The backend acts as an **OAuth2 Resource Server** - it validates Bearer tokens from the UI/browser without needing to be an OIDC client. Token validation is done locally using JWKS fetched from Keycloak. - -### Architecture Overview - -``` -┌─────────┐ Bearer Token ┌──────────────┐ JWKS ┌────────────┐ -│ UI │ ───────────────────► │ Backend │ ◄─────────── │ Keycloak │ -│ Browser │ │ Resource Srv │ (cached) │ JWKS │ -└─────────┘ └──────────────┘ └────────────┘ - │ - ▼ - ┌──────────────┐ - │ PostgreSQL │ - │ (users) │ - └──────────────┘ -``` - -### Claims Mapping (Hardcoded) - -| JWT Claim | Field | Description | -|-----------|-------|-------------| -| `nip` | `NIP` | Primary key (e.g., "p021050") | -| `name` | `Name` | Full name | -| `email` | `Email` | Email (not unique, can be duplicated) | -| `resource_access.{client_id}.roles` | `Roles` | Authorization roles | - -### Domain Model - -```go -package domain - -import "time" - -type User struct { - NIP string `db:"nip" json:"nip"` - Name string `db:"name" json:"name"` - Email string `db:"email" json:"email"` - IsActive bool `db:"is_active" json:"is_active"` - LastLoginAt time.Time `db:"last_login_at" json:"last_login_at"` - CreatedAt time.Time `db:"created_at" json:"created_at"` - UpdatedAt time.Time `db:"updated_at" json:"updated_at"` - - Roles []string `db:"-" json:"roles,omitempty"` -} -``` - -### Inbound Port: Auth Service - -```go -package inbound - -import ( - "context" - "myapp/internal/core/domain" -) - -type AuthService interface { - Authenticate(ctx context.Context, token string) (*domain.User, error) - HasRole(user *domain.User, role string) bool - HasAnyRole(user *domain.User, roles ...string) bool -} -``` - -### Keycloak Provider Adapter - -```go -package keycloak - -import ( - "context" - "fmt" - "time" - - "github.com/MicahParks/keyfunc/v3" - "github.com/golang-jwt/jwt/v5" -) - -type Provider struct { - jwks *keyfunc.JWKS - clientID string -} - -type TokenClaims struct { - jwt.RegisteredClaims - - NIP string `json:"nip"` - Name string `json:"name"` - Email string `json:"email"` - - ResourceAccess map[string]struct { - Roles []string `json:"roles"` - } `json:"resource_access"` -} - -func NewProvider(ctx context.Context, jwksURL, clientID string) (*Provider, error) { - jwks, err := keyfunc.Get(jwksURL, keyfunc.Options{ - RefreshInterval: time.Hour, - RefreshRateLimit: time.Minute * 5, - }) - if err != nil { - return nil, fmt.Errorf("failed to fetch JWKS: %w", err) - } - - return &Provider{ - jwks: jwks, - clientID: clientID, - }, nil -} - -func (p *Provider) VerifyToken(tokenString string) (*TokenClaims, error) { - token, err := jwt.ParseWithClaims(tokenString, &TokenClaims{}, p.jwks.Keyfunc) - if err != nil { - return nil, fmt.Errorf("invalid token: %w", err) - } - - if !token.Valid { - return nil, fmt.Errorf("token is not valid") - } - - claims, ok := token.Claims.(*TokenClaims) - if !ok { - return nil, fmt.Errorf("failed to extract claims") - } - - return claims, nil -} - -func (p *Provider) ExtractRoles(claims *TokenClaims) []string { - if client, ok := claims.ResourceAccess[p.clientID]; ok { - return client.Roles - } - return nil -} -``` - -### Auth Usecase Implementation - -```go -package usecase - -import ( - "context" - "time" - - "myapp/internal/core/domain" - "myapp/internal/core/port/inbound" - "myapp/internal/core/port/outbound" - "myapp/pkg/keycloak" - - "github.com/pkg/errors" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" -) - -var tracer = otel.Tracer("usecase") - -type authService struct { - keycloakProvider *keycloak.Provider - userRepo outbound.UserRepository -} - -func NewAuthService(keycloakProvider *keycloak.Provider, userRepo outbound.UserRepository) inbound.AuthService { - return &authService{ - keycloakProvider: keycloakProvider, - userRepo: userRepo, - } -} - -func (s *authService) Authenticate(ctx context.Context, token string) (*domain.User, error) { - ctx, span := tracer.Start(ctx, "AuthService.Authenticate") - defer span.End() - - claims, err := s.keycloakProvider.VerifyToken(token) - if err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "token validation failed") - } - - span.SetAttributes(attribute.String("user.nip", claims.NIP)) - - if claims.NIP == "" { - return nil, errors.New("nip claim is required") - } - - user := &domain.User{ - NIP: claims.NIP, - Name: claims.Name, - Email: claims.Email, - Roles: s.keycloakProvider.ExtractRoles(claims), - } - - if err := s.syncUser(ctx, user); err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "failed to sync user") - } - - existing, err := s.userRepo.GetByNIP(ctx, user.NIP) - if err != nil { - span.RecordError(err) - return nil, errors.Wrap(err, "failed to get user") - } - - if existing == nil { - return nil, errors.New("user not found") - } - - if !existing.IsActive { - return nil, errors.New("user is deactivated") - } - - user.IsActive = existing.IsActive - user.LastLoginAt = existing.LastLoginAt - user.CreatedAt = existing.CreatedAt - user.UpdatedAt = existing.UpdatedAt - - span.SetAttributes(attribute.StringSlice("user.roles", user.Roles)) - - return user, nil -} - -func (s *authService) syncUser(ctx context.Context, user *domain.User) error { - existing, err := s.userRepo.GetByNIP(ctx, user.NIP) - if err != nil { - return err - } - - now := time.Now() - - if existing == nil { - user.IsActive = true - user.LastLoginAt = now - user.CreatedAt = now - user.UpdatedAt = now - return s.userRepo.Create(ctx, user) - } - - existing.Name = user.Name - existing.Email = user.Email - existing.LastLoginAt = now - existing.UpdatedAt = now - - return s.userRepo.Update(ctx, existing) -} - -func (s *authService) HasRole(user *domain.User, role string) bool { - for _, r := range user.Roles { - if r == role { - return true - } - } - return false -} - -func (s *authService) HasAnyRole(user *domain.User, roles ...string) bool { - for _, role := range roles { - if s.HasRole(user, role) { - return true - } - } - return false -} -``` - -### User Repository Extension - -```go -type UserRepository interface { - GetByNIP(ctx context.Context, nip string) (*domain.User, error) - Create(ctx context.Context, user *domain.User) error - Update(ctx context.Context, user *domain.User) error - UpdateLastLogin(ctx context.Context, nip string) error - List(ctx context.Context) ([]domain.User, error) -} -``` - ---- - -## Structured Logging with slog - -### Logger Package - -**pkg/logger/logger.go:** - -````go -package logger - -import ( - "context" - "io" - "log/slog" - "os" - "runtime" - "strings" - "sync" - "time" -) - -type Logger struct { - *slog.Logger - mu sync.Mutex - buildInfo BuildInfo -} - -type BuildInfo struct { - Version string - Commit string - GoVersion string -} - -var ( - defaultLogger *Logger - once sync.Once -) - -func Init(level, format, maskFields string, buildInfo BuildInfo) { - once.Do(func() { - var handler slog.Handler - var opts &slog.HandlerOptions - - // Parse level - var logLevel slog.Level - switch strings.ToLower(level) { - case "debug": - logLevel = slog.LevelDebug - case "warn", "warning": - logLevel = slog.LevelWarn - case "error": - logLevel = slog.LevelError - default: - logLevel = slog.LevelInfo - } - - opts = &slog.HandlerOptions{ - Level: logLevel, - AddSource: false, - ReplaceAttr: func(groups []string, a slog.Attr) slog.Attr { - // Add goroutine ID - if a.Key == "gid" { - a.Value = slog.StringValue(getGoroutineID()) - } - return a - }, - } - - if format == "json" { - handler = slog.NewJSONHandler(os.Stdout, opts) - } else { - handler = slog.NewTextHandler(os.Stdout, opts) - } - - defaultLogger = &Logger{ - Logger: slog.New(handler), - buildInfo: buildInfo, - } - }) -} - -func Default() *Logger { - if defaultLogger == nil { - Init("info", "json", "", BuildInfo{}) - } - return defaultLogger -} - -func getGoroutineID() string { - var buf [24]byte - n := runtime.Stack(buf[:], false) - idField := strings.Fields(string(buf[:n]))[1] - return idField -} - -// WithContext returns a logger with values extracted from context -func (l *Logger) WithContext(ctx context.Context) *Logger { - newLogger := &Logger{Logger: l.Logger, buildInfo: l.buildInfo} - - // Extract common values from context - if requestID := ctx.Value("request_id"); requestID != nil { - newLogger.Logger = newLogger.Logger.With("request_id", requestID) - } - if traceID := ctx.Value("trace_id"); traceID != nil { - newLogger.Logger = newLogger.Logger.With("trace_id", traceID) - } - - return newLogger -} - -// WithBuildInfo adds build information to all logs -func (l *Logger) WithBuildInfo() *Logger { - newLogger := &Logger{Logger: l.Logger, buildInfo: l.buildInfo} - newLogger.Logger = newLogger.Logger.With( - "version", l.buildInfo.Version, - "commit", l.buildInfo.Commit, - ) - return newLogger -} - -**pkg/logger/mask.go (Sensitive Data Masking):** - -```go -package logger - -import ( - "encoding/json" - "regexp" - "strings" - "sync" -) - -type Masker struct { - mu sync.RWMutex - sensitiveKeys map[string]bool - patterns []*regexp.Regexp -} - -var defaultMasker = &Masker{ - sensitiveKeys: map[string]bool{ - "password": true, - "passwd": true, - "token": true, - "api_key": true, - "apikey": true, - "api-key": true, - "secret": true, - "secret_key": true, - "access_token": true, - "refresh_token": true, - "authorization": true, - "auth": true, - "credential": true, - "private_key": true, - "private-key": true, - "x-api-key": true, - }, -} - -func SetSensitiveFields(fields []string) { - defaultMasker.mu.Lock() - defer defaultMasker.mu.Unlock() - - defaultMasker.sensitiveKeys = make(map[string]bool) - for _, f := range fields { - defaultMasker.sensitiveKeys[strings.ToLower(f)] = true - } -} - -// MaskJSON masks sensitive fields in JSON data -func MaskJSON(data []byte) []byte { - if len(data) == 0 { - return data - } - - var raw json.RawMessage - if err := json.Unmarshal(data, &raw); err != nil { - return data // Not valid JSON, return as-is - } - - masked, err := maskValue(raw) - if err != nil { - return data - } - - result, _ := json.Marshal(masked) - return result -} - -func maskValue(v interface{}) (interface{}, error) { - switch val := v.(type) { - case map[string]interface{}: - result := make(map[string]interface{}) - for k, v := range val { - if defaultMasker.isSensitive(k) { - result[k] = "***" - } else { - result[k] = v - } - } - return result, nil - - case []interface{}: - result := make([]interface{}, len(val)) - for i, v := range val { - result[i] = v - } - return result, nil - - default: - return v, nil - } -} - -func (m *Masker) isSensitive(key string) bool { - m.mu.RLock() - defer m.mu.RUnlock() - - keyLower := strings.ToLower(key) - if m.sensitiveKeys[keyLower] { - return true - } - - // Check patterns - for _, pattern := range m.patterns { - if pattern.MatchString(keyLower) { - return true - } - } - - return false -} - -// MaskString masks sensitive data in a string (e.g., "Bearer xyz123" -> "Bearer ***") -func MaskString(data string, patterns ...string) string { - result := data - for _, pattern := range patterns { - re := regexp.MustCompile(pattern) - result = re.ReplaceAllString(result, "***") - } - return result -} -```` - -### Request/Response Logging Middleware - -**middlewares/logging.go:** - -```go -package middlewares - -import ( - "bytes" - "encoding/json" - "io" - "time" - - "myapp/pkg/logger" - "myapp/pkg/metrics" - - "github.com/gin-gonic/gin" - "github.com/google/uuid" -) - -var skipBodyLoggingPaths = map[string]bool{ - "/v1/upload": true, - "/v1/files": true, - "/v1/media": true, - "/v1/bulk-import": true, - "/metrics": true, - "/healthz": true, -} - -type responseWriter struct { - gin.ResponseWriter - body *bytes.Buffer -} - -func (w *responseWriter) Write(b []byte) (int, error) { - w.body.Write(b) - return w.ResponseWriter.Write(b) -} - -func RequestLogger(log *logger.Logger, metrics *metrics.Prometheus) gin.HandlerFunc { - return func(c *gin.Context) { - start := time.Now() - - // Generate or extract request ID - requestID := c.GetHeader("X-Request-ID") - if requestID == "" { - requestID = uuid.New().String() - } - c.Set("request_id", requestID) - c.Header("X-Request-ID", requestID) - - // Extract trace ID for distributed tracing - traceID := c.GetHeader("X-Trace-ID") - if traceID != "" { - c.Set("trace_id", traceID) - } - - // Get logger with context - reqLog := log.WithContext(c.Request.Context()) - - // Read request body - var requestBody []byte - if !shouldSkipBodyLogging(c.Request.URL.Path) { - requestBody, _ = io.ReadAll(c.Request.Body) - c.Request.Body = io.NopCloser(bytes.NewBuffer(requestBody)) - } - - // Wrap response writer to capture response body - rw := &responseWriter{ - ResponseWriter: c.Writer, - body: bytes.NewBuffer(nil), - } - c.Writer = rw - - // Process request - c.Next() - - // Calculate duration - duration := time.Since(start) - - // Prepare log entry - logEntry := map[string]interface{}{ - "request_id": requestID, - "trace_id": traceID, - "method": c.Request.Method, - "path": c.Request.URL.Path, - "query": c.Request.URL.RawQuery, - "status": c.Writer.Status(), - "duration_ms": duration.Milliseconds(), - "client_ip": c.ClientIP(), - "user_agent": c.Request.UserAgent(), - "goroutine_id": getGoroutineID(), - } - - // Add user context if available - if nip, exists := c.Get("nip"); exists { - logEntry["user_nip"] = nip - } - if projectID, exists := c.Get("project_id"); exists { - logEntry["project_id"] = projectID - } - - // Add request/response bodies (masked) - if len(requestBody) > 0 && !shouldSkipBodyLogging(c.Request.URL.Path) { - logEntry["request_body"] = logger.MaskJSON(requestBody) - } - if rw.body.Len() > 0 && !shouldSkipBodyLogging(c.Request.URL.Path) { - logEntry["response_body"] = logger.MaskJSON(rw.body.Bytes()) - } - - // Log at appropriate level based on status code - status := c.Writer.Status() - if status >= 500 { - reqLog.Error("request completed with server error", logEntry) - } else if status >= 400 { - reqLog.Warn("request completed with client error", logEntry) - } else { - reqLog.Info("request completed", logEntry) - } - - // Record metrics - metrics.RecordHTTPRequest( - c.Request.Method, - c.FullPath(), - c.Writer.Status(), - duration, - len(requestBody), - rw.body.Len(), - ) - } -} - -func shouldSkipBodyLogging(path string) bool { - for skipPath := range skipBodyLoggingPaths { - if len(path) >= len(skipPath) && path[:len(skipPath)] == skipPath { - return true - } - } - return false -} -``` - -## Prometheus Metrics - -**pkg/metrics/prometheus.go:** - -```go -package metrics - -import ( - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promauto" -) - -type Prometheus struct { - HTTPRequests *prometheus.CounterVec - HTTPDuration *prometheus.HistogramVec - HTTPReqSize *prometheus.HistogramVec - HTTPResSize *prometheus.HistogramVec - ActiveConns prometheus.Gauge - CacheHits *prometheus.CounterVec - CacheMisses *prometheus.CounterVec - DBQueryDuration *prometheus.HistogramVec -} - -func NewPrometheus() *Prometheus { - return &Prometheus{ - HTTPRequests: promauto.NewCounterVec( - prometheus.CounterOpts{ - Name: "http_requests_total", - Help: "Total number of HTTP requests", - }, - []string{"method", "endpoint", "status"}, - ), - - HTTPDuration: promauto.NewHistogramVec( - prometheus.HistogramOpts{ - Name: "http_request_duration_seconds", - Help: "HTTP request duration in seconds", - Buckets: []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}, - }, - []string{"method", "endpoint", "status"}, - ), - - HTTPReqSize: promauto.NewHistogramVec( - prometheus.HistogramOpts{ - Name: "http_request_size_bytes", - Help: "HTTP request body size in bytes", - Buckets: []float64{100, 1000, 10000, 100000, 1000000}, - }, - []string{"method", "endpoint"}, - ), - - HTTPResSize: promauto.NewHistogramVec( - prometheus.HistogramOpts{ - Name: "http_response_size_bytes", - Help: "HTTP response body size in bytes", - Buckets: []float64{100, 1000, 10000, 100000, 1000000}, - }, - []string{"method", "endpoint"}, - ), - - ActiveConns: promauto.NewGauge( - prometheus.GaugeOpts{ - Name: "http_active_connections", - Help: "Number of active HTTP connections", - }, - ), - - CacheHits: promauto.NewCounterVec( - prometheus.CounterOpts{ - Name: "cache_hits_total", - Help: "Total number of cache hits", - }, - []string{"cache_type", "operation"}, - ), - - CacheMisses: promauto.NewCounterVec( - prometheus.CounterOpts{ - Name: "cache_misses_total", - Help: "Total number of cache misses", - }, - []string{"cache_type", "operation"}, - ), - - DBQueryDuration: promauto.NewHistogramVec( - prometheus.HistogramOpts{ - Name: "db_query_duration_seconds", - Help: "Database query duration in seconds", - Buckets: []float64{.001, .005, .01, .025, .05, .1, .25, .5, 1}, - }, - []string{"operation", "table"}, - ), - } -} - -func (p *Prometheus) RecordHTTPRequest(method, endpoint string, status int, duration interface{}, reqSize, resSize int) { - statusStr := statusCodeToString(status) - - p.HTTPRequests.WithLabelValues(method, endpoint, statusStr).Inc() - - // Handle duration as various types - var durationSec float64 - switch d := duration.(type) { - case float64: - durationSec = d - default: - durationSec = 0 - } - p.HTTPDuration.WithLabelValues(method, endpoint, statusStr).Observe(durationSec) - - p.HTTPReqSize.WithLabelValues(method, endpoint).Observe(float64(reqSize)) - p.HTTPResSize.WithLabelValues(method, endpoint).Observe(float64(resSize)) -} - -func (p *Prometheus) RecordCacheHit(cacheType, operation string) { - p.CacheHits.WithLabelValues(cacheType, operation).Inc() -} - -func (p *Prometheus) RecordCacheMiss(cacheType, operation string) { - p.CacheMisses.WithLabelValues(cacheType, operation).Inc() -} - -func (p *Prometheus) RecordDBQuery(operation, table string, duration float64) { - p.DBQueryDuration.WithLabelValues(operation, table).Observe(duration) -} - -func statusCodeToString(status int) string { - switch { - case status >= 500: - return "500" - case status >= 400: - return "400" - case status >= 300: - return "300" - case status >= 200: - return "200" - default: - return "unknown" - } -} -``` - -**Metrics Middleware:** - -```go -package middlewares - -import ( - "net/http" - "strconv" - "time" - - "myapp/pkg/metrics" - - "github.com/gin-gonic/gin" -) - -func PrometheusMiddleware(m *metrics.Prometheus) gin.HandlerFunc { - return func(c *gin.Context) { - // Track active connections - m.ActiveConns.Inc() - defer m.ActiveConns.Dec() - - start := time.Now() - - c.Next() - - // Record metrics after request is processed - duration := time.Since(start).Seconds() - status := c.Writer.Status() - - m.HTTPRequests.WithLabelValues( - c.Request.Method, - c.FullPath(), - strconv.Itoa(status), - ).Inc() - - m.HTTPDuration.WithLabelValues( - c.Request.Method, - c.FullPath(), - strconv.Itoa(status), - ).Observe(duration) - } -} -``` - ---- - -## OpenTelemetry Tracing with Tempo - -**pkg/tracing/otel.go:** - -```go -package tracing - -import ( - "context" - "fmt" - - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp" - "go.opentelemetry.io/otel/propagation" - "go.opentelemetry.io/otel/sdk/resource" - "go.opentelemetry.io/otel/sdk/trace" - semconv "go.opentelemetry.io/otel/semconv/v1.21.0" -) - -type TracerProvider struct { - *trace.TracerProvider -} - -func NewTracerProvider(ctx context.Context, endpoint, serviceName, serviceVersion string) (*TracerProvider, error) { - // Create OTLP exporter - exporter, err := otlptracehttp.New(ctx, - otlptracehttp.WithEndpoint(endpoint), - otlptracehttp.WithInsecure(), // Use OTEL_EXPORTER_OTLP_ENDPOINT with https for TLS - ) - if err != nil { - return nil, fmt.Errorf("failed to create exporter: %w", err) - } - - // Create resource with service info - res, err := resource.New(ctx, - resource.WithAttributes( - semconv.ServiceName(serviceName), - semconv.ServiceVersion(serviceVersion), - ), - ) - if err != nil { - return nil, fmt.Errorf("failed to create resource: %w", err) - } - - // Create tracer provider with adaptive sampler - sampler := NewAdaptiveSampler(0.1) // 10% sampling rate - tp := trace.NewTracerProvider( - trace.WithBatcher(exporter), - trace.WithResource(res), - trace.WithSampler(sampler), - ) - - // Set global tracer provider - otel.SetTracerProvider(tp) - - // Set global propagator (W3C Trace Context) - otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator( - propagation.TraceContext{}, - propagation.Baggage{}, - )) - - return &TracerProvider{TracerProvider: tp}, nil -} - -func (tp *TracerProvider) Shutdown(ctx context.Context) error { - return tp.TracerProvider.Shutdown(ctx) -} -``` - -**pkg/tracing/sampler.go (Adaptive Sampling):** - -```go -package tracing - -import ( - "math/rand" - - "go.opentelemetry.io/otel/sdk/trace" -) - -type AdaptiveSampler struct { - sampleRate float64 - parentBased bool -} - -func NewAdaptiveSampler(sampleRate float64) *AdaptiveSampler { - return &AdaptiveSampler{ - sampleRate: sampleRate, - parentBased: true, - } -} - -func (s *AdaptiveSampler) ShouldSample(p trace.SamplingParameters) trace.SamplingResult { - // Always sample if parent is sampled - if s.parentBased && p.Parent != nil { - if p.Parent.IsSampled() { - return trace.SamplingResult{Decision: trace.RecordAndSample } - } - return trace.SamplingResult{Decision: trace.Drop } - } - - // Adaptive sampling based on error rate - // If it's an error, always sample (higher probability) - if containsError(p.Attributes) { - return trace.SamplingResult{ - Decision: trace.RecordAndSample, - Tracestate: p.Tracestate, - } - } - - // Random sampling based on sample rate - if rand.Float64() < s.sampleRate { - return trace.SamplingResult{ - Decision: trace.RecordAndSample, - Tracestate: p.Tracestate, - } - } - - return trace.SamplingResult{ - Decision: trace.Drop, - Tracestate: p.Tracestate, - } -} - -func (s *AdaptiveSampler) Description() string { - return "AdaptiveSampler" -} - -func containsError(attrs []attribute.KeyValue) bool { - for _, attr := range attrs { - if attr.Key == "error" && attr.Value.AsBool() { - return true - } - if attr.Key == "http.status_code" && attr.Value.AsInt64() >= 500 { - return true - } - } - return false -} -``` - -**Tracing Middleware:** - -```go -package middlewares - -import ( - "github.com/gin-gonic/gin" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/propagation" - "go.opentelemetry.io/otel/trace" -) - -func TracingMiddleware(serviceName string) gin.HandlerFunc { - tracer := otel.Tracer(serviceName) - - return func(c *gin.Context) { - // Extract trace context from incoming request - ctx := otel.GetTextMapPropagator().Extract(c.Request.Context(), propagation.HeaderCarrier(c.Request.Header)) - - // Generate or use existing span name - spanName := c.Request.Method + " " + c.FullPath() - - // Start span with span kind server - ctx, span := tracer.Start(ctx, spanName, - trace.WithSpanKind(trace.SpanKindServer), - trace.WithAttributes( - attribute.String("http.method", c.Request.Method), - attribute.String("http.url", c.Request.URL.String()), - attribute.String("http.route", c.FullPath()), - attribute.String("http.host", c.Request.Host), - attribute.String("http.user_agent", c.Request.UserAgent()), - attribute.String("net.peer.ip", c.ClientIP()), - ), - ) - defer span.End() - - // Store span in context for downstream use - c.Request = c.Request.WithContext(ctx) - - // Process request - c.Next() - - // Add response attributes - span.SetAttributes( - attribute.Int("http.status_code", c.Writer.Status()), - attribute.Int("http.response_size", c.Writer.Size()), - ) - - // Add error if present - if len(c.Errors) > 0 { - span.SetAttributes(attribute.Bool("error", true)) - span.RecordError(c.Errors.Last().Err) - } - } -} -``` - -**Usage in Business Logic:** - -```go -func (s *userService) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { - ctx, span := tracer.Start(ctx, "UserService.GetByNIP", - trace.WithAttributes( - attribute.String("user.nip", nip), - attribute.String("operation", "get_user"), - ), - ) - defer span.End() - - span.AddEvent("Validating input") - - if nip == "" { - err := errors.New("nip is required") - span.RecordError(err) - span.SetStatus(codes.Error, "validation failed") - return nil, err - } - - span.AddEvent("Querying database") - user, err := s.userRepo.GetByNIP(ctx, nip) - if err != nil { - span.RecordError(err) - span.SetStatus(codes.Error, "database error") - return nil, err - } - - if user == nil { - span.AddEvent("User not found") - span.SetStatus(codes.Ok, "not found") - return nil, nil - } - - span.AddEvent("User found", - trace.WithAttributes( - attribute.String("user.name", user.Name), - ), - ) - span.SetStatus(codes.Ok, "success") - - return user, nil -} -``` - -### Auth Middleware - -**middlewares/auth.go:** - -```go -package middlewares - -import ( - "net/http" - "strings" - - "myapp/internal/core/domain" - "myapp/internal/core/port/inbound" - - "github.com/gin-gonic/gin" -) - -const UserContextKey = "authenticated_user" - -func AuthMiddleware(authSvc inbound.AuthService) gin.HandlerFunc { - return func(c *gin.Context) { - authHeader := c.GetHeader("Authorization") - if authHeader == "" { - c.AbortWithStatusJSON(http.StatusUnauthorized, - gin.H{"error": "missing authorization header"}) - return - } - - parts := strings.SplitN(authHeader, " ", 2) - if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") { - c.AbortWithStatusJSON(http.StatusUnauthorized, - gin.H{"error": "invalid authorization format"}) - return - } - - user, err := authSvc.Authenticate(c.Request.Context(), parts[1]) - if err != nil { - c.AbortWithStatusJSON(http.StatusUnauthorized, - gin.H{"error": err.Error()}) - return - } - - c.Set(UserContextKey, user) - c.Next() - } -} - -func RequireRole(authSvc inbound.AuthService, roles ...string) gin.HandlerFunc { - return func(c *gin.Context) { - user := GetUserFromContext(c) - if user == nil { - c.AbortWithStatusJSON(http.StatusUnauthorized, - gin.H{"error": "unauthenticated"}) - return - } - - for _, role := range roles { - if authSvc.HasRole(user, role) { - c.Next() - return - } - } - - c.AbortWithStatusJSON(http.StatusForbidden, - gin.H{"error": "insufficient permissions"}) - } -} - -func GetUserFromContext(c *gin.Context) *domain.User { - if user, exists := c.Get(UserContextKey); exists { - if u, ok := user.(*domain.User); ok { - return u - } - } - return nil -} -``` - ---- - -## Server Wiring (Dependency Injection) - -**cmd/server/main.go:** - -```go -package server - -import ( - "context" - "fmt" - "log" - "net/http" - "os" - "os/signal" - "strings" - "syscall" - "time" - - "myapp/config" - "myapp/internal/adapter/inbound/rest/v1" - "myapp/internal/adapter/outbound/cache" - "myapp/internal/adapter/outbound/persistence" - "myapp/internal/core/usecase" - "myapp/middlewares" - "myapp/pkg/grace" - "myapp/pkg/logger" - "myapp/pkg/metrics" - "myapp/pkg/tracing" - - "github.com/gin-contrib/cors" - "github.com/gin-gonic/gin" - "github.com/prometheus/client_golang/prometheus/promhttp" - "golang.org/x/sync/errgroup" -) - -func Run(ctx context.Context) error { - // Load configuration - cfg, err := config.Init() - if err != nil { - return fmt.Errorf("failed to load config: %w", err) - } - - // Initialize logger - buildInfo := logger.BuildInfo{ - Version: getEnv("VERSION", "dev"), - Commit: getEnv("COMMIT", "local"), - GoVersion: getEnv("GO_VERSION", ""), - } - logger.Init(cfg.Log.Level, cfg.Log.Format, cfg.Log.MaskFields, buildInfo) - log := logger.Default() - - log.Info("Starting application", - "version", buildInfo.Version, - "commit", buildInfo.Commit, - ) - - // Initialize tracing - if cfg.Tempo.Enabled { - tp, err := tracing.NewTracerProvider( - ctx, - cfg.Tempo.Endpoint, - "messaging-be", - buildInfo.Version, - ) - if err != nil { - log.Warn("Failed to initialize tracing", "error", err) - } else { - defer tp.Shutdown(ctx) - log.Info("Tracing initialized", "endpoint", cfg.Tempo.Endpoint) - } - } - - // Initialize persistence pool (main database) - persistencePool, err := persistence.NewPool( - ctx, - cfg.Persistence.DSN, - cfg.Persistence.MaxConns, - cfg.Persistence.MinConns, - ) - if err != nil { - return fmt.Errorf("failed to connect to persistence: %w", err) - } - defer persistencePool.Close() - log.Info("Persistence pool established") - - // Initialize cache pool (separate schema) - cachePool, err := persistence.NewPool( - ctx, - cfg.Cache.DSN, - cfg.Cache.MaxConns, - cfg.Cache.MinConns, - ) - if err != nil { - return fmt.Errorf("failed to connect to cache: %w", err) - } - defer cachePool.Close() - log.Info("Cache pool established") - - // Initialize repositories - userRepo := persistence.NewUserRepository(persistencePool) - auditRepo := persistence.NewAuditLogRepository(persistencePool) - cacheRepo := cache.NewPostgresCacheRepository(cachePool) - - // Start cache cleanup scheduler (HA-safe with advisory locks) - go cacheRepo.StartCleanupScheduler(ctx, cfg.Cache.CleanupInterval) - - // Initialize metrics - promMetrics := metrics.NewPrometheus() - - // Initialize services - userSvc := usecase.NewUserService(userRepo, auditRepo, cacheRepo) - - // Initialize handlers - userHandler := v1.NewUserHandler(userSvc) - - // Setup Gin - gin.SetMode(cfg.App.Mode) - r := gin.New() - r.Use(gin.Recovery()) - - // CORS - r.Use(cors.New(cors.Config{ - AllowOrigins: strings.Split(cfg.App.AllowedOrigins, ","), - AllowMethods: []string{"GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"}, - AllowHeaders: []string{"Origin", "Content-Type", "Authorization", "X-API-Key", "X-Request-ID", "X-Trace-ID"}, - ExposeHeaders: []string{"Content-Length", "X-Request-ID"}, - AllowCredentials: true, - MaxAge: 12 * time.Hour, - })) - - // Global middlewares - r.Use(middlewares.TracingMiddleware("messaging-be")) - r.Use(middlewares.PrometheusMiddleware(promMetrics)) - r.Use(middlewares.RequestLogger(log, promMetrics)) - - // Health check - r.GET("/healthz", func(c *gin.Context) { - c.JSON(200, gin.H{"status": "ok"}) - }) - - // Metrics endpoint - r.GET("/metrics", gin.WrapH(promhttp.Handler())) - - // API routes - v1 := r.Group("/v1") - userHandler.RegisterRoutes(v1) - - // Graceful shutdown - port := cfg.App.Port - if !strings.HasPrefix(port, ":") { - port = ":" + port - } - - return grace.Serve(port, r.Handler()) -} - -func getEnv(key, defaultVal string) string { - if val := os.Getenv(key); val != "" { - return val - } - return defaultVal -} -``` - ---- - -## Provider Pattern (Option B) - -Use a Provider struct to wire all dependencies explicitly: - -**internal/provider/provider.go:** - -```go -package provider - -import ( - "context" - "fmt" - - "myapp/config" - "myapp/internal/adapter/outbound/cache" - "myapp/internal/adapter/outbound/keycloak" - "myapp/internal/adapter/outbound/persistence" - "myapp/internal/core/port/inbound" - "myapp/internal/core/usecase" - "myapp/pkg/logger" - "myapp/pkg/metrics" - "myapp/pkg/tracing" -) - -type Provider struct { - // Config - Config *config.Config - - // Infrastructure - Logger *logger.Logger - Metrics *metrics.Prometheus - Tracer *tracing.TracerProvider - - // Persistence - PersistencePool *persistence.Pool - CachePool *persistence.Pool - - // Repositories - UserRepo outbound.UserRepository - CacheRepo outbound.CacheRepository - - // External Services - Keycloak *keycloak.Provider - - // Services - AuthSvc inbound.AuthService - UserSvc inbound.UserService -} - -func NewProvider(ctx context.Context, cfg *config.Config) (*Provider, error) { - p := &Provider{Config: cfg} - - // 1. Infrastructure - if err := p.initLogger(); err != nil { - return nil, fmt.Errorf("logger: %w", err) - } - - if err := p.initMetrics(); err != nil { - return nil, fmt.Errorf("metrics: %w", err) - } - - if err := p.initTracer(ctx); err != nil { - return nil, fmt.Errorf("tracer: %w", err) - } - - // 2. Persistence - if err := p.initPersistence(ctx); err != nil { - return nil, fmt.Errorf("persistence: %w", err) - } - - if err := p.initCache(ctx); err != nil { - return nil, fmt.Errorf("cache: %w", err) - } - - // 3. External Services - if err := p.initKeycloak(ctx); err != nil { - return nil, fmt.Errorf("keycloak: %w", err) - } - - // 4. Repositories - p.UserRepo = persistence.NewUserRepository(p.PersistencePool) - p.CacheRepo = cache.NewPostgresCacheRepository(p.CachePool) - - // 5. Services - p.AuthSvc = usecase.NewAuthService(p.Keycloak, p.UserRepo) - p.UserSvc = usecase.NewUserService(p.UserRepo, p.CacheRepo) - - return p, nil -} - -func (p *Provider) initLogger() error { - buildInfo := logger.BuildInfo{ - Version: p.Config.Build.Version, - Commit: p.Config.Build.Commit, - } - logger.Init(p.Config.Log.Level, p.Config.Log.Format, p.Config.Log.MaskFields, buildInfo) - return nil -} - -func (p *Provider) initMetrics() error { - p.Metrics = metrics.NewPrometheus() - return nil -} - -func (p *Provider) initTracer(ctx context.Context) error { - if !p.Config.Tempo.Enabled { - return nil - } - - tp, err := tracing.NewTracerProvider( - ctx, - p.Config.Tempo.Endpoint, - "backend", - p.Config.Build.Version, - ) - if err != nil { - return err - } - p.Tracer = tp - return nil -} - -func (p *Provider) initPersistence(ctx context.Context) error { - pool, err := persistence.NewPool( - ctx, - p.Config.Persistence.DSN, - p.Config.Persistence.MaxConns, - p.Config.Persistence.MinConns, - ) - if err != nil { - return err - } - p.PersistencePool = pool - return nil -} - -func (p *Provider) initCache(ctx context.Context) error { - pool, err := persistence.NewPool( - ctx, - p.Config.Cache.DSN, - p.Config.Cache.MaxConns, - p.Config.Cache.MinConns, - ) - if err != nil { - return err - } - p.CachePool = pool - return nil -} - -func (p *Provider) initKeycloak(ctx context.Context) error { - prov, err := keycloak.NewProvider(ctx, p.Config.Keycloak.JWKSURL, p.Config.Keycloak.ClientID) - if err != nil { - return err - } - p.Keycloak = prov - return nil -} - -func (p *Provider) Close(ctx context.Context) { - if p.PersistencePool != nil { - p.PersistencePool.Close() - } - if p.CachePool != nil { - p.CachePool.Close() - } - if p.Tracer != nil { - p.Tracer.Shutdown(ctx) - } -} -``` - -**Usage in cmd/server/main.go:** - -```go -func Run(ctx context.Context) error { - cfg, err := config.Load(ctx) - if err != nil { - return fmt.Errorf("config: %w", err) - } - - provider, err := provider.NewProvider(ctx, cfg) - if err != nil { - return fmt.Errorf("provider: %w", err) - } - defer provider.Close(ctx) - - // Setup Gin with provider components - gin.SetMode(cfg.App.Mode) - r := gin.New() - r.Use(gin.Recovery()) - - // CORS - r.Use(cors.New(cors.Config{ - AllowOrigins: strings.Split(cfg.App.AllowedOrigins, ","), - AllowMethods: []string{"GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"}, - AllowHeaders: []string{"Origin", "Content-Type", "Authorization", "X-Request-ID", "X-Trace-ID"}, - ExposeHeaders: []string{"Content-Length", "X-Request-ID"}, - AllowCredentials: true, - MaxAge: 12 * time.Hour, - })) - - // Middlewares - r.Use(middlewares.TracingMiddleware("backend")) - r.Use(middlewares.PrometheusMiddleware(provider.Metrics)) - r.Use(middlewares.RequestLogger(logger.Default(), provider.Metrics)) - r.Use(middlewares.AuthMiddleware(provider.AuthSvc)) - - // Routes - v1 := r.Group("/v1") - v1.GET("/profile", userHandler.GetProfile) - - return grace.Serve(cfg.App.Port, r.Handler()) -} -``` - ---- - -## Configuration - -**config/model.go:** - -```go -package config - -type AppConfig struct { - Mode string `env:"APP_MODE" default:"release"` - Port string `env:"APP_PORT" default:"8080"` - AllowedOrigins string `env:"APP_ALLOWED_ORIGINS" default:"*"` -} - -type PersistenceConfig struct { - DSN string `env:"PERSISTENCE_DSN" required:"true"` - MaxConns int `env:"PERSISTENCE_MAX_CONNS" default:"25"` - MinConns int `env:"PERSISTENCE_MIN_CONNS" default:"5"` -} - -type CacheConfig struct { - DSN string `env:"CACHE_DSN" required:"true"` - MaxConns int `env:"CACHE_MAX_CONNS" default:"10"` - MinConns int `env:"CACHE_MIN_CONNS" default:"2"` - DefaultTTL time.Duration `env:"CACHE_DEFAULT_TTL" default:"5m"` - CleanupInterval time.Duration `env:"CACHE_CLEANUP_INTERVAL" default:"10m"` -} - -type LogConfig struct { - Level string `env:"LOG_LEVEL" default:"info"` - Format string `env:"LOG_FORMAT" default:"json"` // json or text - MaskFields string `env:"LOG_MASK_FIELDS" default:"password,token,api_key,secret,authorization"` -} - -type TempoConfig struct { - Enabled bool `env:"TEMPO_ENABLED" default:"false"` - Endpoint string `env:"TEMPO_ENDPOINT" default:"tempo:4318"` - SampleRate float64 `env:"TEMPO_SAMPLE_RATE" default:"0.1"` -} - -type KeycloakConfig struct { - JWKSURL string `env:"KEYCLOAK_JWKS_URL" required:"true"` - ClientID string `env:"KEYCLOAK_CLIENT_ID" required:"true"` -} - -type OpenBaoConfig struct { - Addr string `env:"OPENBAO_ADDR" required:"true"` - Token string `env:"OPENBAO_TOKEN" required:"true"` - MountPath string `env:"OPENBAO_MOUNT_PATH" default:"secret"` - SecretPath string `env:"OPENBAO_SECRET_PATH" required:"true"` // e.g., "backend/production" -} - -type Config struct { - App AppConfig - Persistence PersistenceConfig - Cache CacheConfig - Log LogConfig - Tempo TempoConfig - Keycloak KeycloakConfig - OpenBao OpenBaoConfig -} -``` - -**Environment Variables (.env):** - -```env -# Application -APP_ENV=local # local, staging, production -APP_MODE=release -APP_PORT=8080 -APP_ALLOWED_ORIGINS=* - -# Persistence (Main Database) -PERSISTENCE_DSN=postgres://user:pass@localhost:5432/messaging?sslmode=disable -PERSISTENCE_MAX_CONNS=25 -PERSISTENCE_MIN_CONNS=5 - -# Cache (Same instance, separate schema) -CACHE_DSN=postgres://user:pass@localhost:5432/messaging?sslmode=disable&search_path=cache -CACHE_MAX_CONNS=10 -CACHE_MIN_CONNS=2 -CACHE_DEFAULT_TTL=5m -CACHE_CLEANUP_INTERVAL=10m - -# Logging -LOG_LEVEL=info -LOG_FORMAT=json -LOG_MASK_FIELDS=password,token,api_key,secret,authorization - -# Tracing -TEMPO_ENABLED=true -TEMPO_ENDPOINT=tempo:4318 -TEMPO_SAMPLE_RATE=0.1 - -# Keycloak (JWT Authentication) -KEYCLOAK_JWKS_URL=https://auth.pharos.id/realms/production/protocol/openid-connect/certs -KEYCLOAK_CLIENT_ID=exodus - -# OpenBao (secrets) - Only used when APP_ENV != local -OPENBAO_ADDR=https://openbao.pharos.id -OPENBAO_TOKEN=your-token-here -OPENBAO_MOUNT_PATH=secret -OPENBAO_SECRET_PATH=backend/production - -# Build Info -VERSION=1.0.0 -COMMIT=$(git rev-parse --short HEAD) -GO_VERSION=$(go version) -``` - -**config/loader.go (OpenBao + Env Fallback):** - -```go -package config - -import ( - "context" - "os" - - "github.com/hashicorp/vault/v2/api" - "github.com/golobby/env/v2" - "github.com/golobby/dotenv" -) - -func Load(ctx context.Context) (*Config, error) { - env := os.Getenv("APP_ENV") - if env == "" { - env = "local" - } - - if env == "local" { - // Local dev: use .env file - dotenv.Load() - return loadFromEnv() - } - - // Staging/production: use OpenBao v2 - return loadFromOpenBao(ctx) -} - -func loadFromEnv() (*Config, error) { - var cfg Config - if err := env.Parse(&cfg); err != nil { - return nil, err - } - return &cfg, nil -} - -func loadFromOpenBao(ctx context.Context) (*Config, error) { - var cfg Config - if err := env.Parse(&cfg.OpenBao); err != nil { - return nil, err - } - - // Fetch secrets from OpenBao v2 KV - client, err := api.NewClient(&api.Config{ - Address: cfg.OpenBao.Addr, - }) - if err != nil { - return nil, err - } - client.SetToken(cfg.OpenBao.Token) - - // Path: {mount_path}/data/{secret_path} - secret, err := client.KVv2(cfg.OpenBao.MountPath).Get(ctx, cfg.OpenBao.SecretPath) - if err != nil { - return nil, err - } - - // Merge secrets into config - data := secret.Data["data"].(map[string]interface{}) - - // Apply OpenBao secrets to config fields - // This maps secrets to config struct fields - if dsn, ok := data["PERSISTENCE_DSN"].(string); ok { - cfg.Persistence.DSN = dsn - } - if jwksURL, ok := data["KEYCLOAK_JWKS_URL"].(string); ok { - cfg.Keycloak.JWKSURL = jwksURL - } - // ... map other secrets - - return &cfg, nil -} -``` - ---- - -## Database Migrations - -**Persistence Schema:** - -```sql --- db/migrations/persistence/0001_initial_schema.up.sql -CREATE TABLE IF NOT EXISTS users ( - nip VARCHAR(20) PRIMARY KEY, -- From JWT "nip" claim - name VARCHAR(255) NOT NULL, -- From JWT "name" claim - email VARCHAR(255), -- From JWT "email" claim (not unique) - is_active BOOLEAN DEFAULT true, - last_login_at TIMESTAMP WITH TIME ZONE, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), - updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() -); - -CREATE INDEX idx_users_email ON users(email); -CREATE INDEX idx_users_is_active ON users(is_active); +## Quick Start --- db/migrations/persistence/0001_initial_schema.down.sql -DROP TABLE IF EXISTS users; -``` +```bash +# 1. Install dependencies +go mod download -**Cache Schema:** +# 2. Configure environment +cp .env.example .env +# Edit .env with your settings -```sql --- db/migrations/cache/0001_cache_schema.up.sql -CREATE SCHEMA IF NOT EXISTS cache; +# 3. Build +make build -CREATE UNLOGGED TABLE cache.entries ( - key TEXT PRIMARY KEY, - value BYTEA NOT NULL, - expires_at TIMESTAMP WITH TIME ZONE, - created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() -); +# 4. Run server (default) +./bin/messaging-be -CREATE INDEX idx_cache_expires ON cache.entries(expires_at) - WHERE expires_at IS NOT NULL; +# 5. Or run admin commands +./bin/messaging-be admin migrate up +./bin/messaging-be admin user list --- db/migrations/cache/0001_cache_schema.down.sql -DROP TABLE IF EXISTS cache.entries; -DROP SCHEMA IF EXISTS cache; +# 6. Verify +curl http://localhost:8080/healthz ``` -**Why Unlogged Tables for Cache?** - -- No Write-Ahead Log (WAL) overhead - 5-10x faster writes -- Data survives crashes but not unclean shutdowns -- Acceptable for cache (rebuilds from source on restart) -- Simpler operations (same PostgreSQL instance) - --- -## Naming Conventions +## Documentation Index -| Element | Convention | Example | -| ---------------- | --------------------------------------------- | ------------------------------- | -| Packages | lowercase, single word | domain, usecase, persistence | -| Structs | PascalCase | UserService, ProjectHandler | -| Interfaces | PascalCase | UserRepository, CacheRepository | -| Functions | PascalCase (exported), camelCase (unexported) | NewUserService, getByNIP | -| Variables | camelCase | projectID, isActive | -| Constants | PascalCase | MaxRetries, DefaultTimeout | -| Database columns | snake_case | project_id, created_at | -| JSON fields | snake_case | project_id, channel_type | -| Environment vars | UPPER_SNAKE | PERSISTENCE_DSN, LOG_LEVEL | -| Schema names | snake_case | cache, persistence | - ---- - -## Import Grouping - -Group imports with blank line between groups: - -```go -import ( - // Standard library - "context" - "encoding/json" - "fmt" - "time" - - // Third-party packages - "github.com/gin-gonic/gin" - "github.com/jackc/pgx/v5" - "github.com/pkg/errors" - "github.com/prometheus/client_golang/prometheus" - - // Internal packages - "myapp/internal/core/domain" - "myapp/internal/core/port/inbound" - "myapp/internal/core/port/outbound" -) -``` +| File | Contents | +|------|----------| +| [ARCHITECTURE.md](ARCHITECTURE.md) | Hexagonal patterns, DDD, SOLID principles, provider | +| [12-FACTOR.md](12-FACTOR.md) | 12-factor methodology compliance | +| [CONFIGURATION.md](CONFIGURATION.md) | OpenBao secrets, env loading, provider wiring | +| [AUTHENTICATION.md](AUTHENTICATION.md) | Keycloak JWT, resource server, auth middleware | +| [DATABASE.md](DATABASE.md) | PostgreSQL schema, migrations, caching | +| [DEPLOYMENT.md](DEPLOYMENT.md) | Docker, CI/CD, admin CLI, health endpoint | +| [CODING-STANDARDS.md](CODING-STANDARDS.md) | REST standards, SOLID, error handling, testing | --- -## Error Handling Patterns - -### Error Wrapping - -Use `github.com/pkg/errors` for error wrapping: - -```go -// Wrap errors with context -func (r *userRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { - var user domain.User - err := r.pool.QueryRow(ctx, - "SELECT nip, name FROM users WHERE nip = $1", nip, - ).Scan(&user.NIP, &user.Name) - - if errors.Is(err, pgx.ErrNoRows) { - return nil, nil // Not found = nil, nil - } - if err != nil { - return nil, errors.Wrap(err, "failed to get user") - } - return &user, nil -} - -// Formatted errors -return nil, errors.Errorf("user %s not found", nip) -``` - -### Response Error Pattern - -```go -func (h *UserHandler) GetByNIP(c *gin.Context) { - resp := response.Response{} - defer resp.Render(c) - - var input struct { - NIP string `uri:"nip" binding:"required"` - } - - if err := c.ShouldBindUri(&input); err != nil { - resp.SetError(err, http.StatusBadRequest) - return - } - - user, err := h.svc.GetByNIP(c.Request.Context(), input.NIP) - if err != nil { - resp.SetError(err, http.StatusInternalServerError) - return - } +## Project Structure - resp.Data = user - resp.StatusCode = http.StatusOK -} ``` - ---- - -## Testing Patterns - -### Unit Test with Mocks - -```go -package usecase - -import ( - "context" - "testing" - - "myapp/internal/core/domain" - "myapp/internal/core/port/outbound" -) - -type mockUserRepository struct { - getByNIPFunc func(ctx context.Context, nip string) (*domain.User, error) - createFunc func(ctx context.Context, user *domain.User) error - listFunc func(ctx context.Context) ([]domain.User, error) -} - -func (m *mockUserRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { - if m.getByNIPFunc != nil { - return m.getByNIPFunc(ctx, nip) - } - return nil, nil -} - -func (m *mockUserRepository) Create(ctx context.Context, user *domain.User) error { - if m.createFunc != nil { - return m.createFunc(ctx, user) - } - return nil -} - -func (m *mockUserRepository) List(ctx context.Context) ([]domain.User, error) { - if m.listFunc != nil { - return m.listFunc(ctx) - } - return nil, nil -} - -type mockCacheRepository struct { - getFunc func(ctx context.Context, key string) ([]byte, error) - setFunc func(ctx context.Context, key string, value []byte, ttl time.Duration) error -} - -func (m *mockCacheRepository) Get(ctx context.Context, key string) ([]byte, error) { - if m.getFunc != nil { - return m.getFunc(ctx, key) - } - return nil, nil -} - -func (m *mockCacheRepository) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error { - if m.setFunc != nil { - return m.setFunc(ctx, key, value, ttl) - } - return nil -} - -func (m *mockCacheRepository) Delete(ctx context.Context, key string) error { return nil } -func (m *mockCacheRepository) Exists(ctx context.Context, key string) (bool, error) { return false, nil } -func (m *mockCacheRepository) Clear(ctx context.Context) error { return nil } - -func TestUserService_GetByNIP(t *testing.T) { - tests := []struct { - name string - nip string - setupMock func(*mockUserRepository, *mockCacheRepository) - expectError bool - }{ - { - name: "user found in cache", - nip: "12345", - setupMock: func(ur *mockUserRepository, cr *mockCacheRepository) { - cr.getFunc = func(ctx context.Context, key string) ([]byte, error) { - return json.Marshal(&domain.User{NIP: "12345", Name: "Test User"}) - } - }, - expectError: false, - }, - { - name: "user not found", - nip: "99999", - setupMock: func(ur *mockUserRepository, cr *mockCacheRepository) { - cr.getFunc = func(ctx context.Context, key string) ([]byte, error) { - return nil, nil - } - ur.getByNIPFunc = func(ctx context.Context, nip string) (*domain.User, error) { - return nil, nil - } - }, - expectError: false, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - userRepo := &mockUserRepository{} - cacheRepo := &mockCacheRepository{} - tt.setupMock(userRepo, cacheRepo) - - svc := NewUserService(userRepo, nil, cacheRepo) - - user, err := svc.GetByNIP(context.Background(), tt.nip) - - if tt.expectError && err == nil { - t.Error("expected error, got nil") - } - if !tt.expectError && err != nil { - t.Errorf("unexpected error: %v", err) - } - }) - } -} +. +├── cmd/ +│ ├── server/ +│ │ └── run.go # Server entry point +│ └── admin/ +│ └── run.go # Admin CLI entry point +├── config/ +│ ├── config.go +│ ├── model.go +│ └── loader.go # OpenBao/env loader +├── internal/ +│ ├── core/ +│ │ ├── domain/ # Entities, value objects, events +│ │ ├── port/ # Inbound/outbound interfaces +│ │ └── usecase/ # Business logic +│ ├── adapter/ +│ │ ├── inbound/ # HTTP handlers +│ │ └── outbound/ # PostgreSQL, cache, keycloak +│ └── provider/ # Dependency provider (Option B) +├── middlewares/ # Auth, logging, metrics, tracing +├── pkg/ # Logger, metrics, response, grace +├── db/migrations/ # SQL migrations +├── deployments/ # Dockerfile, docker-compose, .gitlab-ci.yml +└── main.go # Bootstrapper ``` --- -## Build Commands - -### Makefile - -```makefile -include .env - -VARS:=$(shell sed -ne 's/ *\#.*$$//; /./ s/=.*$$// p' .env ) -$(foreach v,$(VARS),$(eval $(shell echo export $(v)="$($(v))"))) +## Core Principles -BINARY_NAME=messaging-be -MAIN_FILE=cmd/server/main.go -BUILD_DIR=build +- **SOLID Design** + - SRP: Single Responsibility - One usecase per operation + - DIP: Dependency Inversion - Depend on ports, not implementations + - See [ARCHITECTURE.md](ARCHITECTURE.md#solid-principles) for detailed examples -LDFLAGS=-ldflags "-s -w" -GCFLAGS=-gcflags="all=-trimpath=$(PWD)" -ASMFLAGS=-asmflags="all=-trimpath=$(PWD)" +- **Hexagonal Architecture** + - Core domain has no external dependencies + - Ports define boundaries, adapters implement them + - See [ARCHITECTURE.md](ARCHITECTURE.md#hexagonal-architecture) -VERSION:=$(shell git describe --tags --always --dirty 2>/dev/null || echo "dev") -COMMIT:=$(shell git rev-parse --short HEAD 2>/dev/null || echo "local") +- **12-Factor Compliance** + - Environment-based config, no hardcoded values + - Stateless processes, graceful shutdown + - See [12-FACTOR.md](12-FACTOR.md) -.PHONY: all clean build build-darwin build-linux build-windows run test tidy migrate migrate-down - -all: clean build - -clean: - rm -rf $(BUILD_DIR) - mkdir -p $(BUILD_DIR) - -build: build-darwin build-linux build-windows - -build-darwin: - GOOS=darwin GOARCH=amd64 go build $(LDFLAGS) $(GCFLAGS) $(ASMFLAGS) \ - -ldflags "-X main.version=$(VERSION) -X main.commit=$(COMMIT)" \ - -o $(BUILD_DIR)/$(BINARY_NAME)-darwin-amd64 $(MAIN_FILE) - -build-linux: - GOOS=linux GOARCH=amd64 go build $(LDFLAGS) $(GCFLAGS) $(ASMFLAGS) \ - -ldflags "-X main.version=$(VERSION) -X main.commit=$(COMMIT)" \ - -o $(BUILD_DIR)/$(BINARY_NAME)-linux-amd64 $(MAIN_FILE) - -build-windows: - GOOS=windows GOARCH=amd64 go build $(LDFLAGS) $(GCFLAGS) $(ASMFLAGS) \ - -ldflags "-X main.version=$(VERSION) -X main.commit=$(COMMIT)" \ - -o $(BUILD_DIR)/$(BINARY_NAME)-windows-amd64.exe $(MAIN_FILE) - -run: - go run $(MAIN_FILE) - -test: - go test -v -race ./... - -tidy: - go mod tidy - go mod verify - -migrate: - migrate -database "$(PERSISTENCE_DSN)" -path db/migrations/persistence up $(step) - -migrate-down: - migrate -database "$(PERSISTENCE_DSN)" -path db/migrations/persistence down $(step) - -migrate-cache: - migrate -database "$(CACHE_DSN)" -path db/migrations/cache up $(step) - -migrate-cache-down: - migrate -database "$(CACHE_DSN)" -path db/migrations/cache down $(step) -``` +- **Error Handling** + - NEVER skip error checking + - Always handle or return errors + - See [CODING-STANDARDS.md](CODING-STANDARDS.md#critical-rules) --- -## Quick Start - -### 1. Create domain entity - -```go -// internal/core/domain/user.go -type User struct { - NIP string `db:"nip" json:"nip"` - Name string `db:"name" json:"name"` - Email string `db:"email" json:"email"` - IsActive bool `db:"is_active" json:"is_active"` - LastLoginAt time.Time `db:"last_login_at" json:"last_login_at"` - CreatedAt time.Time `db:"created_at" json:"created_at"` - UpdatedAt time.Time `db:"updated_at" json:"updated_at"` - Roles []string `db:"-" json:"roles,omitempty"` -} -``` - -### 2. Define ports - -```go -// internal/core/port/outbound/user.go -type UserRepository interface { - GetByNIP(ctx context.Context, nip string) (*domain.User, error) - Create(ctx context.Context, user *domain.User) error - Update(ctx context.Context, user *domain.User) error - UpdateLastLogin(ctx context.Context, nip string) error - List(ctx context.Context) ([]domain.User, error) -} - -// internal/core/port/inbound/user.go -type UserService interface { - GetByNIP(ctx context.Context, nip string) (*domain.User, error) - Create(ctx context.Context, user *domain.User) error - List(ctx context.Context) ([]domain.User, error) -} - -// internal/core/port/inbound/auth.go -type AuthService interface { - Authenticate(ctx context.Context, token string) (*domain.User, error) - HasRole(user *domain.User, role string) bool - HasAnyRole(user *domain.User, roles ...string) bool -} -``` - -### 3. Implement repository (pgx) - -```go -// internal/adapter/outbound/persistence/user_repository.go -type userRepository struct { - pool *persistence.Pool -} - -func NewUserRepository(pool *persistence.Pool) outbound.UserRepository { - return &userRepository{pool: pool} -} - -func (r *userRepository) GetByNIP(ctx context.Context, nip string) (*domain.User, error) { - // Implementation with pgx -} - -func (r *userRepository) Create(ctx context.Context, user *domain.User) error { - // Implementation with pgx -} - -func (r *userRepository) Update(ctx context.Context, user *domain.User) error { - // Implementation with pgx -} -``` - -### 4. Implement service with caching and tracing - -```go -// internal/core/usecase/user.go -func NewUserService( - userRepo outbound.UserRepository, - auditRepo outbound.AuditLogRepository, - cache outbound.CacheRepository, -) inbound.UserService { - // Implementation with caching and tracing -} - -// internal/core/usecase/auth.go -func NewAuthService( - keycloakProvider *keycloak.Provider, - userRepo outbound.UserRepository, -) inbound.AuthService { - return &authService{ - keycloakProvider: keycloakProvider, - userRepo: userRepo, - } -} -``` - -### 5. Create handler - -```go -// internal/adapter/inbound/rest/v1/user_handler.go -type UserHandler struct { - svc inbound.UserService -} - -func NewUserHandler(svc inbound.UserService) *UserHandler -func (h *UserHandler) RegisterRoutes(r *gin.RouterGroup) -``` - -### 6. Wire in server - -```go -// cmd/server/main.go -persistencePool, _ := persistence.NewPool(ctx, cfg.Persistence.DSN, ...) -cachePool, _ := persistence.NewPool(ctx, cfg.Cache.DSN, ...) - -// Keycloak/JWT provider -keycloakProvider, err := keycloak.NewProvider(ctx, cfg.Keycloak.JWKSURL, cfg.Keycloak.ClientID) -if err != nil { - return fmt.Errorf("failed to initialize Keycloak: %w", err) -} - -userRepo := persistence.NewUserRepository(persistencePool) -cacheRepo := cache.NewPostgresCacheRepository(cachePool) - -authSvc := usecase.NewAuthService(keycloakProvider, userRepo) -userSvc := usecase.NewUserService(userRepo, auditRepo, cacheRepo) -userHandler := v1.NewUserHandler(userSvc) +## Key Environment Variables -// Protected routes -v1 := r.Group("/v1") -v1.Use(middlewares.AuthMiddleware(authSvc)) -{ - v1.GET("/profile", userHandler.GetProfile) - v1.GET("/users", userHandler.ListUsers) -} +| Variable | Description | Required | +|----------|-------------|----------| +| `APP_ENV` | Environment (`local`, `staging`, `production`) | Yes | +| `PERSISTENCE_DSN` | PostgreSQL connection string | Yes | +| `KEYCLOAK_JWKS_URL` | Keycloak JWKS endpoint | Yes | +| `KEYCLOAK_CLIENT_ID` | Keycloak client ID for role extraction | Yes | +| `OPENBAO_ADDR` | OpenBao server (non-local only) | Staging/Prod | +| `OPENBAO_TOKEN` | OpenBao token (non-local only) | Staging/Prod | -// Admin-only routes -admin := v1.Group("/admin") -admin.Use(middlewares.RequireRole(authSvc, "team-trade.admin")) -{ - admin.GET("/dashboard", adminHandler.GetDashboard) -} -``` +See [CONFIGURATION.md](CONFIGURATION.md#environment-variables) for full list. --- -## Admin CLI (Factor 12) - -One-off administrative processes as CLI commands. - -**cmd/admin/main.go:** - -```go -package main - -import ( - "context" - "fmt" - "os" - - "myapp/config" - "myapp/internal/provider" - - "github.com/urfave/cli/v2" -) - -func main() { - ctx := context.Background() - - cfg, err := config.Load(ctx) - if err != nil { - fmt.Fprintf(os.Stderr, "Failed to load config: %v\n", err) - os.Exit(1) - } - - pvd, err := provider.NewProvider(ctx, cfg) - if err != nil { - fmt.Fprintf(os.Stderr, "Failed to initialize provider: %v\n", err) - os.Exit(1) - } - defer pvd.Close(ctx) - - app := &cli.App{ - Name: "admin", - Usage: "Administrative commands", - Commands: []*cli.Command{ - migrateCommand(pvd), - seedCommand(pvd), - userCommand(pvd), - cacheCommand(pvd), - healthCommand(pvd), - }, - } - - if err := app.Run(os.Args); err != nil { - fmt.Fprintf(os.Stderr, "Error: %v\n", err) - os.Exit(1) - } -} -``` - -**Available Commands:** +## Commands ```bash -# Database migrations -admin migrate up # Run all pending migrations -admin migrate down 1 # Rollback 1 migration -admin migrate status # Check migration status - -# Data seeding -admin seed # Seed development data -admin seed --env=staging # Seed staging (with confirmation) +# Server (default) +./bin/messaging-be # Run server on :8080 +./bin/messaging-be server # Explicit server +./bin/messaging-be server -port=8081 -# User management -admin user list # List all users -admin user deactivate # Deactivate user -admin user activate # Reactivate user -admin user info # Show user details +# Admin commands +./bin/messaging-be admin migrate up # Run migrations +./bin/messaging-be admin migrate down 1 # Rollback (disabled in prod) +./bin/messaging-be admin user list # List users +./bin/messaging-be admin user deactivate # Deactivate user +./bin/messaging-be admin cache clear # Clear cache +./bin/messaging-be admin health # Health check -# Cache management -admin cache clear # Clear all cache entries -admin cache stats # Show cache statistics - -# Health checks -admin health # Check all services -admin health --json # JSON output for monitoring +# Other +./bin/messaging-be version # Show version ``` -**CLI Command Implementations:** - -```go -func migrateCommand(pvd *provider.Provider) *cli.Command { - return &cli.Command{ - Name: "migrate", - Usage: "Database migration commands", - Subcommands: []*cli.Command{ - { - Name: "up", - Usage: "Run all pending migrations", - Action: func(c *cli.Context) error { - return runMigrationsUp(c.Context, pvd) - }, - }, - { - Name: "down", - Usage: "Rollback migrations", - Args: true, - Action: func(c *cli.Context) error { - steps := 1 - if args := c.Args(); args.Len() > 0 { - steps, _ = strconv.Atoi(args.First()) - } - return runMigrationsDown(c.Context, pvd, steps) - }, - }, - { - Name: "status", - Usage: "Check migration status", - Action: func(c *cli.Context) error { - return printMigrationStatus(c.Context, pvd) - }, - }, - }, - } -} - -func userCommand(pvd *provider.Provider) *cli.Command { - return &cli.Command{ - Name: "user", - Usage: "User management commands", - Subcommands: []*cli.Command{ - { - Name: "list", - Usage: "List all users", - Action: func(c *cli.Context) error { - return listUsers(c.Context, pvd) - }, - }, - { - Name: "deactivate", - Usage: "Deactivate a user", - Args: true, - Action: func(c *cli.Context) error { - return deactivateUser(c.Context, pvd, c.Args().First()) - }, - }, - { - Name: "activate", - Usage: "Activate a user", - Args: true, - Action: func(c *cli.Context) error { - return activateUser(c.Context, pvd, c.Args().First()) - }, - }, - }, - } -} -``` +See [DEPLOYMENT.md](DEPLOYMENT.md#operations) for all commands. --- -## REST Standards - -### URL Naming Conventions - -| Pattern | Correct | Incorrect | -|---------|---------|-----------| -| Resources | `/users` | `/getUsers` | -| Nested | `/users/123/orders` | `/getUserOrders` | -| Actions | `/users/123/activate` | `/activateUser` | -| Plural | Always plural | Singular | - -### HTTP Methods - -| Method | Usage | Response | -|--------|-------|----------| -| GET | Retrieve resource(s) | 200 + body | -| POST | Create new resource | 201 + body | -| PUT | Full replace | 200/204 + body | -| PATCH | Partial update | 200/204 + body | -| DELETE | Remove resource | 204 No Content | - -### Status Codes - -| Code | Usage | -|------|-------| -| 200 | GET success, PUT/PATCH success | -| 201 | POST create success | -| 204 | DELETE success, no body | -| 400 | Malformed request, validation error | -| 401 | Authentication required | -| 403 | Authenticated but forbidden | -| 404 | Resource not found | -| 409 | Conflict (duplicate, state violation) | -| 422 | Semantic validation error | -| 500 | Internal server error | - -### Response Envelope - -```go -type Response struct { - Data interface{} `json:"data,omitempty"` - Error string `json:"error,omitempty"` - Meta *Meta `json:"meta,omitempty"` -} - -type Meta struct { - Page int `json:"page,omitempty"` - PerPage int `json:"per_page,omitempty"` - Total int `json:"total,omitempty"` - NextPage int `json:"next_page,omitempty"` -} -``` - -### Request/Response Examples - -**Create User (POST /users):** +## Key Dependencies -Request: -```json -{ - "name": "John Doe", - "email": "john@example.com" -} -``` - -Response (201): -```json -{ - "data": { - "nip": "p021050", - "name": "John Doe", - "email": "john@example.com", - "is_active": true, - "created_at": "2024-01-15T10:00:00Z" - } -} -``` +| Package | Purpose | +|---------|---------| +| gin-gonic/gin | HTTP framework | +| jackc/pgx/v5 | PostgreSQL driver | +| golang-migrate/migrate | Database migrations | +| MicahParks/keyfunc/v3 | JWT validation | +| hashicorp/vault/v2 | OpenBao secrets | +| urfave/cli/v2 | Admin CLI | +| prometheus/client_golang | Metrics | +| go.opentelemetry.io/otel | Distributed tracing | -**List Users (GET /users):** +--- -Response (200): -```json -{ - "data": [ - {"nip": "p021050", "name": "John Doe", ...}, - {"nip": "p021051", "name": "Jane Doe", ...} - ], - "meta": { - "page": 1, - "per_page": 20, - "total": 2 - } -} -``` +## Health Endpoint -**Validation Error (400):** +`GET /healthz` returns 200 if healthy, 503 if degraded: ```json { - "error": "validation failed", - "meta": { - "fields": { - "email": "invalid email format" - } + "status": "ok", + "version": "1.0.0", + "commit": "abc123", + "checks": { + "database": true, + "cache": true } } ``` --- -## DDD Value Objects and Domain Events - -### Value Objects - -```go -// internal/core/domain/valueobject/nip.go -package valueobject - -type NIP string - -func (n NIP) String() string { - return string(n) -} - -func (n NIP) Validate() error { - if len(n) == 0 { - return errors.New("nip is required") - } - if len(n) > 20 { - return errors.New("nip must not exceed 20 characters") - } - return nil -} -``` - -```go -// internal/core/domain/valueobject/email.go -package valueobject - -type Email string - -func (e Email) String() string { - return string(e) -} - -func (e Email) Validate() error { - if len(e) == 0 { - return nil // Email can be empty - } - if !strings.Contains(string(e), "@") { - return errors.New("invalid email format") - } - return nil -} -``` - -### Domain Events - -```go -// internal/core/domain/event/event.go -package event - -type EventType string - -const ( - UserCreated EventType = "user.created" - UserUpdated EventType = "user.updated" - UserDeactivated EventType = "user.deactivated" -) - -type DomainEvent interface { - EventType() EventType - Timestamp() time.Time -} - -type UserCreatedEvent struct { - NIP string - Name string - Email string - timestamp time.Time -} - -func (e *UserCreatedEvent) EventType() EventType { - return UserCreated -} - -func (e *UserCreatedEvent) Timestamp() time.Time { - return e.timestamp -} -``` - -**Event Publisher Interface:** - -```go -// internal/core/port/outbound/event.go -package outbound - -import ( - "context" - "myapp/internal/core/domain/event" -) - -type EventPublisher interface { - Publish(ctx context.Context, evt event.DomainEvent) error -} -``` - ---- - -## Docker and Docker Compose (Factor 10) - -### Dockerfile - -```dockerfile -# deployments/Dockerfile -FROM golang:1.21-alpine AS builder - -WORKDIR /app -COPY go.mod go.sum ./ -RUN go mod download - -COPY . . -RUN CGO_ENABLED=0 GOOS=linux go build -ldflags "-s -w" -o server ./cmd/server - -FROM alpine:3.19 - -RUN apk --no-cache add ca-certificates tzdata - -WORKDIR /app -COPY --from=builder /app/server . -COPY --from=builder /app/db/migrations ./db/migrations - -EXPOSE 8080 - -ENV APP_ENV=production - -CMD ["./server"] -``` - -### Docker Compose - -```yaml -# deployments/docker-compose.yml -version: '3.8' - -services: - app: - build: .. - env_file: ../.env.local - depends_on: - postgres: - condition: service_healthy - keycloak: - condition: service_started - ports: - - "8080:8080" - healthcheck: - test: ["CMD", "wget", "-qO-", "http://localhost:8080/healthz"] - interval: 30s - timeout: 10s - retries: 3 - - postgres: - image: postgres:15-alpine - environment: - POSTGRES_DB: messaging - POSTGRES_USER: user - POSTGRES_PASSWORD: pass - volumes: - - postgres_data:/var/lib/postgresql/data - healthcheck: - test: ["CMD-SHELL", "pg_isready -U user -d messaging"] - interval: 10s - timeout: 5s - retries: 5 - - keycloak: - image: quay.io/keycloak/keycloak:24.0 - command: start-dev - environment: - KEYCLOAK_ADMIN: admin - KEYCLOAK_ADMIN_PASSWORD: admin - ports: - - "8081:8080" - -volumes: - postgres_data: -``` - ---- - -## GitLab CI/CD (Factor 5) - -```yaml -# deployments/.gitlab-ci.yml -stages: - - build - - test - - release - - deploy - -variables: - IMAGE_NAME: $CI_REGISTRY_IMAGE/backend +## Authentication -build: - stage: build - image: golang:1.21-alpine - before_script: - - apk add git make - script: - - make tidy - - make build - artifacts: - paths: - - bin/ - expire_in: 1 day +Backend acts as OAuth2 Resource Server: +- Validates Bearer tokens from UI +- No client secret needed (uses JWKS) +- User auto-created/updated on login +- Roles extracted from `resource_access.{client_id}` -test: - stage: test - image: golang:1.21-alpine - services: - - postgres:15-alpine - variables: - POSTGRES_DB: test - POSTGRES_USER: test - POSTGRES_PASSWORD: test - before_script: - - apk add git make - - go install github.com/俚语/migrate/v4/cmd/migrate@latest - script: - - migrate -database "$TEST_DATABASE_URL" -path db/migrations/persistence up - - go test -race -coverprofile=coverage.out ./... - - go tool cover -func=coverage.out - coverage: '/total:\s+\(statements\)\s+(\d+\.\d+)%/' - -release: - stage: release - image: docker:24.0-cli - services: - - docker:24.0-dind - script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - - docker build -t $IMAGE_NAME:$CI_COMMIT_SHA -t $IMAGE_NAME:latest . - - docker push $IMAGE_NAME:$CI_COMMIT_SHA - - docker push $IMAGE_NAME:latest - rules: - - main - -deploy-production: - stage: deploy - image: bitnami/kubectl:latest - environment: - name: production - url: https://api.example.com - script: - - kubectl set image deployment/backend server=$IMAGE_NAME:$CI_COMMIT_SHA - - kubectl rollout status deployment/backend - rules: - - main - when: manual -``` +See [AUTHENTICATION.md](AUTHENTICATION.md) for details. --- -## go.mod Dependencies - -```go -module messaging-be +## Build & Run -go 1.21 - -require ( - // Web Framework - github.com/gin-gonic/gin v1.12.0 - - // PostgreSQL (pgx) - github.com/jackc/pgx/v5 v5.5.0 - - // Migrations - github.com/golang-migrate/migrate/v4 v4.19.0 - - // Configuration - github.com/golobby/env/v2 v2.2.4 - github.com/golobby/dotenv v1.3.2 - github.com/hashicorp/vault/v2 v2.0.0 - - // Observability - github.com/prometheus/client_golang v1.18.0 - go.opentelemetry.io/otel v1.21.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.21.0 - go.opentelemetry.io/otel/sdk v1.21.0 +```bash +# Build +make build - // Authentication - github.com/MicahParks/keyfunc/v3 v3.0.0 - github.com/golang-jwt/jwt/v5 v5.2.0 +# Development +make run - // CLI - github.com/urfave/cli/v2 v2.27.0 +# Docker +docker build -t messaging-be . +docker run -p 8080:8080 --env-file .env messaging-be - // Utilities - github.com/pkg/errors v0.9.1 - github.com/google/uuid v1.6.0 -) +# Docker Compose (just app, devs provide their own postgres/keycloak) +docker-compose up -d ``` ---- - -## Best Practices - -### Do - -- Pass context.Context as first parameter in all service/repository methods -- Use dependency injection via constructors -- Return interfaces from constructors, accept interfaces as parameters -- Wrap errors with context using errors.Wrap() -- Use table-driven tests for multiple test cases -- Add tracing spans for observability in all layers -- Implement cache-aside pattern for read-heavy operations -- Use advisory locks for HA-safe background jobs - -### Don't - -- Use global variables -- Put business logic in HTTP handlers -- Import database drivers in domain or service packages -- Return concrete types from public constructors -- Leave TODO comments - fix now or create issues -- Mix HTTP concerns with business logic -- Log sensitive data without masking - ---- - -## References - -- [Hexagonal Architecture by Alistair Cockburn](https://alistair.cockburn.us/hexagonal-architecture/) -- [Go Project Layout](https://github.com/golang-standards/project-layout) -- [pgx Documentation](https://pkg.go.dev/github.com/jackc/pgx/v5) -- [OpenTelemetry Go SDK](https://opentelemetry.io/docs/instrumentation/go/) -- [Prometheus Go Client](https://prometheus.io/docs/instrumenting/go/) -- [Go slog Package](https://pkg.go.dev/log/slog) -- [Grafana Tempo](https://grafana.com/docs/tempo/latest/) +See [DEPLOYMENT.md](DEPLOYMENT.md) for CI/CD and operations.