# Kedebah Commerce — Docker Deployment

Containerised, production-oriented stack for running **Kedebah Commerce** on a
single Ubuntu host, served at **https://topup.kedebah.com**.

It bundles every module Commerce needs — the Finance API, the Commerce SPA, and
the supporting Auth / Onboarding / Tenant-Processor / Email services — plus all
the background processes (queue workers, schedulers, and the Reverb websocket
server), a bundled Redis, and a TLS-terminating edge proxy.

The host's **existing PostgreSQL + PgBouncer** is used as-is; no database
container is created.

---

## 1. Architecture

```
                          Internet (topup.kedebah.com)
                                     │  :80 / :443
                          ┌──────────▼───────────┐
                          │   edge (nginx + TLS)  │
                          └───┬───────┬────────┬──┘
             /api/v1/financeApi│  /app/, /apps/ │  /  (default)
             /finance/ , /storage/, …           │
                              ▼        ▼        ▼
                      ┌──────────┐ ┌────────┐ ┌──────────┐
                      │ finance  │ │ reverb │ │ commerce │
                      │ (API+UI) │ │  (ws)  │ │  (SPA)   │
                      └────┬─────┘ └────────┘ └──────────┘
```

- **`/`** → Commerce SPA (POS / inventory)
- **`/finance/`** → Finance SPA portal (same host; no subdomain required)
- **`/api/v1/financeApi/`** → Finance API

Only **edge** publishes ports (80/443). Everything else talks over the internal
`kedebah` docker network.

### Services

| Service                      | Role                                   | Public? |
|------------------------------|----------------------------------------|:-------:|
| `edge`                       | nginx reverse proxy + TLS              |  ✅ 80/443 |
| `commerce`                   | Vue SPA served by its Laravel app      |  via edge |
| `finance`                    | Finance/Commerce API (+ its own UI)    |  via edge |
| `reverb`                     | Laravel Reverb websocket server        |  via edge |
| `finance-worker`             | `queue:work` (commerce backfill jobs)  |  internal |
| `finance-scheduler`          | `schedule:work` (alerts, bulk actions) |  internal |
| `auth`                       | Authentication service                 |  internal |
| `onboarding` / `-worker`     | Onboarding + media conversions         |  internal |
| `tenant-processor` / workers | Tenant DB provisioning + maintenance   |  internal |
| `email` / `email-worker`     | Transactional email + queued mail      |  internal |
| `sms`                        | SMS gateway proxy (Deywuro)            |  internal |
| `redis`                      | Cache + sessions                       |  internal |
| `certbot`                    | Let's Encrypt issuance + renewal       |  internal |
| `grafana` (+ loki stack)     | Logs / metrics / Telegram alerts       |  via edge `/monitoring/` *(profile)* |

Observability services start only with `--profile observability` (see
`docs/OBSERVABILITY.md`).

---

## 2. Prerequisites

- Ubuntu host with **Docker Engine 24+** and the **compose plugin**
  (`docker compose version`).
- **PostgreSQL + PgBouncer** already running on the host, with the Kedebah
  database and the required tables/data. Note the PgBouncer port (commonly
  `6432`).
- DNS: an **A/AAAA record for `topup.kedebah.com` → this server's public IP**.
- Ports **80** and **443** open to the internet.
- All seven application repositories available on the server as build sources:
  `kedebah_v2_finance`, `kedebah_v2_commerce_and_distribution`,
  `kedebah_v2_auth_api`, `kedebah_v2_onboarding_api`,
  `kedebah_v2_tenant_processor`, `kedebah_v2_email_api`, `kedebah_v2_sms_api`.

This deployment is its own repository. Docker uses it as the primary build
context and supplies each application repository as a named BuildKit context.
The application paths are configured through `*_SOURCE` values in `.env`; they
can be relative or absolute and do not need to be committed here.

### Let PgBouncer accept container connections

Containers reach the host via `host.docker.internal` (mapped to the docker
bridge gateway, typically `172.17.0.1`). Make sure PgBouncer listens on that
interface and allows it:

- `pgbouncer.ini`: `listen_addr = 0.0.0.0` (or add the docker bridge IP)
- host firewall: allow the docker bridge subnet to reach the PgBouncer port

---

## 3. Configure

```bash
cd kedebah-commerce-deploy
cp .env.example .env

# Generate six APP_KEYs and paste them into .env
bin/generate-keys.sh
```

Edit `.env` and set at minimum:

- `DOMAIN` (already `topup.kedebah.com`)
- Finance portal URL is **`https://${DOMAIN}/finance/`** (path proxy; leave `FINANCE_DOMAIN` empty)
- `FINANCE_SOURCE`, `COMMERCE_SOURCE`, `AUTH_SOURCE`, `ONBOARDING_SOURCE`,
  `TENANT_PROCESSOR_SOURCE`, `EMAIL_SOURCE`, and `SMS_SOURCE` → paths to the existing module
  checkouts. The defaults work when all repositories are sibling directories.
- `DB_HOST` / `DB_PORT` / `DB_DATABASE` / `DB_USERNAME` / `DB_PASSWORD`
  → point `DB_PORT` at **PgBouncer**
- `REDIS_PASSWORD`
- `FINANCE_SERVICE_KEY`, `AUTH_SERVICE_KEY`, `ONBOARDING_SERVICE_KEY`, and
  `TENANT_PROCESSOR_SERVICE_KEY` → each must match the `key` for its named
  caller in the central `kedebah_services` table. Receiving APIs validate the
  complete `(service_name, service_key)` pair.
- `AUTH_INTERNAL_CALLBACK_SERVICE_KEY` → callback key expected by the
  Administrative API; it may equal `AUTH_SERVICE_KEY` in your environment.
- `APP_KEY_*` (from `generate-keys.sh`)
- `REVERB_APP_ID` / `REVERB_APP_KEY` / `REVERB_APP_SECRET`
- `MAIL_*` (set `MAIL_MAILER=smtp` for real email)
- `LETSENCRYPT_EMAIL`

> **Important:** `REVERB_APP_KEY` is baked into the Commerce SPA at **build
> time** (as `VITE_REVERB_APP_KEY`) and also used by the server. If you change
> it later, rebuild the `commerce` image.

---

## 4. Build & start

```bash
# Build all images (first build downloads PHP/Node layers — a few minutes)
docker compose build

# Start the whole stack
docker compose up -d

# Watch it come up
docker compose ps
docker compose logs -f edge finance commerce reverb
```

At this point the site is reachable over **HTTP** at `http://topup.kedebah.com`.

### Enable HTTPS (Let's Encrypt)

Once DNS resolves to this server and port 80 is reachable:

```bash
bin/init-letsencrypt.sh
```

This obtains the certificate and recreates `edge`, which then auto-switches to
HTTPS (HTTP redirects to HTTPS). The `certbot` service renews automatically.

> Because the SPA is built with `VITE_REVERB_SCHEME=https` / port `443`, the
> production site is intended to run over HTTPS. Run the step above before
> going live so websockets (`wss://`) work.

---

## 5. Database migrations

Migrations are **OFF by default** (the entrypoint does not migrate) because this
stack points at a pre-existing database. When you *do* want to migrate:

```bash
# Central migrations for every service
bin/migrate.sh

# Central + tenant migrations
bin/migrate.sh tenants
```

You can also enable automatic migration on boot per service by setting
`RUN_MIGRATIONS=true` (and/or `RUN_TENANT_MIGRATIONS=true`) in that service's
environment — not recommended for a shared/production DB.

---

## 6. Day-2 operations

```bash
# Tail logs for a service
docker compose logs -f finance-worker

# Run artisan in any service
docker compose exec finance php artisan about
docker compose exec finance php artisan queue:failed

# Clear/rebuild caches after an env change
docker compose exec finance php artisan optimize:clear

# Restart just the workers
docker compose restart finance-worker email-worker tenant-processor-worker

# Rebuild after pulling new module code
docker compose build && docker compose up -d
```

### Queues, schedules & websockets (what runs where)

- **Finance** — `finance-worker` (`queue:work`) handles commerce inventory
  backfill jobs; `finance-scheduler` (`schedule:work`) runs
  `commerce:operational-alerts` (daily) and `bulk-actions:process` (every
  minute). `reverb` serves `ShouldBroadcastNow` inventory/commerce events.
- **Tenant Processor** — `tenant-processor-worker` consumes the `tenants` queue
  (tenant provisioning) plus default replica jobs; `tenant-processor-scheduler`
  runs `replicas:maintain --cleanup` every 5 minutes.
- **Email** — `email-worker` consumes the `emails` queue and media conversions.
- **Onboarding** — `onboarding-worker` handles queued avatar media conversions.

---

## 6b. High-traffic tuning & scaling

Defaults below are sized for **Top Up go-live** (Top Up + Lena + Express Care
outlets) on a **~64 GB host with ~28 GB free** (shared with Postgres). Peak
budget for this stack ≈ **14–18 GB**.

| Service | FPM workers | Approx RAM |
|---------|-------------|------------|
| finance | 96 | ~6.0 GB |
| commerce | 32 | ~2.0 GB |
| auth | 32 | ~2.0 GB |
| onboarding | 24 | ~1.5 GB |
| email | 16 | ~1.0 GB |
| tenant-processor | 16 | ~1.0 GB |
| redis | — | capped at 2 GB (`REDIS_MAXMEMORY`) |

Total concurrent PHP capacity ≈ **216** requests. Adjust via `*_FPM_MAX_CHILDREN`
in `.env`. If this host becomes dedicated to Commerce only, finance can go to
112–128 (watch RAM + PgBouncer).

The stack ships tuned for concurrency out of the box:

- **Edge proxy** — `worker_connections 16384`, edge gzip, proxy buffering,
  `open_file_cache`, and `nofile=65535`. Backend names are resolved through
  Docker DNS every ~10s (`resolver` + variable `proxy_pass`), so recreated
  containers are picked up automatically without restarting the edge. See
  `docker/edge/nginx.conf` and `docker/edge/snippets/app-locations.conf`.
- **App containers** — nginx keeps a fastcgi keep-alive pool to php-fpm; the
  php-fpm pool size is set at startup from `PHP_FPM_MAX_CHILDREN` (see table
  above). `max_children` is the max concurrent PHP requests **per container**.
- **OPcache** — production mode (`validate_timestamps=0`); code changes require
  container recreate/restart.

**Sizing php-fpm:** budget ~50–70 MB per worker. Change `*_FPM_MAX_CHILDREN` in
`.env` and recreate the affected web containers (no image rebuild needed for
FPM size alone; rebuild if you changed `docker/common/*` nginx/php/opcache).

**Database is the real bottleneck under load.** Every busy php-fpm worker + every
queue worker can hold a DB connection. With these defaults expect **~250–320**
potential clients. Make sure **PgBouncer** absorbs this:

- Run PgBouncer in **transaction** pooling mode (`pool_mode = transaction`).
- Suggested: `max_client_conn >= 600`, `default_pool_size` 50–80 per DB user.
- Point `DB_PORT` at PgBouncer, **not** Postgres directly.

**Horizontal scaling (more replicas of a backend):** the edge already resolves
backend names through Docker DNS with variable `proxy_pass`, so extra replicas
are load-balanced automatically (DNS round-robin):

```bash
docker compose up -d --scale finance=2
docker compose up -d --scale finance-worker=2
```

For most single-server deployments, raising `*_FPM_MAX_CHILDREN` and giving the
box more RAM/CPU is simpler and sufficient before going multi-replica.

**Apply go-live tuning on the server** (after pulling these config changes):

```bash
cd /var/www/html/KEDEBAH/kedebah-commerce-deploy

# 1) Bump live .env FPM values (match .env.example)
# FINANCE_FPM_MAX_CHILDREN=96
# COMMERCE_FPM_MAX_CHILDREN=32
# AUTH_FPM_MAX_CHILDREN=32
# ONBOARDING_FPM_MAX_CHILDREN=24

# 2) Rebuild images that bake nginx/php/opcache/fpm configs
docker compose build finance commerce auth onboarding edge
docker compose up -d

# 3) Confirm FPM pool sizes
docker compose exec finance sh -lc 'grep -E "^pm\.(max_children|start_servers)" /usr/local/etc/php-fpm.d/zz-www.conf'
docker compose exec finance sh -lc 'curl -s http://127.0.0.1/fpm-status'
```

**Monitoring:** php-fpm exposes `/fpm-status` and `/fpm-ping` (localhost-only)
inside each app container, e.g.:
```bash
docker compose exec finance sh -lc 'curl -s http://127.0.0.1/fpm-status'
```

See also `docs` in finance: `TOPUP_OFFICIAL_MIGRATION_PLAN.md` § go-live load.
---

## 7. Notes, assumptions & known caveats

- **Cache/session use Redis; queues use the database.** Queues stay on the
  `database` connection to match how the apps define their named queues
  (`tenants`, `emails`) and their worker instructions. Redis is used for cache
  and sessions to avoid extra DB load. Adjust in `docker-compose.yml`
  (`x-laravel-env`) if you prefer Redis queues.
- **Internal service URLs** are wired to the API prefixes discovered in each
  app's `bootstrap/app.php`:
  - `AUTH_SERVICE_URL=http://auth/api/v1/authService`
  - `ONBOARDING_SERVICE_URL=http://onboarding/api/v1/onboardingService`
  - `EMAIL_SERVICE_URL=http://email/api/v1/emailService`
  - `SMS_SERVICE_URL=http://sms/api/v1/smsService`
  - `TENANT_PROCESSOR_URL=http://tenant-processor/api/v1/tenantProcessorService`
- **Finance PDF export (Browsershot).** The finance image ships Chromium + Node
  and sets `CHROME_PATH` / `PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium`. Spatie
  Browsershot also expects the `puppeteer` Node module. If PDF/print export is
  used, install it once inside the finance image/container:
  ```bash
  docker compose exec finance sh -lc \
    'cd /var/www/html && PUPPETEER_SKIP_DOWNLOAD=1 npm i puppeteer --no-save'
  ```
  (Core Commerce/POS features do not depend on this.)
- **Uploaded files** (product images, attachments) persist in the
  `finance-storage` volume and are served via the edge `/storage/` route.
- **Trusted proxies:** the edge sets `X-Forwarded-Proto`. `APP_URL` is set to
  `https://topup.kedebah.com` so generated asset/storage URLs use HTTPS. If you
  hit redirect-scheme issues, configure the apps to trust proxies.
- **TLS alternative:** if you already terminate TLS upstream (e.g. Cloudflare or
  a host load balancer), you can skip `init-letsencrypt.sh` and simply proxy to
  the edge on port 80.

---

## 7b. Observability (logs, metrics, Telegram)

Optional profile — **no new subdomain**. UI:

**https://topup.kedebah.com/monitoring/**

```bash
# Set GRAFANA_ADMIN_* and TELEGRAM_* in .env, then:
docker compose build edge && docker compose up -d edge
docker compose --profile observability up -d
```

Includes Grafana, Loki + Promtail (container logs), Prometheus + cAdvisor +
node-exporter, and sample Telegram alerts for ERROR spikes / low host memory.
Full runbook: [`docs/OBSERVABILITY.md`](docs/OBSERVABILITY.md).

---

## 8. Layout

```
kedebah-commerce-deploy/
├── docker-compose.yml          # all services, workers, schedulers, edge, redis
├── .dockerignore               # keeps deployment build context lean
├── .env.example                # single config file (copy to .env)
├── .gitignore                  # prevents production .env being committed
├── README.md
├── bin/
│   ├── generate-keys.sh        # print six APP_KEYs
│   ├── init-letsencrypt.sh     # first TLS cert issuance
│   └── migrate.sh              # central / tenant migrations
├── docs/
│   ├── GO_LIVE_LOAD_TUNING.md
│   └── OBSERVABILITY.md        # Grafana / Loki / Telegram profile
└── docker/
    ├── common/                 # shared php.ini, opcache, fpm, nginx, entrypoint, supervisord
    ├── laravel-api.Dockerfile  # auth, onboarding, tenant-processor, email
    ├── laravel-web.Dockerfile  # finance (+chromium), commerce (Vite build)
    ├── edge/                   # reverse proxy image, templates, TLS switch
    └── observability/          # loki, promtail, prometheus, grafana provisioning
```