# TipSharks — Racing ratings and predictions platform

Multi-service platform for near-realtime horse racing data ingestion, Elo-style ratings computation, and mobile racing tips delivery. Supports thoroughbred, harness, and greyhound racing across Australia and New Zealand.

---

## Architecture Overview

```mermaid
flowchart LR
    subgraph External["External APIs"]
        TAB[TAB Affiliates API]
        HRNZ[HRNZ Racing API]
    end

    subgraph Ingest["tab-api-ingest"]
        INGEST_SVC[Express API<br/>Port 9090]
        PG_INGEST[(PostgreSQL<br/>Port 5432)]
        REDIS[(Redis<br/>Port 6379)]
        BULLMQ[BullMQ Workers]
        SCHED[Cron Scheduler]
    end

    subgraph ELO["tipsharks-elo-api"]
        ELO_SVC[FastAPI + Web UI<br/>Port 8000]
        PG_ELO[(PostgreSQL<br/>Port 5433)]
        RATINGS[Rating Engine]
        WORKER[CLI Worker]
    end

    subgraph Client["tipsharks-client"]
        CLIENT_BE[FastAPI Backend<br/>Port 8001]
        MONGO[(MongoDB<br/>Port 27017)]
        MOBILE[React Native App<br/>Port 8081]
    end

    subgraph Observability["Observability"]
        JAEGER[Jaeger UI<br/>Port 16686]
        PROM[Prometheus<br/>Port 9091]
        GRAFANA[Grafana<br/>Port 3000]
    end

    TAB -->|Data fetch| INGEST_SVC
    HRNZ -->|Data fetch| INGEST_SVC
    SCHED -->|Triggers| BULLMQ
    BULLMQ -->|Read/Write| PG_INGEST
    BULLMQ -->|Queue| REDIS
    INGEST_SVC -->|Metrics & Traces| JAEGER
    INGEST_SVC -->|Metrics| PROM

    PG_INGEST -.->|Shared data| ELO_SVC
    TAB -->|Direct fetch| ELO_SVC
    HRNZ -->|Direct fetch| ELO_SVC
    ELO_SVC -->|Read/Write| PG_ELO
    WORKER -->|Batch recompute| PG_ELO
    ELO_SVC -->|Metrics & Traces| JAEGER
    ELO_SVC -->|Metrics| PROM

    PG_ELO -.->|Ratings data| CLIENT_BE
    CLIENT_BE -->|Read/Write| MONGO
    MOBILE -->|REST API| CLIENT_BE
    CLIENT_BE -->|Metrics| PROM

    PROM -->|Data source| GRAFANA
```

### Data Flow Summary

1. **tab-api-ingest** fetches raw racing data from TAB and HRNZ APIs on a cron schedule, validates it with Zod, stores it in PostgreSQL, and processes jobs via BullMQ/Redis.
2. **tipsharks-elo-api** consumes racing data (from its own PostgreSQL or via direct API fetch) and computes multi-entity Elo ratings for horses, drivers, and trainers.
3. **tipsharks-client backend** reads ratings from the elo-api's PostgreSQL and serves them through its own FastAPI endpoints backed by MongoDB.
4. **tipsharks-client frontend** (React Native / Expo) presents race data, ratings, and generated tips to end users.
5. **Observability**: OpenTelemetry traces flow to Jaeger, application metrics are scraped by Prometheus, and Grafana provides dashboards for all services.

---

## Service Overview

| Service | Directory | Port | Stack |
|---------|-----------|------|-------|
| tab-api-ingest | `tab-api-ingest/` | 9090 | TypeScript, Express 5, Prisma, BullMQ, Redis |
| tipsharks-elo-api | `tipsharks-elo-api/` | 8000 | Python 3.12+, FastAPI, SQLAlchemy, Alembic |
| tipsharks-client backend | `tipsharks-client/backend/` | 8001 | Python 3.12+, FastAPI, Motor (MongoDB) |
| tipsharks-client frontend | `tipsharks-client/frontend/` | 8081 | React Native (Expo SDK 54), TypeScript |
| health-check (unified) | `scripts/health-check-server.py` | 8082 | Python stdlib, aggregated service health |

---

## Port Map

All ports used across docker-compose configurations:

| Port | Service | Description |
|------|---------|-------------|
| 5432 | PostgreSQL (ingest) | tab-api-ingest primary database |
| 5433 | PostgreSQL (elo-api) | tipsharks-elo-api ratings database |
| 6379 | Redis | Queue and cache for tab-api-ingest |
| 27017 | MongoDB | tipsharks-client backend document store |
| 9090 | tab-api-ingest | Express API + Prometheus metrics endpoint |
| 8000 | tipsharks-elo-api | FastAPI REST API + Web UI |
| 8001 | tipsharks-client backend | FastAPI mobile backend API |
| 8081 | tipsharks-client frontend | Expo dev server / Metro bundler |
| 8082 | health-check | Unified health aggregator (all services) |
| 16686 | Jaeger | Distributed tracing UI |
| 9091 | Prometheus | Metrics collection and querying |
| 3000 | Grafana | Dashboards and alerting |
| 8080 | Adminer | PostgreSQL web administration |

---

## Quick Start

### Prerequisites

- Docker & Docker Compose
- Node.js >= 18 (for tab-api-ingest and frontend)
- Python >= 3.12 (for elo-api and client backend)
- Yarn (for frontend)

### Start Infrastructure

Each service manages its own infrastructure dependencies via Docker Compose:

```bash
# Start tab-api-ingest dependencies (PostgreSQL, Redis, Jaeger, Prometheus, Grafana)
cd tab-api-ingest && docker compose up -d

# Start tipsharks-elo-api dependencies (PostgreSQL, Adminer)
cd tipsharks-elo-api && docker compose up -d

# Start MongoDB for client backend
docker run -d --name tipsharks-mongo -p 27017:27017 mongo:7
```

### Run Database Migrations

```bash
# tab-api-ingest
cd tab-api-ingest
cp .env.example .env        # Edit as needed
npm install
npm run prisma:generate
npm run prisma:migrate
npm run dev                  # Start dev server on :9090

# tipsharks-elo-api
cd tipsharks-elo-api
cp .env.example .env         # Edit as needed
pip install -e ".[dev]"
alembic upgrade head
uvicorn apps.backend.api.main:app --reload  # Start on :8000
```

### Start Client Backend

```bash
cd tipsharks-client/backend
pip install -r requirements.txt
# Ensure MONGO_URL and DB_NAME are set in .env
uvicorn server:app --reload --port 8001
```

### Start Mobile Frontend

```bash
cd tipsharks-client/frontend
yarn install
yarn start                   # Expo dev server on :8081
# In another terminal:
yarn android                 # or yarn ios, yarn web
```

---

## Environment Variables Checklist

### tab-api-ingest

| Variable | Default | Description |
|----------|---------|-------------|
| `DATABASE_URL` | `postgresql://racing:racing123@localhost:5432/racing_db` | PostgreSQL connection string |
| `REDIS_URL` | `redis://localhost:6379` | Redis connection string |
| `TAB_API_BASE_URL` | `https://api.tab.com.au` | TAB Affiliates API base URL |
| `TAB_API_KEY` | — | TAB API key (public API, often not required) |
| `MORNING_SCRAPE_CRON` | `0 6 * * *` | Cron expression for daily data scrape |
| `TIMEZONE` | `Pacific/Auckland` | Timezone for scheduling |
| `API_RATE_LIMIT_PER_MINUTE` | `100` | Max outbound requests per minute |
| `API_RETRY_ATTEMPTS` | `3` | HTTP retry count |
| `ENABLE_HARNESS_RACING` | `true` | Enable harness racing data collection |
| `ENABLE_THOROUGHBRED_RACING` | `true` | Enable thoroughbred racing data collection |
| `ENABLE_AUS_RACING` | `true` | Enable Australian racing data |
| `ENABLE_NZ_RACING` | `true` | Enable New Zealand racing data |
| `LOG_LEVEL` | `info` | Pino log level |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4318` | OpenTelemetry OTLP HTTP endpoint |
| `METRICS_PORT` | `9090` | Prometheus metrics HTTP port |

### tipsharks-elo-api

| Variable | Default | Description |
|----------|---------|-------------|
| `DATABASE_URL` | `postgresql+psycopg://tipsharks:tipsharks@db:5432/tipsharks` | PostgreSQL connection string |
| `TAB_BASE_URL` | `https://api.tab.co.nz/affiliates/v1` | TAB Affiliates API base URL |
| `TAB_MOCK_MODE` | `false` | Use mock data instead of live API |
| `TAB_DEFAULT_CATEGORY` | `H` | Default racing category (T/H/G) |
| `TAB_DEFAULT_COUNTRY` | `NZ` | Default country filter |
| `API_PORT` | `8000` | FastAPI server port |
| `API_ADMIN_TOKEN` | `change_me_in_production` | Bearer token for admin endpoints |
| `API_CORS_ALLOW_ORIGINS` | `*` | CORS allowed origins |
| `LOG_LEVEL` | `INFO` | Log level |
| `ELO_SCALE_C` | `250.0` | Elo scale factor |
| `ELO_K_BASE` | `24.0` | Base K-factor for rating updates |
| `INITIAL_RATING` | `1500.0` | Default starting rating |
| `INITIAL_RD` | `350.0` | Default rating deviation |
| `DRIVER_WEIGHT_ALPHA` | `0.15` | Driver contribution weight |
| `TRAINER_WEIGHT_BETA` | `0.05` | Trainer contribution weight |
| `HRNZ_CLUB_CODES` | `all` | HRNZ club codes to scrape |

### tipsharks-client backend

| Variable | Default | Description |
|----------|---------|-------------|
| `MONGO_URL` | `mongodb://localhost:27017` | MongoDB connection string |
| `DB_NAME` | `test_database` | MongoDB database name |

### tipsharks-client frontend

| Variable | Default | Description |
|----------|---------|-------------|
| `EXPO_PUBLIC_BACKEND_URL` | — | Backend API URL for mobile app |
| `EXPO_TUNNEL_SUBDOMAIN` | — | Expo tunnel subdomain |

---

## Development Commands

### tab-api-ingest (TypeScript)

| Task | Command |
|------|---------|
| Install dependencies | `npm install` |
| Start dev server | `npm run dev` |
| Build | `npm run build` |
| Type check | `npm run type-check` |
| Lint | `npm run lint` |
| Format | `npm run format` |
| Prisma generate | `npm run prisma:generate` |
| Prisma migrate | `npm run prisma:migrate` |
| Prisma studio | `npm run prisma:studio` |

### tipsharks-elo-api (Python)

| Task | Command |
|------|---------|
| Install with dev deps | `pip install -e ".[dev]"` |
| Start dev server | `uvicorn apps.backend.api.main:app --reload` |
| Lint | `ruff check .` |
| Format | `black .` |
| Type check | `mypy packages/ apps/` |
| Alembic migrate | `alembic upgrade head` |
| New migration | `alembic revision --autogenerate -m "description"` |
| Run CLI worker | `python -m apps.worker.cli` |

### tipsharks-client backend (Python)

| Task | Command |
|------|---------|
| Install dependencies | `pip install -r requirements.txt` |
| Start dev server | `uvicorn server:app --reload --port 8001` |
| Format | `black .` |
| Lint | `flake8 .` |

### tipsharks-client frontend (TypeScript / React Native)

| Task | Command |
|------|---------|
| Install dependencies | `yarn install` |
| Start Expo dev server | `yarn start` |
| Run on Android | `yarn android` |
| Run on iOS | `yarn ios` |
| Run on web | `yarn web` |
| Lint | `yarn lint` |
| Format | `yarn format` |
| Type check | `yarn typecheck` |

---

## Testing

### tab-api-ingest

```bash
# All tests
npm test

# By category
npm run test:unit
npm run test:integration
npm run test:e2e

# With coverage
npm run test:coverage

# Single file
npx jest path/to/test.ts
```

### tipsharks-elo-api

```bash
# All tests
pytest

# With coverage
pytest --cov=packages --cov=apps

# Single file
pytest tests/test_file.py

# By keyword pattern
pytest -k "rating"
```

### tipsharks-client frontend

```bash
cd tipsharks-client/frontend

# All tests
yarn test

# Watch mode
yarn test:watch

# With coverage
yarn test:coverage
```

### tipsharks-client backend

```bash
cd tipsharks-client

# Backend integration tests
python backend_test.py
```

---

## Project Structure

```
tipsharks/
├── AGENTS.md                          # Agent instructions (root)
├── tab-api-ingest/                    # TypeScript data ingestion service
│   ├── src/                           # Express API, services, workers, routes
│   ├── prisma/                        # Schema + migrations
│   ├── scripts/                       # Utility scripts
│   ├── docker-compose.yml             # PostgreSQL, Redis, Jaeger, Prometheus, Grafana
│   └── Dockerfile                     # Production image
│
├── tipsharks-elo-api/                 # Python ratings engine
│   ├── apps/                          # API, Web UI, CLI worker
│   │   ├── api/                       # FastAPI REST endpoints
│   │   ├── web/                       # Web UI (Bootstrap 5, Chart.js)
│   │   └── worker/                    # Click CLI for ingest/recompute
│   ├── packages/                      # Shared libraries
│   │   ├── common/                    # Settings, logging, utilities
│   │   ├── tab_client/                # TAB API HTTP client
│   │   ├── hrnz_scraper/              # HRNZ data scraper
│   │   ├── ratings/                   # Multi-entity Elo engine
│   │   └── storage/                   # SQLAlchemy models, repositories
│   ├── tests/                         # pytest suite
│   ├── docs/                          # Architecture, data model, rating math
│   └── infrastructure/                # Docker, CI/CD configs
│
└── tipsharks-client/                  # Mobile app + backend
    ├── frontend/                      # React Native / Expo app
    │   ├── app/                       # Expo Router screens
    │   ├── src/                       # Components, services, store, types
    │   └── package.json               # Dependencies
    ├── backend/                       # FastAPI + MongoDB backend
    │   ├── server.py                  # All endpoints + mock data (single file)
    │   └── requirements.txt           # Python dependencies
    └── tests/                         # Test suite
```

---

---

## Design Tokens

The monorepo includes a shared design token system to ensure visual consistency across all three services.

| File | Format | Purpose |
|------|--------|---------|
| [`design-tokens.json`](design-tokens.json) | JSON | Single source of truth for all tokens |
| [`design-tokens.schema.json`](design-tokens.schema.json) | JSON Schema | Validate the tokens file |
| [`design-tokens.css`](design-tokens.css) | CSS custom properties | Web UIs (tab-api-ingest, tipsharks-elo-api) |
| [`design-tokens.js`](design-tokens.js) | ES module | React Native (tipsharks-client) |
| [`shared/design-tokens/README.md`](shared/design-tokens/README.md) | Markdown | Full usage documentation |

### Color Scheme

- **Primary dark navy** `#0f172a` — navbars, headers, dark mode backgrounds
- **Cyan accent** `#06b6d4` — CTAs, links, highlights
- **Semantic colors** for success (green), warning (amber), danger (red), info (blue)

### Quick Start

**Web** — Import the CSS file and use custom properties:

```html
<link rel="stylesheet" href="/design-tokens.css" />
<div style="background: var(--ts-color-brand-navy); color: var(--ts-color-text-inverse);">
  <h1 style="color: var(--ts-color-brand-cyan);">TipSharks</h1>
</div>
```

**React Native** — Import the JS module:

```js
import { colors, spacing, borderRadius } from '../../design-tokens';
```

See [`shared/design-tokens/README.md`](shared/design-tokens/README.md) for full documentation.

---

## Observability

| Tool | Access | Purpose |
|------|--------|---------|
| **Jaeger** | http://localhost:16686 | Distributed tracing across services |
| **Prometheus** | http://localhost:9091 | Metrics collection (tab-api-ingest) |
| **Grafana** | http://localhost:3000 | Dashboards (default: admin/admin) |
| **Adminer** | http://localhost:8080 | PostgreSQL admin (tipsharks-elo-api) |
| **Health Check** | http://localhost:8082/health | Unified health status for all services |

---

## Technology Stack Summary

| Layer | Technology |
|-------|-----------|
| **Frontend** | React Native (Expo SDK 54), TypeScript, Zustand, React Query |
| **Backend APIs** | FastAPI (Python), Express 5 (TypeScript) |
| **Databases** | PostgreSQL 16, Redis 7, MongoDB 7 |
| **ORM / ODM** | Prisma, SQLAlchemy 2.0, Motor |
| **Message Queue** | BullMQ (Redis-backed) |
| **Rating Engine** | Multi-entity Elo with pairwise logistic model |
| **Observability** | OpenTelemetry, Jaeger, Prometheus, Grafana |
| **Validation** | Zod (TypeScript), Pydantic (Python) |
| **Containerization** | Docker + Docker Compose |

---

## Production Deployment

Production deployment configuration lives in the root of the monorepo:

| File | Purpose |
|------|---------|
| [`docker-compose.prod.yml`](docker-compose.prod.yml) | Production service orchestration with resource limits and health checks |
| [`.env.prod.example`](.env.prod.example) | Environment variable template with all required secrets |
| [`infrastructure/nginx/nginx.conf`](infrastructure/nginx/nginx.conf) | Nginx reverse proxy config (HTTP baseline; HTTPS-ready) |
| [`infrastructure/README.md`](infrastructure/README.md) | Full deployment guide (SSL, scaling, backups, monitoring) |

### Quick Start

```bash
# 1. Configure secrets
cp .env.prod.example .env.prod
# Edit .env.prod — change all passwords and tokens

# 2. Start the stack
docker compose -f docker-compose.prod.yml --env-file .env.prod up -d

# 3. Verify
docker compose -f docker-compose.prod.yml ps
curl http://localhost/health
```

See [`infrastructure/README.md`](infrastructure/README.md) for SSL setup, scaling guidance, backup scripts, and security hardening.

---

## License

MIT