# AGENTS.md **tab-api-ingest** — Near-realtime horse racing data ingestion service for AUS/NZ thoroughbred and harness racing. **Precedence:** this file overrides root `AGENTS.md` for files in `tab-api-ingest/`. ## Commands > Source: package.json scripts | Task | Command | ~Time | |------|---------|-------| | Install | `npm install` | ~30s | | Dev | `npm run dev` | - | | Build | `npm run build` | ~10s | | Typecheck | `npm run type-check` | ~5s | | Lint | `npm run lint` | ~5s | | Lint fix | `npm run lint:fix` | ~5s | | Format | `npm run format` | ~5s | | Test (all) | `npm test` | ~30s | | Test (unit) | `npm run test:unit` | ~15s | | Test (integration) | `npm run test:integration` | ~15s | | Test (e2e) | `npm run test:e2e` | ~30s | | Test (single) | `npx jest ` | ~5s | | Prisma generate | `npm run prisma:generate` | ~5s | | Prisma migrate | `npm run prisma:migrate` | ~10s | | Prisma studio | `npm run prisma:studio` | - | ## Stack - **Language**: TypeScript (Node.js >=18) - **Framework**: Express 5 - **Database**: PostgreSQL via Prisma ORM - **Queue**: BullMQ + Redis - **Validation**: Zod - **Logging**: Pino - **Testing**: Jest + Supertest + Nock - **Linting**: ESLint + Prettier - **Metrics**: OpenTelemetry + Prometheus - **Scheduling**: node-cron ## File Map ``` tab-api-ingest/ ├── src/ │ ├── index.ts # Entry point │ ├── services/ # Business logic services │ ├── workers/ # BullMQ workers │ ├── schedulers/ # Cron-based schedulers │ ├── routes/ # Express routes │ ├── middleware/ # Express middleware │ ├── utils/ # Shared utilities │ └── types/ # TypeScript type definitions ├── prisma/ │ ├── schema.prisma # Database schema │ └── migrations/ # Prisma migrations ├── scripts/ # Utility scripts ├── docs/ # Documentation ├── grafana/ # Grafana dashboard configs ├── jest.config.js # Jest configuration ├── tsconfig.json # TypeScript configuration ├── .eslintrc.json # ESLint configuration ├── .prettierrc # Prettier configuration ├── docker-compose.yml # Docker Compose (Redis, PostgreSQL) ├── Dockerfile # Production Docker image └── package.json # Dependencies and scripts ``` ## Overview Near-realtime horse racing data ingestion service for AUS/NZ thoroughbred and harness racing. Fetches data from TAB and HRNZ APIs, validates with Zod, stores via Prisma, and processes jobs through BullMQ workers. ## Setup ```bash npm install cp .env.example .env # Edit .env with your configuration npm run prisma:generate npm run prisma:migrate npm run dev ``` **Dependencies**: Node.js >=18, PostgreSQL, Redis **Docker**: `docker compose up -d` starts PostgreSQL + Redis ## Code style - TypeScript strict mode - ESLint + Prettier for formatting - Zod for all external data validation - Pino for logging (no console.log) - File names: kebab-case, exports: PascalCase for classes, camelCase for functions ## Security - Never commit `.env` or credentials - Sanitize all external API responses before storage - Use environment variables for all configuration - Rate limit outbound API requests - Validate all incoming data with Zod schemas ## Checklist Before committing: - [ ] `npm run type-check` passes - [ ] `npm run lint` passes - [ ] Relevant tests pass - [ ] No console.log statements - [ ] Zod validation on new endpoints ## Examples | For | Reference | Key patterns | |-----|-----------|--------------| | Service | `src/services/` | Zod validation, Pino logging, error handling | | Worker | `src/workers/` | BullMQ job processing, retry logic | | Route | `src/routes/` | Express router, middleware chain | | Prisma model | `prisma/schema.prisma` | Schema definitions, relations | | Test | `src/**/*.test.ts` | Jest patterns, Nock mocking | ## When stuck - Check `docs/` for architecture and API documentation - Review `prisma/schema.prisma` for data model - Check Grafana dashboards for runtime metrics - Review BullMQ queue state in Redis ## Utilities | Need | Use | Location | |------|-----|----------| | DB client | Prisma Client | `@prisma/client` | | HTTP client | Axios + axios-retry | `src/` | | Rate limiter | Bottleneck | `src/` | | Queue | BullMQ | `src/workers/` | | Validation | Zod schemas | `src/types/` | | Logging | Pino | `src/utils/` | ## Heuristics | When | Do | |------|-----| | Adding DB field | Update Prisma schema + run migration | | Adding API endpoint | Add route + Zod validation + test | | Adding background job | Add BullMQ worker + test | | Adding dependency | Ask first - we minimize deps | | Unsure about pattern | Check Golden Samples above | ## Repository Settings - **Package manager**: npm - **Node version**: >=18.0.0 - **Test framework**: Jest - **Lint**: ESLint + Prettier - **Type check**: `tsc --noEmit` ## Boundaries ### Always Do - Run `npm run type-check` before committing TypeScript changes - Run `npm run lint` before committing - Run relevant tests before committing - Use Zod for all external data validation - Use Pino for all logging (no console.log) - Handle errors with proper HTTP status codes ### Never Do - Commit `.env` files (use `.env.example` as template) - Use `console.log` — use Pino logger instead - Skip Zod validation on external API responses - Hardcode TAB API URLs — use environment variables - Store raw API responses without sanitizing ## Terminology | Term | Means | |------|-------| | TAB | Totalisator Agency Board — Australian/NZ betting operator | | HRNZ | Harness Racing New Zealand | | Meeting | A race day at a specific venue | | Race | An individual race within a meeting | | Runner/Starter | A horse entered in a race | | Scraper | Data ingestion from external APIs |