# 🏇 Horse Racing Data Scraper

Near-realtime horse racing data collection system for Australian and New Zealand thoroughbred and harness racing.

## 📋 Overview

This system scrapes race data from TAB Affiliates API, collecting meeting, race, and runner information throughout the day. The data is used for future Elo rating calculations to generate win/place probabilities.

## 🚀 Quick Start

### Prerequisites

- Node.js >= 18.0.0
- PostgreSQL >= 14
- Redis >= 6.0
- npm >= 9.0.0

### Installation

```bash
# Clone the repository
git clone <repository-url>
cd racing-scraper

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Edit .env with your configuration

# Generate Prisma client
npm run prisma:generate

# Run database migrations
npm run prisma:migrate

# Start development server
npm run dev
```

## 📁 Project Structure

```
racing-scraper/
├── src/
│   ├── api/              # API client implementations
│   ├── config/           # Configuration
│   ├── models/           # Prisma client and types
│   ├── schedulers/       # Cron jobs
│   ├── services/         # Business logic
│   ├── utils/            # Utilities and helpers
│   ├── workers/          # Queue workers
│   └── index.ts          # Application entry point
├── prisma/
│   └── schema.prisma     # Database schema
├── docs/                 # Documentation
├── scripts/              # Utility scripts
└── tests/                # Test files
```

## 🔧 Development

```bash
# Run in development mode with watch
npm run dev

# Build for production
npm run build

# Start production server
npm start

# Run tests
npm test

# Run tests with coverage
npm test:coverage

# Lint code
npm run lint

# Format code
npm run format

# Type check
npm run type-check
```

## 📊 Database

```bash
# Open Prisma Studio (database GUI)
npm run prisma:studio

# Create a new migration
npm run prisma:migrate

# Reset database (⚠️ destroys all data)
npx prisma migrate reset
```

## 🗓️ Data Collection Flow

1. **Morning Scrape (6:00 AM)**: Fetch all meetings and races for the day
2. **Pre-Race Updates (T-60, T-15)**: Check for scratches and condition changes
3. **Post-Race (T+5)**: Collect race results

## 📚 Documentation

See [PROJECT_BRIEF.md](./PROJECT_BRIEF.md) for comprehensive project documentation including:
- Technical architecture
- Database schema
- API specifications
- Implementation phases
- Configuration options

## 🔐 Environment Variables

See `.env.example` for all available configuration options.

## 🧪 Testing

```bash
# Run all tests
npm test

# Run tests in watch mode
npm test:watch

# Run with coverage
npm test:coverage
```

## 📝 License

MIT

## 👥 Authors

- Tom

## 🤝 Contributing

See [CONTRIBUTING.md](./CONTRIBUTING.md) for contribution guidelines.