# Grafana Dashboard Troubleshooting Guide

## Issue: "No Data" and "No default database configured" Errors

### Root Cause ✅ FIXED
The PostgreSQL datasource provisioning file was using the wrong type identifier:
- **Wrong**: `type: postgres`
- **Correct**: `type: grafana-postgresql-datasource`

### What Was Fixed

**File**: `grafana/provisioning/datasources/postgres.yml`

The datasource type has been corrected to use the proper Grafana PostgreSQL plugin identifier.

## Verification Steps

### 1. Check Datasource Configuration
```bash
./scripts/validate-grafana-datasources.sh
```

**Expected Output**:
```
✅ PostgreSQL: Connected
✅ Prometheus: Connected
```

### 2. Test SQL Queries
```bash
./scripts/test-dashboard-queries.sh
```

**Expected Output**: All queries should succeed and return data

### 3. Verify Data Exists
```bash
docker compose exec postgres psql -U racing -d racing_db -c \
  'SELECT COUNT(*) FROM races WHERE "startTime"::date = CURRENT_DATE;'
```

**Current Data**:
- ✅ 71 races today
- ✅ 9 meetings today
- ✅ 19 job runs recorded
- ✅ Last update: ~11 minutes ago

## Current Status

### ✅ All Systems Operational

1. **PostgreSQL Datasource**:
   - Type: `grafana-postgresql-datasource` ✅
   - Database: `racing_db` ✅
   - User: `racing` ✅
   - Connection: Healthy ✅

2. **SQL Queries**:
   - All column names properly quoted ✅
   - Date filtering using correct columns ✅
   - Queries execute without errors ✅

3. **Data Availability**:
   - Database has real race data ✅
   - Job runs recorded ✅
   - Timestamps current ✅

## If You Still See "No Data" Errors

### Solution 1: Hard Refresh the Browser
1. Open the Grafana dashboard
2. Press `Ctrl+Shift+R` (Windows/Linux) or `Cmd+Shift+R` (Mac)
3. This clears browser cache and forces reload

### Solution 2: Check Panel Query Settings
In Grafana UI:
1. Edit a panel showing "No Data"
2. Check the "Query" tab
3. Verify datasource shows "PostgreSQL" (not "-- Mixed --")
4. Click "Run queries" button to test

### Solution 3: Check Browser Console
1. Open browser DevTools (F12)
2. Go to Console tab
3. Look for any error messages
4. Common issues:
   - CORS errors (unlikely with local setup)
   - Query timeout errors
   - Datasource UID mismatch

### Solution 4: Verify Dashboard UID References
The dashboard should reference datasource UID `PCC52D03280B7034C`. Check:
```bash
grep -c "PCC52D03280B7034C" grafana/dashboards/scheduler-monitoring.json
```
Should return: `16` (one for each PostgreSQL panel)

### Solution 5: Force Dashboard Reload
```bash
# Touch the file to update timestamp
touch grafana/dashboards/scheduler-monitoring.json

# Restart Grafana
docker compose restart grafana

# Wait 10 seconds for reload
sleep 10
```

## Understanding the Error Message

When you see: "You do not currently have a default database configured..."

This means:
1. Grafana found a datasource reference
2. But the datasource type doesn't match any installed plugin
3. Or the datasource isn't properly configured

The fix was changing `type: postgres` → `type: grafana-postgresql-datasource` because:
- `postgres` is just a generic identifier
- `grafana-postgresql-datasource` is the actual plugin ID Grafana recognizes

## Dashboard Configuration

### Datasource References
Every PostgreSQL panel in the dashboard uses:
```json
{
  "datasource": {
    "type": "grafana-postgresql-datasource",
    "uid": "PCC52D03280B7034C"
  }
}
```

### SQL Query Format
All queries use quoted identifiers for camelCase columns:
```sql
SELECT
  "startTime",
  "meetingId",
  "raceNumber"
FROM races
WHERE "startTime"::date = CURRENT_DATE
```

### Auto-Refresh
Dashboard is configured to auto-refresh every 10 seconds:
```json
{
  "refresh": "10s"
}
```

## Testing Individual Panels

To test a specific panel's query:

1. **Get the query from dashboard JSON**:
```bash
jq '.dashboard.panels[12].targets[0].rawSql' \
  grafana/dashboards/scheduler-monitoring.json
```

2. **Test it directly in PostgreSQL**:
```bash
docker compose exec postgres psql -U racing -d racing_db -c "<query>"
```

3. **If query works in PostgreSQL but not Grafana**:
   - Check datasource UID matches
   - Verify datasource type is correct
   - Try hard refresh in browser

## Common PostgreSQL Query Issues

### Issue: Column "xxx" does not exist
**Cause**: CamelCase column names need quotes in PostgreSQL
**Fix**: Use `"columnName"` instead of `columnName`

### Issue: No such column "date" on races table
**Cause**: The `date` column is on `meetings` table, not `races`
**Fix**: Use `"startTime"::date` to cast the timestamp

### Issue: Query timeout
**Cause**: Query is too slow or database is under load
**Fix**:
- Check database indexes
- Simplify query
- Increase timeout in panel settings

## Direct Database Access

For debugging, connect directly to the database:
```bash
# Connect to PostgreSQL
docker compose exec postgres psql -U racing -d racing_db

# List tables
\dt

# Describe races table
\d races

# Test a query
SELECT COUNT(*) FROM races WHERE "startTime"::date = CURRENT_DATE;

# Exit
\q
```

## Useful Commands

```bash
# Check Grafana logs
docker compose logs grafana --tail 50

# Check PostgreSQL logs
docker compose logs postgres --tail 50

# Restart just Grafana
docker compose restart grafana

# Restart everything
docker compose restart

# Check datasource health via API
curl -u admin:admin -X POST \
  http://localhost:3000/api/datasources/uid/PCC52D03280B7034C/health

# List all datasources
curl -u admin:admin http://localhost:3000/api/datasources | jq '.'
```

## Files Modified

1. ✅ `grafana/provisioning/datasources/postgres.yml`
   - Changed type to `grafana-postgresql-datasource`
   - Verified database, user, password are set

2. ✅ `grafana/dashboards/scheduler-monitoring.json`
   - All datasource references use correct type
   - All datasource references include UID
   - All SQL queries use quoted identifiers

## Next Steps

If everything validates correctly but you still see errors in the UI:

1. ✅ Check that you're logged into Grafana (admin/admin)
2. ✅ Navigate directly to: http://localhost:3000/d/scheduler-monitoring
3. ✅ Wait 10 seconds for auto-refresh
4. ✅ Try hard refresh (Ctrl+Shift+R)
5. ✅ Check browser console for JavaScript errors
6. ✅ Verify you're on the correct dashboard (check title)

## Support

If issues persist after following this guide:
1. Run `./scripts/validate-grafana-datasources.sh` and share output
2. Run `./scripts/test-dashboard-queries.sh` and share output
3. Check browser console and share any error messages
4. Share screenshot of the specific panel showing "No Data"