$iSrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJ r J r SSK r SSK J r SSKJr SSKJrJr SSKJr SS KJr SS KJr SS KJr SS KJrJr SS KJrJ r \"5 \"\!5r"\ "5r#S\ S\$S\ 4Sjr%\ RL"5S5r'\'RQS5\ RR"SSSSS9\ RR"SSSSS9\ RR"SSSSS 9S\*S\*S!\$SS4S"j5555r+\'RQ5\ RR"SSSSS9\ RR"SSSSS9\ RR"S#S$SS%S9\ RR"S&S'S\ RX"/S(QSS)9S*S+9S\*S\*S$\*S'\*4S,j55555r-\'RQ5\ RR"SSSSS9\ RR"SSSSS9\ RR"S-SS.S/9S\*S\*S0\.4S1j5555r/\'RQ5\ RR"S2S3SS4S9\ RR"S5SSS6S 9\ RR"SSSS7S9\ RR"SSSS8S9S3\*S9\.S\*S\*4S:j55555r0\'RQ5\ RR"SSSSS9\ RR"SSSSS9\ RR"S;S<\ RX"/S=Q5S>SS?S@9\ RR"SASBSCSSDS 9\ RR"S5SSS6S 9S\*S\*S<\*SB\*S9\.4 SEj555555r1\'RQ5SF5r2\'RQ5SG5r3\'RQ5\ RR"SSSSS9\ RR"SSSSS9\ RR"SHSISSJS9SK5555r4\'RQ5\ RR"SLSM\ Rj"SNSO5SSPSQ9\ RR"S&\ RX"/S(QSS)9S*SR9\ RR"SS\ RX"STSU/SS)9STSSVS@9\ RR"S-SS.S/9\ RR"SWSSXS/9\ RR"SYSSZS/9S[5555555r6\!S\:Xa\'"5 gg)]z9Worker CLI for TipSharks ingestion and computation tasks.N) monthrange)date timedelta)Console)Table) get_logger setup_logging) get_settings) parse_date) get_session)IngestionService)MeetingRace)StarterRepositorynormalize_race_datadmonthsreturncURS- U- nURUS--nUS-S-n[X25Sn[URU5n[ X2U5$)zSubtract *months* from *d*, clamping day to month length. Args: d: Starting date. months: Number of months to subtract (must be >= 0). Returns: Date *months* months before *d*.  )monthyearrmindayr)rr target_month target_yearmax_day target_days  )!- = 5aS?@  s86 I)3I3D I)I) I&!I)) K3AJ>>Kz--clearz'Clear existing ratings before recompute)is_flagr)clearc[U5n[U5n[RSUSUS35 U(a[RS5 SSKJn [ 5nU"XcXBS9nSSS5 [RS WS 35 g!,(df  N(=f![ aPn[RS U3S S 9 [RSUS35 [R"S5 SnAgSnAff=f)zRecompute ratings from stored race results. This will deterministically recompute all ratings in the date range. Examples: recompute --from 2024-01-01 --to 2024-12-31 recompute --from 2024-01-01 --to 2024-12-31 --clear [bold]Recomputing ratings from r_r`z-[yellow]Clearing existing ratings...[/yellow]rrecompute_ratings)clear_existingNu [green]✓ Recompute complete: ! rating snapshots created[/green]Recompute failed: Trt [red]✗ Recompute failed: rvr) r r@rApackages.core.ratings.recomputerr rrrrwrx) r'r+rrJrKrrLsnapshot_countrs r recomputers6I&J'"H MM +J# T"5IShvN n[T!S5GHup[RSUS[T!5SUS35 UR U5IShvN nUR S5nU(a7TR U5nTUs=::aT::dO [RS5 MT"5nURU5nURU5n[5n T(aUU R[5R[RUS:H5RS S 9 U R5 TR!X5 T S ==S- ss'US H9n TR!U U S U SU R S55 T S ==S- ss'M; USH2n TR!XSU R S 5S9 T S==S- ss'M4 USH3n TR!U U SU R S 5S9 T S==S- ss'M5 UR#X7S5n 0nU H6nTR!XSU5nURXS'T S==S- ss'M8 UR%X>5nUH7nTR "U USUUR S55 T S==S- ss'M9 U R5 SSS5 [RS[W 5S[W5S35 GM SSS5IShvN gGN GN!,(df  NX=f![&aMnT S==S- ss'[(R+SUSU35 [RSUS 35 SnAGMcSnAff=fNv!,IShvN (df  g=f7f)!Nr[r4 ] Scraping ...r) [dim]Skipped (outside date range)[/dim]meetingFsynchronize_sessionrrr;namer1r driver_idr trainer_idr=rrMr3r2 [green]✓ Imported:  races,  starters[/green]rqFailed to scrape :  [red]✗ Error: rv)rBr@rAr?get_meeting_resultsrD fromisoformat map_meeting map_entitiesr r6rr8r;deleterIrH map_races map_startersrrr)"scraperrRurlscrapedmeeting_date_strr9mapperrentitiesrLhorsedrivertrainerr race_id_maprSrace_objr2rUrrrrrrrrr date_typerKrrJrurlss" r scrape_and_import&scrape_hrnz..scrape_and_imporths"}} )$ 2HCY! #aD {+cU#&NO)0(C(CC(H"H,3;;v+>(++4+B+BCS+TL$.,$J($J ' $O!"!)"0!1"("4"4W"=#)#6#6w#?)]g( ' g 6 = =$+JJ')2D$D!""(&U&"C ' 0.44WF!*-2-*2(); / 6 6$+$)$K$)&M$)IIj$9 !" !&h1 4*<+39*= 0 7 7$+F^vzzRVGW!8!"!&i 0A 5 0 +> ,4J+? 1 8 8$+$+FO/6{{4/@!9!" !&j 1Q 6 1 ,@%+$4$4Wi>P$QE*,K(-+9+@+@$+Y-?,"DL;; ,? @ %g! 3 ).(.':':7'PH+3 1 8 8$+$+I$6$+$+KK $: !" !&j 1Q 6 1,4$NN,q+t 5c%j\"8}o->@c!3%}} #I$+]~%!h1, '8Rs%CD (:1#V&DE !o%}}}s N'LN'N ? MM[  EF\ !\ !\ !|  %'(34 0 A j#eJ&7"89 gs5>23 j#eJ&7"89 hE(O 45 iU9%5!67 j#eJ&7"89  h  ?Q.%G   d e ?Q  MM x 11Z[  MM\  MMP Q MMZ i@_  4YKvFG  n  (,t < 4QCv>?  sh* K.5K:KK*K-K.FL.,*L.K K+&K.+K..9L+*L+. N8ANNz --race-type race_day_type) OfficialRaces TrialRaces WorkoutRacesrz-Race day type filter from the results enquiry)r]r-r.r)z--clubclub_noz7Optional club number filter (leave blank for all clubs)cL^^^^^ ^ ^ ^ ^ ^^^^^^SSKJm SSKJmJm Jm Jm JmJm SSK J m SSK J m [U5m[U5m[RSTSTS35 [RS 5 SSSSSSSS .mUU U U U U UUUUUUUUU4S jn[ R""U"55 [%S S 9nUR'SSS9 UR'SSSS9 UR)S[+TS55 UR)S[+TS55 UR)S[+TS55 UR)S[+TS55 UR)S[+TS55 UR)S[+TS 55 UR)S![+TS"5TS"S:aS#OSS9 [RS$5 [RU5 TS"S:a2[RS%TS"S&35 [RS'5 g0[RS(5 [RS)5 g0![,aPn[.R1S*U3S+S,9 [RS-US.35 [2R4"S/5 S0nAg0S0nAff=f)1z# T"5IShvN nURT#T T"TS9Vs/sShvN nUPM N'N Os snfnn[US5GHup4USn[RSUS[ U5SUS35 UR XT5IShvN nUR S5(d,UR S 5(aUS R5US'UR S 5(d[RS 5 MUR S5nU(a8TRU5nT#Us=::aT ::dO [RS 5 GM T"5n U RU5nU RU5n [5n T!(aUU R[5R[RUS :H5R!SS9 U R#5 TR%X5 T$S==S- ss'U SH9n TR%U U SU SU R S55 T$S==S- ss'M; U SH2n TR%XSU R S5S9 T$S==S- ss'M4 U SH3nTR%U USUR S5S9 T$S==S- ss'M5 U R'XaS 5n0nUH7nTR%XS U5nURUUS'T$S ==S- ss'M9 U R)UU5nUH7nTR$"U USUUR S55 T$S==S- ss'M9 U R#5 SSS5 O!,(df  O=f[RS[ W5S[ W5S35 GMq![*a\nT$S ==S- ss'[,R/S!UR S5S"U35 [RS#US$35 SnAGMSnAff=f SSS5IShvN g!,IShvN (df  g=f7f)%N)rrr results_urlrr4rrrr9rz% [dim]Skipped (no races found)[/dim]rrFrrrr;rr1rrrrr=rMr3r2rrrrqrrrrv) iter_meetingsrBr@rAr?rrD isoformatrrrr r6rr8r;rrIrHrrrrr)%rrrrR meeting_metarrrr9rrrLrrrrrrSrr2rUrrrrrrrrrrrrKrrrJrs% r r.scrape_hrnz_enquiry..scrape_and_import8sA355*1)>)>" &3 ' *?*%6*38Q)?%CZ!*=9 #aH k#c&RS(/(C(CC(V"V"V&{{622|7G7G7W7W.:>.J.T.T.VGFO&{{733#MM*QR$+2;;v+>(++4+B+BCS+TL$.,$J($J ' $O!"!)!/!1"("4"4W"=#)#6#6w#?(]g( ' g 6 = =$+JJ')2D$D!""(&U&"C ' 0-44WF!*-2-)1(); / 6 6$+$)$K$)&M$)IIj$9 !" !&h1 4*<+39*= 0 7 7$+F^vzzRVGW!8!"!&i 0A 5 0 +> ,4J+? 1 8 8$+$+FO/6{{4/@!9!" !&j 1Q 6 1 ,@%+$4$4Wi>P$QE*,K(-+9+@+@$+Y-?,"DL;; D,? @ %g! 3 ).(.':':7K'PH+3 1 8 8$+$+I$6$+$+KK $: !" !&j 1Q 6 1,4$NN,i+]]l 5c%j\"8}o->@%!h1, / 0@0@0O/PPRSTRUV (:1#V&DE !k*@655555s Q7QP>=;9 ;=Q;=P>AOBA2O P> A OP>3OG&M?6 O? N 5O>P> P' AP"P>"P''P>, Q7P:8Q>QQ  QQz%HRNZ Results Enquiry Scraping Resultsrbrdrerfrhrirjrkrmrrnrror2rrrrrrrprqrrrrsrrrrrTrtrrvrN)rrrrrrrrrrrrrr r@rAr|r}rrrrrrrrwrx)r'r+rrrrrrrrrrrrrrrrKrJrs ``` @@@@@@@@@@@@r scrape_hrnz_enquiryrsZL+C;I&J'"H MM -j\hZyQ MM[  EOf !f !f !P  %'(CD 0 A j#eJ&7"89 gs5>23 j#eJ&7"89 hE(O 45 iU9%5!67 j#eJ&7"89  h  ?Q.%G   d e ?Q  MM x 11Z[  MM\  MMP Q MMZ   (,t < 4QCv>?  s FI *I J#AJJ#c $SSKJnJn [R S5 U"5nU"U5n[R U5 UR S/5nUR S/5nUR S5(a[R SUSS35 U(a.[R S 5 [R S U<35 U(aU(d[R S 5 gU(d;U(d3[R S [ UR S /55S35 ggg![aPn[RSU3SS9 [R SUS35 [R"S5 SnAgSnAff=f)zRefresh HRNZ club codes by scraping the HRNZ results pages. Fetches the latest list of club codes from HRNZ, compares them with the hardcoded ``HRNZ_ALL_CLUB_CODES`` list, and reports any differences (new, missing, or unmatched codes). r)generate_diff_reportrefresh_club_codesz6 [bold cyan]Refreshing HRNZ club codes...[/bold cyan] newmissingrrs [/yellow]u [yellow]⚠ New club codes detected. Update HRNZ_ALL_CLUB_CODES in packages/core/common/settings.py to include these codes.[/yellow]z Add to settings.py: u? [green]✓ All hardcoded club codes were found online.[/green]u5 [green]✓ Hardcoded club codes are up to date (all fetchedz codes confirmed).[/green]zClub code refresh failed: Trtu$ [red]✗ Club code refresh failed: rvrN) "packages.hrnz_scraper.club_refreshrrr@rArDr?rrrrwrx)rrresultreport new_codes missing_codesrs r rrsK  MMLM!#%%f- fJJub)  9b1 ::g   MMN6'?*;9E F  MM3  MM5i]C D  MMR = MMFJJy"5677QS $1  1!5 E =aSGH  s CD52AD55 F?AF  Fc[5n[SS9nURSSS9 URSSS9 URSURR 5 URS URR 5 URS URR5 URS URR(aS OS 5 URSSURR;a(URRRS5SOURR5 URSURR5 URS[URR 55 URS[URR"55 URS[URR$55 URS[URR&55 URSURR((aS OS 5 URSURR*(aS OS 5 URSURR,(aS OS 5 [.R1U5 g)z"Display configuration information.zTipSharks ConfigurationrbSettingrerfValueyellowz TAB Base URLzTAB Default CategoryzTAB Default Countryz TAB Mock Modeu✓u✗z Database URL@z Log Levelz Elo Scale (C)zElo KuDriver Weight (α)uTrainer Weight (β)z Enable DriverzEnable TrainerzEnable AdjustmentsN)r rrrrzbase_urlr{default_country mock_modedatabasersplitlogginglevelrrating elo_scale_c elo_k_basedriver_weight_alphatrainer_weight_beta enable_driverenable_trainerenable_adjustmentsr@rA)rrs r infors~H 1 2E Yf- WH- MM.(,,"7"78 MM((,,*G*GH MM')E)EF MM/HLL,B,B5N MMh''+++    ! ! ' ' ,R 0""&&  MM+x//556 MM/3x'B'B#CD MM'3x99:; MM&HOO,O,O(PQ MM'X__-P-P)QR MM/HOO,I,I5uU MM"X__-K-KEQVW MMx'I'Ieu MM%r"z--out output_filez Output JSON file path (optional)cZ SSKnSSKJnJn [R S5 [ U5n[ U5n[R SUSU35 [5nU"U5n U RXg5n U"U5n U (aU RRU 5 SSS5 [SS9n U RS S S 9 U RS S S 9 U RS[W R55 U RS[U R 55 U RS[U R"55 U RSSU R$S35 U RSSU R&S35 U RS[[)SU R5555 [R U 5 U R*(a[SS9n U RS S S 9 U RS S S 9 U R*R-5H@upUR/SS5R15nU RU[U55 MB [R U 5 U R(GaI[R S5 SGH-nU RVs/sHnUR2U:XdMUPM nnU(dM;[UR55S3S9nURSS S 9 URS S S 9 URS!S"S 9 USS#HPnURUR6UR8UR:(a[UR:5OS$5 MR [R U5 [=U5S#:dGM[R S%[=U5S#- S&US'35 GM0 U(Ga<U R>RA5U RBRA5U RDRA5U RU R U R"U R$U R&S(.U R*U RVs/sHSnUR2UR6UR8UR:URFURHURJS).PMU snS*.n[MUS+5nURN"UUS,S-9 SSS5 [R S.US/35 U RP(a,[R S05 [RRT"S15 g[R S25 [RRT"S5 g!,(df  GN=fs snfs snf!,(df  N=f)3zlGenerate data quality report for date range. Validates data completeness, accuracy, and identifies issues. rN)DataQualityValidatorcheck_data_freshnessz8[bold cyan]Generating Data Quality Report...[/bold cyan]z Date range: r_zData Quality SummaryrbMetricrerfrr zTotal Meetingsz Total RaceszTotal Startersrpz[red]rvWarningsz[yellow]rInfoc3H# UHoRS:XdMSv M g7f)rrN)severity).0is r data_quality..`sI=aJJ&4H=s" "z Data Metrics_ z! [bold]Issues by Category:[/bold])rwarningrz IssuesCategoryMessagezRace IDmagenta-z [dim]... and z more z issues[/dim])total_meetings total_racestotal_starters error_count warning_count)r$rXmessagerM meeting_id starter_iddetails)rJrK generated_atsummarymetricsissuesw)indentz [green]Report saved to r5u3 [bold red]⚠ Data quality check FAILED[/bold red]ru7 [bold green]✓ Data quality check PASSED[/bold green])+json!packages.core.common.data_qualityrrr@rAr r generate_reportr=appendrrrrr1r2r3r4r5sumr<itemsreplacercr$ryrXr6rMr?rJrrKr:r7r8r9rdump has_errorsrwrx)r'r+rrArrrJrKrL validatorrfreshness_issue summary_table metrics_tablekeyvalue formatted_keyrXr&category_issues issues_tableissue report_datars r data_qualityrU#s0  MMLMI&J'"H MML D ;< '(1 **:@/w7  MM  1  67MXV4WH5*C0E0E,FG-V-?-?)@A*C0E0E,FG(eF,>,>+?v$FG*1E1E0Fi&PQCI6==IIJ MM- ~~N3    8   9 ....0JCKKS1779M  ! !-U <1  m$}}} :;4H*0--R-Q1::;Qq-OR$hnn.>-?w+GH '' &'A'' 'B'' 'C,Sb1E (( .3mmEMM*2 l+'",MM'O(U;aOUS>(aEUS>S?nUS>S@n[RSAUSBSC35 [RSDUSBSC35 O[RSE5 [OSHSI9nURQSJSKSL9 URQSMSNSOSP9 URSSQ[+U 55 URSSR[+U S55 URSSS[+U S55 URSST[+U S55 URSSU[+U55 UbURSSVUSB5 UbURSSWUSB5 URSSX[+U S5U SS :aSYOSNSL9 U (a#URSSZ[+[-U 55SYSL9 [RS 5 [RU5 U SS :a[RS[U SS\35 g[RS]5 g!,(df  GN=f!["ajn[$R'SU SUSU3SS 9 [RS!US"35 U R)X[+U545 U S==S- ss'SnAGNSnAff=f!,(df  GN=f!["aQn[$R'S3U3SS 9 [RS4US"35 [2R4"S5 SnAGNSnAff=f!["a;n[$R'SFU3SS 9 [RSGUS+35 SnAGNSnAff=f)^a`Backfill historical data and recompute ratings. Ingests data week by week from (today - months) to today, then recomputes all ratings and evaluates prediction accuracy. Examples: backfill --months 12 --category H --source tab backfill --months 6 --category T --clear --learn-adjustments backfill --months 3 --source ingest --skip-eval z [bold]Backfilling z data from r_z[/bold]zSource: z , Months: z , Clear: z, Learn adjustments: rr)rrr2rq)daysrz[bold cyan]Week rz [/bold cyan])sourcerarrr2rqNu [green]✓ z meetings, rrzWeek z failed: Trtu [red]✗ Week failed: rvz! [bold]Ingestion complete:[/bold]z Weeks processed: z Total meetings: z Total races: z Total starters: rsz week(s) had errors:[/yellow]z [yellow]ru: [yellow]No data ingested — skipping recompute.[/yellow]rz ...[/bold]r)rlearn_adjustmentsu [green]✓ rrrz/ [bold]Evaluating prediction accuracy...[/bold]z..scriptszevaluate_accuracy.pyr&r*z--jsoniX)capture_outputtextchecktimeoutr>brier winner_accz [green]Winner accuracy: z.4fr5z [green]Brier score: z1 [yellow]Evaluation returned no results[/yellow]zEvaluation failed: u/ [yellow]⚠ Evaluation skipped due to error: zBackfill Summaryrbr rerfrrirjrkzWeeks processedzMeetings ingestedzRaces ingestedzStarters ingestedzRating snapshotsz Brier scorezWinner accuracyrprrz Failed weeksu [yellow]⚠ Completed with z error(s)[/yellow]u3 [green]✓ Backfill completed successfully[/green])*rtodayr!r ryrzr{r@rArrr r r|r}r~rrrrrDrr?rrrwrxospathr7dirnameabspath__file__ subprocess executablerrAloadsstdoutrrr) rrXrZrr[ skip_evalrcrJrrtotals failed_weekscurrent week_countweek_endrLrrrr2rfw_startfw_endfw_errrrrarb script_path eval_resultdatars r backfillrysd JJLE!%0J~H-5)8<<;X;X MM 123|4wg / MM 6(*VH--.?-@ D F 24LGJ  w!22E:a  zl"WIT(< P  "'*7B,3KK--4F.-)% z"h."w5(z"h."x GMM($;;  MMz'(+<> YA..=  B MM67 MM' |45 MM&vj'9&:;< MMOF7O#456 MM&vj'9&:;< s<'8&99VWX(4 $Hff MMJxjVHBvhiP Q)5N jQ ST /J? MM$c.&9:  mc{4 'Js+;= MM F8 h'!+e  nc#l*;& ?   CF ; < 8  !    ">  LL-aS1DL A MM9!FC D HHQKK T Z LL.qc2TL B MMKA3iX Y Y Zs& W>0A3W,#'W>.Z> Y5 !ZD[%6[%, W;6W>> Y2AY--Y25 Z?Z ["A[["% \*/0\%%\*__main__)7__doc__r|rArdrirwcalendarrrrrclick rich.consoler rich.tablerpackages.core.common.loggingrr packages.core.common.settingsr packages.core.common.utilsr packages.core.storage.databaser packages.core.storage.ingestionr packages.core.storage.modelsrrrrr__name__rr@intr!groupr%commandoptionrrVChoicerboolrrrrrrUIntRangeryr$r"r rs?  $ B616<6U H  )77c7d7$      "        /  0s0S000    &0f   "         -    oe < H @c@C@c@S@   4@F   "        2 ##s#4#   $#L   :    S     *     (  @3@4@C@#@    2@F   "       E F  8    B   S  tt t14t?BtOSt   Dtn//d$$N   "         +  |   &|~  2  0  oe < H  uh' >  H   2   >   3 vN    FvNr zEr"