o ٭jR@s\dZddlZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z mZmZmZmZddlZddlmZddlmZmZddlmZmZmZmZmZgd Zeee eeffZ eZ!Gd d d eZ"d e e#d e e e#eeffddZ$edZ%de%d e%fddZ&GdddZ'Gddde"Z(Gddde"Z)Gddde"Z*Gddde"Z+dS)z}Callback library containing training routines. See :doc:`Callback Functions ` for a quick introduction. N)ABC) AnyCallableDictListOptionalSequenceTuple TypeAliasTypeVarUnioncast) collective)EvalsLog _ScoreList)BoosterDMatrix XGBoostError_deprecate_positional_args_parse_eval_str)TrainingCallbackLearningRateScheduler EarlyStoppingEvaluationMonitorTrainingCheckPointCallbackContainerc@seZdZUdZeZeed<dddZdedefdd Z dedefd d Z ded e d ede fddZ ded e d ede fddZdS)rzCInterface for training callback. .. versionadded:: 1.3.0 rreturnNcCsdSN)selfrrM/var/www/html/CropPilot/venv/lib/python3.10/site-packages/xgboost/callback.py__init__<szTrainingCallback.__init__modelcC|S)zRun before training starts.rr r#rrr!before_training?z TrainingCallback.before_trainingcCr$)zRun after training is finished.rr%rrr!after_trainingCr'zTrainingCallback.after_trainingepoch evals_logcCdS)zRun before each iteration. Returns True when training should stop. See :py:meth:`after_iteration` for details. Frr r#r)r*rrr!before_iterationGsz!TrainingCallback.before_iterationcCr+)aRun after each iteration. Returns `True` when training should stop. Parameters ---------- model : Eeither a :py:class:`~xgboost.Booster` object or a CVPack if the cv function in xgboost is being used. epoch : The current training iteration. evals_log : A dictionary containing the evaluation history: .. code-block:: python {"data_name": {"metric_name": [0.5, ...]}} Frr,rrr!after_iterationNsz TrainingCallback.after_iteration)rN)__name__ __module__ __qualname____doc__rr __annotations__r"_Modelr&r(intboolr-r.rrrr!r3s   rrlistrcCsi}|dd}|D]B}|}||dksJt|ddD]+\}}t|ts/|}|d\}}||f|vrBg|||f<|||ft|q"q |} g} t|dddD](\\} } } t | }t| tsq| } t |t |}}| | ||fgq]| S)z#Aggregate cross-validation results.rrN:cSs |ddSNrr)xrrr!us z_aggcv..)key)split enumerate isinstancestrdecodeappendfloatsorteditemsnumpyarraymeanstdextend)r7cvmapidxlinearr metric_idxitkvmsgresults_namesas_arrrHrIrrr!_aggcvds,      rY_ARTscorecCsZt}|dks J|dkr|St|trtdt|g}t|tjj |}|dS)zHelper function for computing customized metric in distributed environment. Not strictly correct as many functions don't use mean value as final result. rrzBxgboost.cv function should not be used in distributed environment.) rget_world_sizer?tuple ValueErrorrFrG allreduceOpSUM)r[worldrNrrr!_allreduce_metrics   rcc @seZdZdZ   ddeedeededed df d d Z d e d e fd dZ d e d e fddZ d e de dedeeeeefd ef ddZdeeeeefeeeeeffde d dfddZd e de dedeeeeefd ef ddZdS)rzfA special internal callback for invoking a list of other callbacks. .. versionadded:: 1.3.0 NTF callbacksmetric output_marginis_cvrcCsxtt||_|D] }t|tstdq d}|dur$t|s$t|||_t |_ ||_ ||_ |j r:d|_dSdS)Nz3callback must be an instance of `TrainingCallback`.zmetric must be callable object for monitoring. For builtin metrics, passing them in training parameter invokes monitor automatically.)listdictfromkeysrdr?r TypeErrorcallablere collections OrderedDicthistory_output_marginrg aggregated_cv)r rdrerfrgcbrSrrr!r"s    zCallbackContainer.__init__r#cCN|jD]!}|j|d}d}|jrt|jtsJ|qt|ts$J|q|S)z Function called before training.r#z'before_training should return the model)rdr&rgr?cvfoldsrhrr r#crSrrr!r&s  z!CallbackContainer.before_trainingcCrs)zFunction called after training.rtz&after_training should return the model)rdr(rgr?rurhrrvrrr!r(s  z CallbackContainer.after_trainingr)dtrainevalscstfddjDS)z*Function called before training iteration.c3 |] }|jVqdSr)r-ro.0rwr)r#r rr! s z5CallbackContainer.before_iteration..)anyrd)r r#r)rxryrr}r!r-sz"CallbackContainer.before_iterationr[c Cs|D]~}|d}|d}|jr"ttttttf|d}||f}n|}|d}|d} d|dd} t|}| |jvrFt |j| <|j| } | | vrVtt g| | <| | } |jrstt tttf|  ttttf|qtt t|  tt|qdS)Nrr-)rgrCr r r@r=joinrcrormrnrrrB) r r[r)drVrWrIr: splited_names data_name metric_name data_historymetric_historyrrr!_update_historys.    z!CallbackContainer._update_historyc sjrjj}t|}|_|n.|dur gn|}|D]\}}|ddks3Jdq$|jj}t |} | t fddj D} | S)z)Function called after training iteration.Nrz#Dataset name should not contain `-`c3rzr)r.ror{r}rr!r~ sz4CallbackContainer.after_iteration..) rgevalrerprYrqrfindeval_setrrrd) r r#r)rxryscoresrUrVr[ metric_scoreretrr}r!r.s  z!CallbackContainer.after_iteration)NTF)r/r0r1r2rrrrr6r"r4r&r(r5rrr r@r-r rCrr.rrrr!rs^      $ rcsZeZdZdZdeeegefeefddffdd Z de ded e de fd d Z ZS) raCallback function for scheduling learning rate. .. versionadded:: 1.3.0 Parameters ---------- learning_rates : If it's a callable object, then it should accept an integer parameter `epoch` and returns the corresponding learning rate. Otherwise it should be a sequence like list or tuple with the same size of boosting rounds. learning_ratesrNcsTtsttjjstdttr|_nfdd|_t dS)Nz=Invalid learning rates, expecting callable or sequence, got: cstt|Sr)r r)r)rrr!r;.sz0LearningRateScheduler.__init__..) rlr?rmabcrrktypersuperr")r r __class__rr!r" s zLearningRateScheduler.__init__r#r)r*cCs|d||dS)N learning_rateF) set_paramrr,rrr!r.1sz%LearningRateScheduler.after_iteration)r/r0r1r2r rr5rCrr"r4rr6r. __classcell__rrrr!rs"rcseZdZdZedddddddedeedeed eed eed e d dffd dZ de d e fddZ de dedede ded ef ddZde deded efddZde d e fddZZS)raUCallback function for early stopping .. versionadded:: 1.3.0 Parameters ---------- rounds : Early stopping rounds. metric_name : Name of metric that is used for early stopping. data_name : Name of dataset that is used for early stopping. maximize : Whether to maximize evaluation metric. None means auto (discouraged). save_best : Whether training should return the best model or the last model. If set to `True`, it will only keep the boosting rounds up to the detected best iteration, discarding the ones that come after. This is only supported with tree methods (not `gblinear`). Also, the `cv` function doesn't return a model, the parameter is not applicable. min_delta : .. versionadded:: 1.5.0 Minimum absolute change in score to be qualified as an improvement. Examples -------- .. code-block:: python es = xgboost.callback.EarlyStopping( rounds=2, min_delta=1e-3, save_best=True, maximize=False, data_name="validation_0", metric_name="mlogloss", ) clf = xgboost.XGBClassifier(tree_method="hist", device="cuda", callbacks=[es]) X, y = load_digits(return_X_y=True) clf.fit(X, y, eval_set=[(X, y)]) NFg)rrmaximize save_best min_deltaroundsrrrrrrcs\||_||_||_||_||_i|_||_|jdkrtdd|_i|_ d|_ t dS)Nrz(min_delta must be greater or equal to 0.) datarrrrstopping_history _min_deltar^current_rounds best_scoresstarting_roundrr")r rrrrrrrrr!r"fs  zEarlyStopping.__init__r#cCs&||_t|ts|jrtd|S)NzP`save_best` is not applicable to the `cv` function as it doesn't return a model.)num_boosted_roundsrr?rrr^r%rrr!r&s zEarlyStopping.before_trainingr[rVrer)c sdtdtfdddtdtdtffdd }dtdtdtffd d }jdurBd }d kr?tfd d|Dr?d_nd_jrH|} n|} js{d_ij|<tt|gj|<ij |<|gj |<|j t |t |dnK| |j |dsj| |jd7_n-j| |j | |j|d} |j t | t |dd_jj krdSdS)NvaluercSst|tr |dS|S)z+get score if it's cross validation history.r)r?r])rrrr!get_ssz+EarlyStopping._update_rounds..get_snewbestcst|j|S)z-New score should be greater than the old one.rFgreaterrrrrr rr!rz.EarlyStopping._update_rounds..maximizecst|j|S)z,New score should be lesser than the old one.rrrrr!minimizerz.EarlyStopping._update_rounds..minimize) aucaucprprezpre@mapndcgzauc@zaucpr@zmap@zndcg@mapec3s|]}|VqdSr) startswith)r|r:)rerr!r~sz/EarlyStopping._update_rounds..TFr) best_scorebest_iterationrr)_ScorerCr6rrrrr rrset_attrr@rBr) r r[rVrer#r)rrmaximize_metrics improve_oprecordr)rrer r!_update_roundss:     zEarlyStopping._update_roundsr*c Cs||j7}d}t|dkrt||jr|j}nt|d}||vr-td|t|ts;tdt |||}|j rF|j }nt|d}||vrYtd|||d}|j |||||dS)Nz;Must have at least 1 validation dataset for early stopping.rrzNo dataset named: z1The name of the dataset should be a string. Got: zNo metric named: )r[rVrer#r)) rlenkeysr^rrhr?r@rkrrr) r r#r)r*rSrdata_logrr[rrr!r.s.     zEarlyStopping.after_iterationc Csp|js|Sz!|j}|j}|dur|dusJ|d|d}||_||_W|Sty7}ztd|d}~ww)Nrz4`save_best` is not applicable to the current booster)rrrr)r r#rrerrr!r(s$zEarlyStopping.after_training)r/r0r1r2rr5rr@r6rCr"r4r&rrrr.r(rrrrr!r7sN.   ?$rc seZdZdZdddejfdedededee gd fffd d Z d e d e de de e de f ddZ dedededefddZdedefddZZS)raPrint the evaluation result at each iteration. .. versionadded:: 1.3.0 Parameters ---------- rank : Which worker should be used for printing the result. period : How many epoches between printing. show_stdv : Used in cv to show standard deviation. Users should not specify it. logger : A callable used for logging evaluation result. rrFrankperiod show_stdvloggerNcs8||_||_||_||_|dksJd|_tdSr9) printer_rankrr_logger_latestrr")r rrrrrrr!r"s zEvaluationMonitor.__init__rrer[rIrcCsR|dur|jrd|d|d|dd|d}|Sd|d|d|d}|S)N rr8z.5f+)r)r rrer[rIrSrrr! _fmt_metric"s "zEvaluationMonitor._fmt_metricr#r)r*c Cs|sdSd|d}t|jkrd|D]1\}}|D](\}}d} t|dtr7|dd} |dd} n|d} ||||| | 7}qq|d7}||jdksW|jdkra||d|_ dS||_ dS)NF[]rrr ) rget_rankrrEr?r]rrrr) r r#r)r*rSrrerlogstdvr[rrr!r.+s(   z!EvaluationMonitor.after_iterationcCs(t|jkr|jdur||j|Sr)rrrrrr%rrr!r(Ds z EvaluationMonitor.after_training)r/r0r1r2rcommunicator_printr5r6rr@r"rCrrr4rr.r(rrrrr!rs8   rc sxeZdZdZdZ   ddeeejfdede d e d d f fd d Z de d e fddZ de de ded e fddZZS)rakCheckpointing operation. Users are encouraged to create their own callbacks for checkpoint as XGBoost doesn't handle distributed file systems. When checkpointing on distributed systems, be sure to know the rank of the worker to avoid multiple workers checkpointing to the same place. .. versionadded:: 1.3.0 Since XGBoost 2.1.0, the default format is changed to UBJSON. Parameters ---------- directory : Output model directory. name : pattern of output model file. Models will be saved as name_0.ubj, name_1.ubj, name_2.ubj .... as_pickle : When set to True, all training parameters will be saved in pickle format, instead of saving only the model. interval : Interval of checkpointing. Checkpointing is slow so setting a larger number can reduce performance hit. ubjr#Fd directoryrV as_pickleintervalrNcs8t||_||_||_||_d|_d|_t dSr9) osfspath_path_name _as_pickle _iterations_epoch_startrr")r rrVrrrrr!r"gs zTrainingCheckPoint.__init__cCs||_|Sr)rrr%rrr!r&vs z"TrainingCheckPoint.before_trainingr)r*cCs|j|jkrPtj|j|jdt||j|j rdnd|j }d|_t dkrP|j rKt |d}t||Wdn1sEwYn|||jd7_dS)NrUz.pkl.rwbrF)rrrpathrrrr@rrdefault_formatrropenpickledump save_model)r r#r)r*rfdrrr!r.zs*     z"TrainingCheckPoint.after_iteration)r#Fr)r/r0r1r2rr r@rPathLiker6r5r"r4r&rr.rrrrr!rJs& "r),r2rmrrrrtypingrrrrrrr r r r r rFr_typingrrcorerrrrr__all__rCrr4rr@rYrZrcrrrrrrrrr!s. 4  $1{'IK