o ۭjƉ@sdZddlmZddlmZddlmZmZmZm Z m Z ddl Z ddl mZmZmZddlmZmZmZmZmZddlmZdd lmZdd lmZmZmZmZm Z m!Z!dd l"m#Z#dd l$m%Z%m&Z&m'Z'erldd l(m)Z)dddZ*dddZ+e dddd d!Z,e dd%d!Z,d&ddd(d!Z,gd)Z-gd*Z.dd.d/Z/dd3d4Z0dd7d8Z1ddddwdxZ?e?   dddzd{Z@e?   ddd|d}ZAe?   ddd~dZBe?   ddddZCdddZDdddZEe@eAdZFddddZGdddZHdddZIdddZJdS)z$ Routines for filling missing data. ) annotations)wraps) TYPE_CHECKINGAnyLiteralcastoverloadN)NaTalgoslib) ArrayLikeAxisIntF ReindexMethodnpt)import_optional_dependency)infer_dtype_from) is_array_like is_bool_dtypeis_numeric_dtypeis_numeric_v_string_likeis_object_dtypeneeds_i8_conversion)DatetimeTZDtype)is_valid_na_for_dtypeisnana_value_for_dtypeIndexmasknpt.NDArray[np.bool_]lengthintcCs8t|rt||krtdt|d|||}|S)zJ Validate the size of the values passed to ExtensionArray.fillna. z'Length of 'value' does not match. Got (z ) expected )rlen ValueError)valuerr!r&P/var/www/html/CropPilot/venv/lib/python3.10/site-packages/pandas/core/missing.pycheck_value_size3s  r(arrr returnc CsNt|\}}t|tjrtj||d}n|}t|s |g}|j||dd}d}t |jr6d}t |}t |}||}tj |j t d}t|jrWt|jsWt|jrWnDt|jrgt|jrgt|jsgn4|D]1} t|| rqqi|rtj |j tjd} ||| k| |<n|| k} t| tjs| jt dd} || O}qi|r|t |O}|S)a  Return a masking array of same size/shape as arr with entries equaling any member of values_to_mask set to True Parameters ---------- arr : ArrayLike values_to_mask: list, tuple, or scalar Returns ------- np.ndarray[bool] )dtypeF)r+copyT)r+na_value)r isinstancenpr+arrayconstruct_array_typer is_list_like_from_sequencerrzerosshapeboolrrrbool_ndarrayto_numpyany) r)values_to_maskr+cls potential_naarr_maskna_masknonnarxnew_maskr&r&r' mask_missingBsR          rC. allow_nearestmethod,Literal['ffill', 'pad', 'bfill', 'backfill']rELiteral[False]Literal['pad', 'backfill']cCdSNr&rFrEr&r&r'clean_fill_methodrM7Literal['ffill', 'pad', 'bfill', 'backfill', 'nearest'] Literal[True]%Literal['pad', 'backfill', 'nearest']cCrJrKr&rLr&r&r'rMrNFr6cCsjt|tr|}|dkrd}n|dkrd}ddg}d}|r%|dd}||vr3td|d ||S) Nffillpadbfillbackfillzpad (ffill) or backfill (bfill)nearestz(pad (ffill), backfill (bfill) or nearestzInvalid fill method. Expecting z. Got )r.strlowerappendr$)rFrE valid_methods expectingr&r&r'rMs  )lineartimeindexvalues)rVzeroslinear quadraticcubic barycentrickroghspline polynomialfrom_derivativespiecewise_polynomialpchipakima cubicsplinerWr^rcKsh|d}|dvr|durtdtt}||vr$td|d|d|dvr2|js2t|d|S) Norder)rfrgz7You must specify the order of the spline or polynomial.zmethod must be one of z. Got 'z ' instead.)rerirjz4 interpolation requires that the index be monotonic.)getr$ NP_METHODS SP_METHODSis_monotonic_increasing)rFr^kwargsrmvalidr&r&r'clean_interp_methods rthowis_valid int | NonecCs|dvsJt|dkrdS|jdkr|jdd}|dkr&|dd}n|dkr9t|d|ddd }||}|sAdS|S) a+ Retrieves the positional index of the first valid value. Parameters ---------- how : {'first', 'last'} Use this parameter to change between the first or last valid index. is_valid: np.ndarray Mask to find na_values. Returns ------- int or None )firstlastrNaxisrxry)r#ndimr:argmax)rurvidxpos chk_notnar&r&r'find_valid_indexs    rlimit_direction&Literal['forward', 'backward', 'both']cCs2gd}|}||vrtd|d|d|S)N)forwardbackwardbothz*Invalid limit_direction: expecting one of z, got 'z'.rXr$)rvalid_limit_directionsr&r&r'validate_limit_directionsr limit_area str | None#Literal['inside', 'outside'] | NonecCs:|durddg}|}||vrtd|d|d|S)Ninsideoutsidez%Invalid limit_area: expecting one of z, got .r)rvalid_limit_areasr&r&r'validate_limit_areasr-Literal['backward', 'forward', 'both'] | None&Literal['backward', 'forward', 'both']cCsd|dur|dvr d}|Sd}|S|dvr |dkr td|d|dvr0|dkr0td|d|S)N)rUrTrr)rSrRz0`limit_direction` must be 'forward' for method ``z1`limit_direction` must be 'backward' for method `)r$)rrFr&r&r'infer_limit_direction#s    rcCs|dkrddlm}|tt|}n$hd}t|jp)t|jtp)t |jd}||vr8|s8t d|dt | rBtd|S) Nr\rr>r]r^r_rVmMz9Index column must be numeric or datetime type when using z_ method other than linear. Try setting a numeric or datetime index column before interpolating.zkInterpolation with NaNs in the index has not been implemented. Try filling those NaNs before interpolating.)pandasrr/aranger#rr+r.rr is_np_dtyper$rr:NotImplementedError)rFr^rmethodsis_numeric_or_datetimer&r&r'get_interp_index8s(      rr\rdata np.ndarrayr}r limit fill_value Any | NoneNonec st|fit|jrt|jdddkr%t|js#tddtt|tj ddt |dfd d } t | ||dS)z Column-wise application of _interpolate_1d. Notes ----- Alters 'data' in-place. The signature does differ from _interpolate_1d because it only includes what is needed for Block.interpolate. F)compatr]zStime-weighted interpolation only works on Series or DataFrames with a DatetimeIndexr_N)nobsryvaluesrr*rc s&td|dd dS)NF) indicesrrFrrrr bounds_errorrr&)_interpolate_1d)rrrrrrlimit_area_validatedrrrFr&r'funcs z$interpolate_2d_inplace..func)rrr*r) rtrr+rrr$rrr validate_limit_index_to_interp_indicesr/apply_along_axis) rr^r}rFrrrrrrrrr&rr'interpolate_2d_inplaceWs   rcCsb|j}t|jr |d}|dkr|}ttj|}|St|}|dvr/|jtjkr/t |}|S)zE Convert Index to ndarray of indices to pass to NumPy/SciPy. i8r\)r_r^) _valuesrr+viewrr/r8asarrayobject_r maybe_convert_objects)r^rFxarrindsr&r&r'rs      rrrrrmc Ks| dur| } nt|} | } | sdS| rdStt| } td| d}|dur-d}tt|}td| d}|durAt|}ttd|t| }|dkr[|tt | |dB}n|dkrj|tt | d|B}ntt | ||}|d kr}|||BO}n|d kr| ||}||O}t |}|j j d v}|r| d }|tvrt|| }t|| || ||| ||| <nt|| || || f||||d | || <| durd| dd<d| |<dS|rtj||<dStj||<dS)a Logic for the 1-d interpolation. The input indices and yvalues will each be 1-d arrays of the same length. Bounds_error is currently hardcoded to False since non-scipy ones don't take it as an argument. Notes ----- Fills 'yvalues' in-place. Nrxrurvrryr{rrrrrr)rFrrrmFT)rr:allsetr/ flatnonzerorranger# _interp_limitsortedr+kindrroargsortinterp_interpolate_scipy_wrapperr r%nan)rrrFrrrrrrmrrrinvalidrsall_nansfirst_valid_index start_nanslast_valid_indexend_nans preserve_nansmid_nansis_datetimelikeindexerr&r&r'rsr            rrAynew_xcKs"|d}td|dddlm} t|}| j| jtttt | j d} gd} || vrD|dkr2|} n|} | j ||| ||d } | |}|S|d krit |sP|dkrWt d || j||fd |i|} | |}|S|jjsq|}|jjsy|}|jjs|}| |} | |||fi|}|S) z Passed off to scipy.interpolate.interp1d. method is scipy's kind. Returns an array interpolated at new_x. Add any new methods to the list in _clean_interp_method. z interpolation requires SciPy.scipy)extrar interpolate)rdrerhrirlrkrj)rVr`rarbrcrgrg)rrrrfz;order needs to be specified and greater than 0; got order: k)rrrr/rbarycentric_interpolatekrogh_interpolate_from_derivatives_cubicspline_interpolate_akima_interpolatepchip_interpolateinterp1drr$UnivariateSplineflags writeabler,)rArrrFrrrmrrrr alt_methodsinterp1d_methodsrterpnew_yr&r&r'r%sN       rxiyiderint | list[int] | None extrapolatec Cs4ddlm}|jj}|||dd||d}||S)a Convenience function for interpolate.BPoly.from_derivatives. Construct a piecewise polynomial in the Bernstein basis, compatible with the specified values and derivatives at breakpoints. Parameters ---------- xi : array-like sorted 1D array of x-coordinates yi : array-like or list of array-likes yi[i][j] is the j-th derivative known at xi[i] order: None or int or array-like of ints. Default: None. Specifies the degree of local polynomials. If not None, some derivatives are ignored. der : int or list How many derivatives to extract; None for all potentially nonzero derivatives (that is a number equal to the number of points), or a list of derivatives to extract. This number includes the function value as 0th derivative. extrapolate : bool, optional Whether to extrapolate to ouf-of-bounds points based on first and last intervals, or to return NaNs. Default: True. See Also -------- scipy.interpolate.BPoly.from_derivatives Returns ------- y : scalar or array-like The result, of length R or length M or M by R. rrr~r{)ordersr)rrBPolyrhreshape) rrrArmrrrrFmr&r&r'rls )rcCs(ddlm}|j|||d}|||dS)aQ Convenience function for akima interpolation. xi and yi are arrays of values used to approximate some function f, with ``yi = f(xi)``. See `Akima1DInterpolator` for details. Parameters ---------- xi : np.ndarray A sorted list of x-coordinates, of length N. yi : np.ndarray A 1-D array of real values. `yi`'s length along the interpolation axis must be equal to the length of `xi`. If N-D array, use axis parameter to select correct axis. x : np.ndarray Of length M. der : int, optional How many derivatives to extract; None for all potentially nonzero derivatives (that is a number equal to the number of points), or a list of derivatives to extract. This number includes the function value as 0th derivative. axis : int, optional Axis in the yi array corresponding to the x-coordinate values. See Also -------- scipy.interpolate.Akima1DInterpolator Returns ------- y : scalar or array-like The result, of length R or length M or M by R, rrr|)nu)rrAkima1DInterpolator)rrrArr}rPr&r&r'rs * r not-a-knotbc_typestr | tuple[Any, Any]cCs(ddlm}|j|||||d}||S)ag Convenience function for cubic spline data interpolator. See `scipy.interpolate.CubicSpline` for details. Parameters ---------- xi : np.ndarray, shape (n,) 1-d array containing values of the independent variable. Values must be real, finite and in strictly increasing order. yi : np.ndarray Array containing values of the dependent variable. It can have arbitrary number of dimensions, but the length along ``axis`` (see below) must match the length of ``x``. Values must be finite. x : np.ndarray, shape (m,) axis : int, optional Axis along which `y` is assumed to be varying. Meaning that for ``x[i]`` the corresponding values are ``np.take(y, i, axis=axis)``. Default is 0. bc_type : string or 2-tuple, optional Boundary condition type. Two additional equations, given by the boundary conditions, are required to determine all coefficients of polynomials on each segment [2]_. If `bc_type` is a string, then the specified condition will be applied at both ends of a spline. Available conditions are: * 'not-a-knot' (default): The first and second segment at a curve end are the same polynomial. It is a good default when there is no information on boundary conditions. * 'periodic': The interpolated functions is assumed to be periodic of period ``x[-1] - x[0]``. The first and last value of `y` must be identical: ``y[0] == y[-1]``. This boundary condition will result in ``y'[0] == y'[-1]`` and ``y''[0] == y''[-1]``. * 'clamped': The first derivative at curves ends are zero. Assuming a 1D `y`, ``bc_type=((1, 0.0), (1, 0.0))`` is the same condition. * 'natural': The second derivative at curve ends are zero. Assuming a 1D `y`, ``bc_type=((2, 0.0), (2, 0.0))`` is the same condition. If `bc_type` is a 2-tuple, the first and the second value will be applied at the curve start and end respectively. The tuple values can be one of the previously mentioned strings (except 'periodic') or a tuple `(order, deriv_values)` allowing to specify arbitrary derivatives at curve ends: * `order`: the derivative order, 1 or 2. * `deriv_value`: array-like containing derivative values, shape must be the same as `y`, excluding ``axis`` dimension. For example, if `y` is 1D, then `deriv_value` must be a scalar. If `y` is 3D with the shape (n0, n1, n2) and axis=2, then `deriv_value` must be 2D and have the shape (n0, n1). extrapolate : {bool, 'periodic', None}, optional If bool, determines whether to extrapolate to out-of-bounds points based on first and last intervals, or to return NaNs. If 'periodic', periodic extrapolation is used. If None (default), ``extrapolate`` is set to 'periodic' for ``bc_type='periodic'`` and to True otherwise. See Also -------- scipy.interpolate.CubicHermiteSpline Returns ------- y : scalar or array-like The result, of shape (m,) References ---------- .. [1] `Cubic Spline Interpolation `_ on Wikiversity. .. [2] Carl de Boor, "A Practical Guide to Splines", Springer-Verlag, 1978. rr)r}rr)rr CubicSpline)rrrAr}rrrrr&r&r'rs M rr_Literal['inside', 'outside']cCst|}|}|sXtd|d}|durd}td|d}|dur%t|}t||||d|dkr:d|||d <n|d krMd|d|<||d d<ntd tj||<dSdS) a Apply interpolation and limit_area logic to values along a to-be-specified axis. Parameters ---------- values: np.ndarray Input array. method: str Interpolation method. Could be "bfill" or "pad" limit: int, optional Index limit on interpolation. limit_area: {'inside', 'outside'} Limit area for interpolation. Notes ----- Modifies values in-place. rxrNrry)rFrrrFr{rz*limit_area should be 'inside' or 'outside')rrrr#pad_or_backfill_inplacer$r/r)r_rFrrrrvrxryr&r&r'_interpolate_with_limit_area%s,  rrScCst|dkrddndd}|jdkr#|dkrtd|td|j}t|}||}t|dd }||||d d S) a Perform an actual interpolation of values, values will be make 2-d if needed fills inplace, returns the result. Parameters ---------- values: np.ndarray Input array. method: str, default "pad" Interpolation method. Could be "bfill" or "pad" axis: 0 or 1 Interpolation axis limit: int, optional Index limit on interpolation. limit_area: str, optional Limit area for interpolation. Can be "inside" or "outside" Notes ----- Modifies values in-place. rcSs|SrKr&rAr&r&r'vsz)pad_or_backfill_inplace..cSs|jSrK)Trr&r&r'rvsr{z0cannot interpolate on a ndim == 1 with axis != 0r{rz)r)rrN)rAssertionErrorrtupler5rM get_fill_func)r_rFr}rrtransftvaluesrr&r&r'rZs  rnpt.NDArray[np.bool_] | NonecCs|durt|}|SrK)r)r_rr&r&r' _fillna_prepsrrrcs(t   dd fdd }tt|S) z> Wrapper to handle datetime64 and timedelta64 dtypes. NrrwrrcsTt|jr"|dur t|}|d|||d\}}||j|fS||||dS)Nr)rrr)rr+rr)r_rrrresultrr&r'new_funcs  z&_datetimelike_compat..new_funcNNN)rrwrr)rrr)rrr&rr'_datetimelike_compats  r(tuple[np.ndarray, npt.NDArray[np.bool_]]cC<t||}|dur|st||tj|||d||fSN)r)rr_fill_limit_area_1dr pad_inplacer_rrrr&r&r'_pad_1d  rcCr r )rrr r backfill_inplacerr&r&r' _backfill_1drrcCDt||}|durt|||jrtj|||d||fS ||fSr )r_fill_limit_area_2dsizer pad_2d_inplacerr&r&r'_pad_2d  rcCrr )rrrr backfill_2d_inplacerr&r&r' _backfill_2drrLiteral['outside', 'inside']cCst|}|}t||dddd}|dkr*d|d|<d||dd<dS|dkr8d||d|<dSdS)aPrepare 1d mask for ffill/bfill with limit_area. Caller is responsible for checking at least one value of mask is False. When called, mask will no longer faithfully represent when the corresponding are NA or not. Parameters ---------- mask : np.ndarray[bool, ndim=1] Mask representing NA values when filling. limit_area : { "outside", "inside" } Whether to limit filling to outside or inside the outer most non-NA value. Nr~r{rFr)rr#)rrneg_maskrxryr&r&r'r s r cCs|j}|dkr#tjj|ddtjj|dddddddd@}ntjj|ddtjj|ddddddddB}d||j<dS)aPrepare 2d mask for ffill/bfill with limit_area. When called, mask will no longer faithfully represent when the corresponding are NA or not. Parameters ---------- mask : np.ndarray[bool, ndim=1] Mask representing NA values when filling. limit_area : { "outside", "inside" } Whether to limit filling to outside or inside the outer most non-NA value. rrr|Nr~F)rr/maximum accumulate)rrrla_maskr&r&r'rs"$rrSrUr{rcCs&t|}|dkr t|Sttd|S)Nr{r )rM _fill_methodsrr)rFrr&r&r'r*srReindexMethod | NonecCs|durdSt|ddS)NTrD)rM)rFr&r&r'clean_reindex_fill_method1s r#rfw_limitbw_limitcst|t}t}d fdd }|dur(|dkr#tt|d}n|||}|durO|dkr2|St||ddd|}tdt|}|dkrO|S||@S) ak Get indexers of values that won't be filled because they exceed the limits. Parameters ---------- invalid : np.ndarray[bool] fw_limit : int or None forward limit to index bw_limit : int or None backward limit to index Returns ------- set of indexers Notes ----- This is equivalent to the more readable, but slower .. code-block:: python def _interp_limit(invalid, fw_limit, bw_limit): for x in np.where(invalid)[0]: if invalid[max(0, x - fw_limit):x + bw_limit + 1].all(): yield x rr"cs`t|}t||dd}tt|d|tt|d|ddkdB}|S)Nr{r)min_rolling_windowrrr/wherecumsum)rrwindowedidxNr&r'inner\s "z_interp_limit..innerNrr~r{)rr")r#rr/r(listr)rr$r%f_idxb_idxr. b_idx_invr&r,r'r7s ! rawindowcCsJ|jdd|jd|d|f}|j|jdf}tjjj|||dS)z [True, True, False, True, False], 2 -> [ [True, True], [True, False], [False, True], [True, False], ] Nr~r{)r5strides)r5r5r/r stride_tricks as_strided)r3r4r5r5r&r&r'r'xs$ r')rr r!r")r)r r*r )rFrGrErHr*rI)rFrOrErPr*rQ)rFrOrEr6r*rQ)rFrWr^rr*rW)rurWrvr r*rw)rrWr*r)rrr*r)rrrFrWr*r)r^rr*r)r\NrNNN)rrr^rr}r rFrWrrwrrWrrrrr*r)r^rrFrWr*r)r\NrNNFNN)rrrrrFrWrrwrrWrrrrrr6rmrwr*r)NFN) rArrrrrrFrWrr6)NrF) rrrrrArrrrr6)rr) rrrrrArrrr}r )rrN) rrrrrArr}r rr) r_rrFrIrrwrrr*r)rSrNN) r_rrFrIr}r rrwrrr*rrK)rrr*r )rrr*rr) r_rrrwrrrrr*r )r_rrrwrrrr)rrwrrrr)rr rrr*rr)rr")r*r")rr r$rwr%rw)r3r r4r"r*r )K__doc__ __future__r functoolsrtypingrrrrrnumpyr/ pandas._libsr r r pandas._typingr r rrrpandas.compat._optionalrpandas.core.dtypes.castrpandas.core.dtypes.commonrrrrrrpandas.core.dtypes.dtypesrpandas.core.dtypes.missingrrrrrr(rCrMrorprtrrrrrrrrrrrrrrrrrrrrr rr!rr#rr'r&r&r&r's         I     &   # F w K 6 5 V7 -         A