tj[ d dl Z d dlmZ d dlZd dlmZ d dlmc mZ d dl m
c m c mZ d dl
mZ ddlmZ ddlmZ ddlmZ ddlmZ dd lmZ dd
lmZ dd l m Z m Z dd l m Z m!Z!m"Z" dd
l#m$Z$ ddl%m&Z& ddl'm(Z(m)Z) ddl*m+Z+m,Z,m-Z-m.Z.m/Z/ ddl0m1Z1m2Z2 ddl3m4Z4 ddl5m6Z6 ddl7m8Z8 ddl9m:Z:m;Z; e/j< e= Z> e,d e G d de8 Z? G d d e: Z@ e,d e G d d e ZA G d d ejB ZC G d! d"ejB ZD G d# d$e& ZE G d% d&e; ZF G d' d(eE ZG G d) d*eE ZHe,e G d+ d,e$ ZI e,d-. G d/ d0eE ZJe, e4d12 G d3 d4e ZKg d5ZLdS )6 N) dataclass)strict )initialization)ACT2CLS)filter_output_hidden_states)PreTrainedConfig)TorchvisionBackend) BatchFeature)group_images_by_shapereorder_images)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDSizeDict)BaseModelOutput)PreTrainedModel) ImagesKwargsUnpack)TransformersKwargsauto_docstringcan_return_tupleis_torchdynamo_compilinglogging)
TensorTypemerge_with_config_defaults)requires)capture_outputs )GotOcr2VisionConfig)GotOcr2VisionAttentionGotOcr2VisionEncoderz&PaddlePaddle/SLANeXt_wired_safetensors)
checkpointc e Zd ZU dZeed< dS )SLANeXtVisionConfig
image_sizeN)__name__
__module__ __qualname__r& int__annotations__ i/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/slanext/modular_slanext.pyr$ r$ 2 s$ Jr- r$ c e Zd ZdS )SLANeXtVisionAttentionNr' r( r) r, r- r. r0 r0 8 Dr- r0 c e Zd ZU dZdZdeiZdZeez dz e d< dZ
e e d< dZ e e d< d Z
e e d
< dZe e d < d Ze e d
< fdZ xZS )
SLANeXtConfiga
vision_config (`dict` or [`SLANeXtVisionConfig`], *optional*):
Configuration for the vision encoder. If `None`, a default [`SLANeXtVisionConfig`] is used.
post_conv_in_channels (`int`, *optional*, defaults to 256):
Number of input channels for the post-encoder convolution layer.
post_conv_out_channels (`int`, *optional*, defaults to 512):
Number of output channels for the post-encoder convolution layer.
out_channels (`int`, *optional*, defaults to 50):
Vocabulary size for the table structure token prediction head, i.e., the number of distinct structure
tokens the model can predict.
hidden_size (`int`, *optional*, defaults to 512):
Dimensionality of the hidden states in the attention GRU cell and the structure/location prediction heads.
max_text_length (`int`, *optional*, defaults to 500):
Maximum number of autoregressive decoding steps (tokens) for the structure and location decoder.
slanext
vision_configN post_conv_in_channelsr% post_conv_out_channels2 out_channels hidden_sizei max_text_lengthc | j t | _ n0t | j t rt di | j | _ t j di | d S Nr, )r6 r$
isinstancedictsuper
__post_init__selfkwargs __class__s r. rC zSLANeXtConfig.__post_init__Y so %!4!6!6D
*D
1
1 K!4!J!Jt7I!J!JD '''''''r- )r' r( r) __doc__
model_typer$ sub_configsr6 rA r+ r8 r* r9 r; r< r= rC
__classcell__rG s @r. r4 r4 < s J"$78K7;M4--4;;;!$3$$$"%C%%%L#K OS ( ( ( ( ( ( ( ( (r- r4 c ` e Zd Z fdZdej dej dej dee fdZ xZ S )SLANeXtAttentionGRUCellc & t t j ||d | _ t j || | _ t j |dd | _ t j ||z | | _ d S )NF)bias ) rB __init__nnLinearinput_to_hiddenhidden_to_hiddenscoreGRUCellrnn)rE
input_sizer< num_embeddingsrG s r. rR z SLANeXtAttentionGRUCell.__init__b sz
!y[u M M M " +{ C C Y{AE:::
:j> 9;GGr- prev_hidden batch_hidden char_onehotsrF c ( | | }| | d }||z }t j | }| | }t
j |dt j |j
}| dd }t j ||
d } t j | |gd }
| |
| } | |fS )NrQ dimdtyper )rU rV unsqueezetorchtanhrW Fsoftmaxfloat32torb transposematmulsqueezecatrY ) rE r\ r] r^ rF batch_hidden_projprev_hidden_projattention_scores attn_weightscontextconcat_context
hidden_statess r. forwardz SLANeXtAttentionGRUCell.forwardk s ! 0 0 > > 00==GGJJ,/?? :&677 ::&677y!1q
NNNQQRbRhii #--a33 ,|\::BB1EE G\#:A>> ==
l**r- )
r' r( r) rR rd FloatTensorr r ru rK rL s @r. rN rN a s H H H H H+&+ '+ ' +
+, + + + + + + + +r- rN c & e Zd Zd fd Zd Z xZS )
SLANeXtMLPNc t t j || | _ t j || | _ |t j nt | | _ d S N) rB rR rS rT fc1fc2Identityr act_fn)rE r< r;
activationrG s r. rR zSLANeXtMLP.__init__ sd
9[+669[,77'1'9bkmmmwz?R?T?T r- c | | }| | }| | }|S rz )r{ r| r~ )rE rt s r. ru zSLANeXtMLP.forward s; //
//
M22
r- rz )r' r( r) rR ru rK rL s @r. rx rx sR U U U U U U r- rx c l e Zd ZU eed< dZdZdZdZddgZ e
j fd Z xZ
S ) SLANeXtPreTrainedModelconfigbackbone pixel_values)imageTstructure_attention_cellstructure_generatorc D t | t |t r!|j t j |j d t |t r;|j r4t j |j d t j |j
d t |t j r|j
dk r dt j |j
z nd}t j |j | | t j |j | | |j t j |j | | |j t j |j | | t |t* rdt j | j j
dz z }|j fD ]t}| D ][}t |t j r?t j |j | | |j t j |j | | \sdS dS )zInitialize the weightsNg r g ?) rB
_init_weightsr@ SLANeXtVisionEncoder pos_embedinit constant_r0 use_rel_pos rel_pos_h rel_pos_wrS rX r< mathsqrtuniform_ weight_ih weight_hhbias_ihbias_hhSLANeXtSLAHeadr r childrenrT weightrP )rE modulestd generatorlayerrG s r. r z$SLANeXtPreTrainedModel._init_weights s f%%% f 2 3 3 6 +v /555 f 4 5 5 6!
6v /555v /555 fbj ) ) 99?9Ka9O9O# &"45555UVC M&*SD# 6 6 6 M&*SD# 6 6 6~)
fnsdC888~)
fnsdC888 fn - - A $+"9C"?@@@C $ 8 :
A
A & / / 1 1 A AE!%33 A
elSD#>>> :1 M%*sdC @ @ @ A A A
A
Ar- )r' r( r) r4 r+ base_model_prefixmain_input_nameinput_modalities supports_gradient_checkpointing _keep_in_fp32_modules_strictrd no_gradr rK rL s @r. r r s "$O!&*#$>@U#V U]__"A "A "A "A _"A "A "A "A "Ar- r c e Zd ZdS )r Nr1 r, r- r. r r r2 r- r c T e Zd Z ddedz f fd
Zdej dee fdZ xZ
S )SLANeXtBackboneNr c t | t |j | _ t j |j |j dddd | _ |
d S )Nr r rQ F) kernel_sizestridepaddingrP ) rB rR r r6 vision_towerrS Conv2dr8 r9 post_conv post_initrE r rF rG s r. rR zSLANeXtBackbone.__init__ sw
01E F F (&*GUV_`jkrw
r- rt rF c | j |fi |}| |j }| d dd }t ||j |j S )Nr rQ )last_hidden_statert
attentions)r r r flattenrj r rt r )rE rt rF
vision_outputs r. ru zSLANeXtBackbone.forward sy ))-BB6BB
}'FGG
%--a00::1a@@
+'5$/
r- rz ) r' r( r) rA rR rd Tensorr r ru rK rL s @r. r r s} #
t
U\
VDV=W
r- r c e Zd ZdeiZ d dedz f fd
Zeee d de
j de
j dz de
e fd Z xZS )
r r Nr c t | t |j |j |j | _ t |j |j | _ | d S rz )
rB rR rN r9 r< r; r rx r r r s r. rR zSLANeXtSLAHead.__init__ so
(? )6+=v?R)
)
% $.f.@&BU#V#V r- rt targetsrF c t j |j d | j j ft j |j }t j |j d gt j |j }g }g }t | j j dz D ]}t j || j j
} | ||
| \ }}| | }
|
d }| |
| | t j |d | j j dz
d r nt j t j |d dt j |j } t3 | | S ) Nr rb device)sizerb r rQ ra r` )r rt )rd zerosshaper r< rh r longranger= rf one_hotr; floatr r argmaxappendstackeqanyallrg ri rb r ) rE rt r rF featurespredicted_charsstructure_preds_liststructure_ids_list_embedding_featurestructure_stepstructure_predss r. ru zSLANeXtSLAHead.forward s ;
#T[%< =U][h[o
+M,?,B+C5:^k^rsss ! t{2Q677 A ! /4;;S T T Z Z \ \ 77-BUBUBWBWYjkkKHa ! 5 5h ? ?N , 3 3 3 : :O ' ' 7 7 7 % %o 6 6 6{-1555889QTU9UVVZZ[]^^bbdd
)EK0D!$L$L$LRT\a\ijjjmm
Pdeeeer- rz )r' r( r) rN _can_record_outputsrA rR r r r rd rv r r r ru rK rL s @r. r r s - # t (,f f (f $f +, f f f ! _ f f f f fr- r c P e Zd ZU dZdZej dz ed< dZej dz ed< dS ) SLANeXtForTableRecognitionOutputam
head_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):
Hidden-states of the SLANeXtSLAHead at each prediction step, varies up to max `self.config.max_text_length` states (depending on early exits).
head_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):
Attentions of the SLANeXtSLAHead at each prediction step, varies up to max `self.config.max_text_length` attentions (depending on early exits).
Nhead_hidden_stateshead_attentions) r' r( r) rH r rd rv r+ r r, r- r. r r sO 48)D077704OU&-44444r- r z
SLANeXt Table Recognition model for table recognition tasks. Wraps the core SLANeXtPreTrainedModel
and returns outputs compatible with the Transformers table recognition API.
) custom_introc e Zd Zdef fd Zeedej de e
de ej e z fd Z
xZS )SLANeXtForTableRecognitionr c t | t | | _ t | | _ | d S )N)r )rB rR r r r headr )rE r rG s r. rR z#SLANeXtForTableRecognition.__init__# sU
'v666
"&111 r- r rF returnc | j |fi |} | j |j fi |}t |j |j |j |j |j S )N)r rt r r r )r r r r rt r )rE r rF backbone_outputs head_outputss r. ru z"SLANeXtForTableRecognition.forward) sp
)4= @@@@ ty!1!CNNvNN / * <*8'2 + 9 ( 3
r- )r' r( r) r4 rR r r rd rv r r tupler ru rK rL s @r. r r s }
! -
9?@R9S
u !$D D
^
r- r )rd )backendsc & e Zd ZdZeZeZdddZdddZ dZ
dZ dZ dZ
dZdddeddfd Zd
ed d eded d
dedededededeee z dz deee z dz dedz dedz dedz deez dz def dZdee f fd Zd Zd Z xZ S ) SLANeXtImageProcessorr r% )heightwidthTr z torch.Tensorr r c 8 |j \ }}}}| ||z || }|j }t |j |j t || z }t
||z } t
||z }
t j |
t j | } | dz t | t |
z z dz
} | t j
}
| |
z
}t j |
dk t j | | }t j |
dk t j |
|
}
t j |
|dz
k t j | | }t j |
|dz
k t j |
|dz
|
}
|dz dz t j
}d|z
}t j | t j | }|dz t | t | z z dz
}| t j
}||
z
}t j |dk t j | | }t j |dk t j | | }t j ||dz
k t j | | }t j ||dz
k t j ||dz
| }|dz dz t j
}d|z
}| dd t j }| t j
}|
}|
dz }| }|dz } |d d |d d d f |d d d f f } |d d |d d d f |d d d f f } |d d | d d d f |d d d f f } |d d | d d d f |d d d f f } | d| d }!| d| d }"| dd|
}#| dd|
}$|"|$| z |#| z z z |!|$| z |#| z z z z }%|%dz d z }%|% dd t j }&|& ||| |
|j
S ) Nr g ?r rQ r i i )rb )r viewr maxr r roundrd arangerh r floorri int32where
zeros_like ones_like full_likeclampuint8r rb )'rE r r
batch_sizechannelsr r r scale
target_height target_width
target_colsrc_col
src_col_floor src_col_frac weight_right weight_left
target_rowsrc_row
src_row_floor src_row_frac
weight_bottom
weight_top image_uint8 image_int32col_left col_rightrow_top
row_bottompixel_top_leftpixel_top_rightpixel_bottom_leftpixel_bottom_rightweight_bottom_3d
weight_top_3dweight_right_3dweight_left_3dinterpresults' r. _resizez SLANeXtImageProcessor._resizeG sL
/4k+
Hfe
:0&%@@ DK,,s65/A/AA fun--
UU]++ \,emFSSS
#e u\7J7J(JKcQ
**5;77
!4!4!6!66 {=1#4e6F|6T6TVbcc MA$5u7G
7V7VXeff
{=EAI#=u|?\?\^jkk UQY &
uqy(Q(QS`
%t+c188::==ekJJ \) \-u}VTTT
#f
m8L8L(LMPSS
**5;77
!4!4!6!66 {=1#4e6F|6T6TVbcc MA$5u7G
7V7VXeff
{=FQJ#>P\@]@]_kll VaZ ' QR
)S)SUb
&,s299;;>>u{KK
M)
kk!S)),,U[99 !nnU[11 %%''"Q&,,.. $$&&#a'--//
$QQQ4(8(47:K%KL%aaaD)99T111W;M&MN ':aaag+>qqq@Q(Q R (Jqqq$w,?4QRQRQR7AS)S T(--a BB"=!<<
&++Aq,??$))!Q == ^ +o.O O
1B B_WiEi i jk G$+ a%%((55{{:x MMPPW\WbPcccr- images do_resizeresamplez"tvF.InterpolationMode | int | Nonedo_center_crop crop_size
do_rescalerescale_factor do_normalize
image_meanN image_stddo_padpad_sizedisable_groupingreturn_tensorsc Z |(t st d t || \ }}i }| D ]#\ }}|r| || }|||< $t
|| }t || \ }}i }| D ]<\ }}|r| || }| |||| |
| }|||< =t
|| }| r| ||
| }t d|i| S )Nz&Resampling is not supported in SLANeXt)r& )r r )r% r& r )data tensor_type) r logger warning_oncer itemsr r
center_croprescale_and_normalizepadr )rE r r r r r r r r r! r" r# r$ r% r& r' rF grouped_imagesgrouped_images_indexresized_images_groupedr stacked_imagesresized_imagesprocessed_images_groupedprocessed_imagess r. _preprocessz!SLANeXtImageProcessor._preprocess s & (@(B(B H I I I 0EV^n/o/o/o,,!# %3%9%9%;%; ; ; !E>
O!% N !N!N,: "5 ) )'(>@TUU 0E^fv/w/w/w,,#% %3%9%9%;%; = = !E>
M!%!1!1.)!L!L ! 7 7
NL*V_ N /= $U + +)*BDXYY p #xx(88^nx o o .2B!CQ_````r- rF c b t j di | | d S r? )rB rR init_decoderrD s r. rR z SLANeXtImageProcessor.__init__ s8 ""6""" r- c g d}|d t d D z
}|d t d D z
}d|vr| d d|v r| d dg|z dgz }d t | D | _ || _ g d
| _ | j d | _ | j d | _ d S ) a
Initialize the decoder vocabulary for table structure recognition.
Builds a character dictionary mapping HTML table structure tokens (e.g., ``, `
`, `
`, colspan/
rowspan attributes) to integer indices. The dictionary includes special `"sos"` (start-of-sequence) and
`"eos"` (end-of-sequence) tokens. Merged `
` tokens are used in place of standalone `
` tokens
when applicable.
)
zzz
zz
z
z
c g | ] }d |dz d S )z
colspan="r "r, .0is r.
z6SLANeXtImageProcessor.init_decoder.. ( DDDQ0A000DDDr- c g | ] }d |dz d S )z
rowspan="r r@ r, rA s r. rD z6SLANeXtImageProcessor.init_decoder.. rE r-
r< soseosc i | ]\ }}|| S r, r, )rB rC chars r.
z6SLANeXtImageProcessor.init_decoder.. s FFFDT1FFFr- )r< r= rH N) r r remove enumeraterA charactertd_tokenbos_ideos_id)rE dict_characters r. r: z"SLANeXtImageProcessor.init_decoder s
DD%))DDDDDD%))DDDD n , , ! !+ . . . ^ # # ! !& ) ) ) >1UG;FFIn,E,EFFF '444
i& i& r- c 0 |j | _ | j dd }t | j t | j g}t | j }| d }| d j }g }|j d }t | D ]}g } g }
t |j d D ]k} t ||| f } | dk r| |k r nE| |v r,| j
| }
| |
|
||| f l| | t j
|
}g d|d z g dz }||dS )aO
Post-process the raw model outputs to decode the predicted table structure into an HTML token sequence.
Converts the model's predicted probability distributions over the structure vocabulary into a sequence of
HTML tokens representing the table structure. The decoded tokens are wrapped with ``, ``, and
`
` tags to form a complete HTML table structure.
Args:
outputs ([`SLANeXtForTableRecognitionOutput`]):
Raw outputs from the SLANeXt model. The `last_hidden_state` field contains the predicted probability
distributions over the structure vocabulary at each decoding step, with shape
`(batch_size, max_text_length, num_classes)`.
Returns:
`dict`: A dictionary containing:
- **structure** (`list[str]`): The predicted HTML table structure as a list of tokens, wrapped with
``, ``, and `
` tags.
- **structure_score** (`float`): The mean confidence score across all predicted tokens.
r rQ r r )zzz
)z
zz) structurestructure_score)r predr* rR rS r r valuesr r rP r rd r meanitem)rE outputsstructure_probsignored_tokensend_idx
structure_idxstructure_str_listr batch_indexstructure_list
score_listpositionchar_idxtextrW rV s r. post_process_table_recognitionz4SLANeXtImageProcessor.post_process_table_recognition s ( - )AaC. dk**C ,<,<=dk""'..1.55
)--!-44; "(+
,,
D
DK NJ !-"5a"8 9 9
J
J }[(-B CDDa<r s ! ! ! ! ! ! 2 2 2 2 2 2 2 2 2 2 2 2 . . . . . . & & & & & & " " " " " " 9 9 9 9 9 9 3 3 3 3 3 3 ; ; ; ; ; ; 2 2 2 2 2 2 E E E E E E E E P P P P P P P P P P / / / / / / - - - - - - 4 4 4 4 4 4 4 4 l l l l l l l l l l l l l l C C C C C C C C * * * * * * 5 5 5 5 5 5 A A A A A A
H % % CDDD - ED 3 CDDD ( ( ( ( ($ ( ( ED (F + + + + +bi + + +B +A +A +A +A +A_ +A +A +A\ /
,
01f 1f 1f 1f 1f+ 1f 1f 1fh
5 5 5 5 5 5 5 5
!7
. : VL VL VL VL VL . VL VL VLr r-