$//r$ c
^ \ rS rSrSrSS/rSrSr SS\S\4U 4S jjjr SS
\
S \ \ \
\\ \\
4 S \\ S
\4S jjrSS jrS rS r\S 5 rSrU =r$ )PixtralProcessorA a
Constructs a Pixtral processor which wraps a Pixtral image processor and a Pixtral tokenizer into a single processor.
[`PixtralProcessor`] offers all the functionalities of [`CLIPImageProcessor`] and [`LlamaTokenizerFast`]. See the
[`~PixtralProcessor.__call__`] and [`~PixtralProcessor.decode`] for more information.
Args:
image_processor ([`PixtralImageProcessor`], *optional*):
The image processor is a required input.
tokenizer ([`LlamaTokenizerFast`], *optional*):
The tokenizer is a required input.
patch_size (`int`, *optional*, defaults to 16):
Patch size from the vision tower.
spatial_merge_size (`int`, *optional*, defaults to 1):
The downsampling factor for the spatial merge operation.
chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages
in a chat into a tokenizable string.
image_token (`str`, *optional*, defaults to `"[IMG]"`):
Special token used to denote image location.
image_break_token (`str`, *optional*, defaults to `"[IMG_BREAK]"`):
Special token used to denote the end of a line of pixels in an image.
image_end_token (`str`, *optional*, defaults to `"[IMG_END]"`):
Special token used to denote the end of an image input.
image_processor tokenizerAutoImageProcessor
AutoTokenizer
patch_sizespatial_merge_sizec > X0l X@l X`l UR U R 5 U l Xpl Xl UR U R 5 U l UR U R
5 U l UR U R 5 U l U R U R U R /U l [ T
U ]- XUS9 g )N)
chat_template) r: r; image_tokenconvert_tokens_to_idsimage_token_idimage_break_tokenimage_end_tokenimage_break_token_idimage_end_token_id image_idssuper__init__) selfr6 r7 r: r; r= r> rA rB kwargs __class__s r% rG PixtralProcessor.__init___ s %"4 & ' = =d>N>N O!2 . ' = =d>N>N O$-$C$CDDZDZ$[!"+"A"A$BVBV"W --t/H/H$JaJab
=Qr$ imagestextrI r' c ~ U R " [ 4SU R R 0UD6nU R U R
- nUGb [
U5 ( a U/nO[ U[ [ 45 ( a [
US 5 ( a Ou[ U[ [ 45 ( aO [ US [ [ 45 ( a1 [
US S 5 ( a U VV s/ sH o H oPM M nnn O [ S5 eU V
s/ sH% n
[ U
[ 5 ( a [ U
5 OU
PM' nn
U R " U4SU0US D6n O0 n [ U[ 5 ( a U/nO8[ U[ 5 ( d# [ US [ 5 ( d [ S5 eUn U R S5 Gb% [! U S 5 n
/ n / nU GH nU R" U; a [% U
5 u nnUU- nUU- nU R" /U- U R& /- /U- nU VVs/ sH
o H nUPM M nnnU R( US
' S R+ U5 nUR- U5 UR/ U R" S S
5 nU R" U; a M S U; a, UR1 S5 nUR/ S US
5 nS U; a M, U R- U5 GM US R1 SS5 nUS R1 SS5 nU R " U 40 US DSS0D6nU R3 U US/S9 U( ai [4 R6 " US 5 n[4 R8 " US 5 nS
U[4 R: " UU R< 5 ' UR? 5 US' [A 0 UEU EUS9$ s sn nf s sn
f s snnf )a
Main method to prepare for the model one or several sequences(s) and image(s). This method forwards the `text`
and `kwargs` arguments to LlamaTokenizerFast's [`~LlamaTokenizerFast.__call__`] if `text` is not `None` to encode
the text. To prepare the image(s), this method forwards the `images` and `kwrags` arguments to
CLIPImageProcessor's [`~CLIPImageProcessor.__call__`] if `images` is not `None`. Please refer to the docstring
of the above two methods for more information.
Args:
images (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `list[PIL.Image.Image]`, `list[np.ndarray]`, `list[torch.Tensor]`):
The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch
tensor. Both channels-first and channels-last formats are supported.
text (`str`, `list[str]`, `list[list[str]]`):
The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings
(pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set
`is_split_into_words=True` (to lift the ambiguity with a batch of sequences).
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors of a particular framework. Acceptable values are:
- `'tf'`: Return TensorFlow `tf.constant` objects.
- `'pt'`: Return PyTorch `torch.Tensor` objects.
- `'np'`: Return NumPy `np.ndarray` objects.
- `'jax'`: Return JAX `jnp.ndarray` objects.
Returns:
[`BatchFeature`]: A [`BatchFeature`] with the following fields:
- **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`.
- **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when
`return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not
`None`).
- **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`.
tokenizer_init_kwargsNr zdInvalid input images. Please provide a single image, a list of images, or a list of lists of images.r: r zAInvalid input text. Please provide a string, or a list of strings pixel_values image_sizes z
r r r r Fimage)
modalities input_idsmm_token_type_ids)data tensor_type)!
_merge_kwargsr r7 init_kwargsr: r; r2 r* listtuple
ValueErrorr+ r r6 TypeErrorgetiterr> nextrA rB joinappendreplacepop_check_special_mm_tokensnparray
zeros_likeisinrE tolistr ) rH rL rM audiovideosrI
output_kwargsr: sublistrT im image_inputsprompt_stringsrQ replace_stringssampleheightwidthnum_height_tokensnum_width_tokensreplace_tokensitem replace_strr r text_inputs array_idsrW s r% __call__PixtralProcessor.__call__w s R ** "
"&.."<"<
__t'>'>>
$V,, FT5M227LVTUY7W7W6D%=11 vay4-88)&)A,77/5KvG7%%7%vK z OUUf
2s(;(;jnCfFU //p:pQ^_nQopL L dC 6DD$''
47C0H0H _` ` N + 7 |M :;K N O &&&0$($5MFE(.*(<%',
':$ ))*-==AWAW@XX&)&*N ;I%].wU\TdU\d.N%])-)=)=N2&"$''."9K#**;7 #^^D,<, '')9A)=AR(RS "- "#c+&6 6 4D m n , ,,r$ c : U R R " U0 UD6$ )z
This method forwards all its arguments to LlamaTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please
refer to the docstring of this method for more information.
)r7 batch_decoderH argsrI s r% r PixtralProcessor.batch_decode s
~~**D;F;;r$ c : U R R " U0 UD6$ )z
This method forwards all its arguments to LlamaTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to
the docstring of this method for more information.
)r7 decoder s r% r PixtralProcessor.decode s
~~$$d5f55r$ c U R R nU R R n[ [ R X- 5 5 $ r0 )r7 model_input_namesr6 r\ dictfromkeys)rH tokenizer_input_namesimage_processor_input_namess r% r "PixtralProcessor.model_input_names s> !% @ @ &*&:&:&L&L#DMM"7"UVWWr$ ) rA rC rB rD rE r> r@ r: r; )NN r Nz[IMG]z [IMG_BREAK]z [IMG_END])NNNNr0 )r r r r! __doc__
attributesimage_processor_classtokenizer_classintrG r r r r
r\ r r r r r r r propertyr r#
__classcell__)rJ s @r% r4 r4 A s 2 $[1J 0%O "#'#R R
R R4 "^b q^q^ I 0$y/4HYCZZ[q^ /0
q^
q^f"-J<6 X Xr$ r4 )!r typingr numpyrh feature_extraction_utilsr image_utilsr r r processing_utilsr r
r r tokenization_utils_baser
r utilsr r image_processing_pixtralr
get_loggerr loggerr boolr. r2 r4 __all__r r$ r% r s 4 A A D 1 F
H %
-U
;4 ;
0bX~ bXJ
r$