i@*ddlZddlZddlmZddlmZddlmZm Z ddl m Z m Z m Z mZddlmZmZddlmZmZmZdd lmZdd lmZerd d lmZmZej:eZGd de dZ eedGdde Z!dgZ"y)N) AudioInput) BatchFeature) ImageInputmake_nested_list_of_images)MultiModalDataProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInput)auto_docstringis_vision_availablelogging)requires) VideoInput)Gemma4ImageProcessorKwargs get_aspect_ratio_preserving_sizec4eZdZUeed<dddddiiddidZy)Gemma4ProcessorKwargs images_kwargsT)paddingreturn_mm_token_type_idsdo_convert_rgbreturn_metadata) text_kwargsr audio_kwargs videos_kwargsN)__name__ __module__ __qualname__r__annotations__ _defaults}/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/transformers/models/gemma4/processing_gemma4.pyrr$s5--(, d +T2 Ir&rF)total)vision)backendsceZdZ ddededeffd Ze ddedzdeeze eze ezde dzd e dzd e e d ef d Zd ed efdZddZefdZxZS)Gemma4ProcessorNimage_seq_lengthaudio_seq_lengthaudio_ms_per_tokenc ||_|j|_|j|_|j|_|j|_|j ddgid|_|j|j |_||_ ||_ t|dd|_ t|dd|_ t|dd|_t|dd|_t! |Dd |||||d| y) u image_seq_length (`int`, *optional*, defaults to 280): The number of soft tokens per image used for placeholder expansion. audio_seq_length (`int`, *optional*, defaults to 750): The maximum number of audio soft tokens per audio segment. Serves as an upper-bound cap when dynamic audio token counts are computed. audio_ms_per_token (`int`, *optional*, defaults to 40): Milliseconds of audio per output soft token. Used to dynamically compute the number of audio placeholder tokens as ``ceil(duration_ms / audio_ms_per_token)``. The default of 40 comes from the SSCP convolution's 4× time reduction on 10ms frames. additional_special_tokensz <|video|>audio_token_idN audio_token boa_token eoa_token)feature_extractorimage_processor tokenizervideo_processor chat_templater%)r-image_token_id boi_token eoi_token image_tokenadd_special_tokens video_tokenconvert_tokens_to_idsvideo_token_idr.r/getattrr2r3r4r5super__init__) selfr6r7r8r9r:r-r.r/kwargs __class__s r'rEzGemma4Processor.__init__6s.!1'66",,",,$00 $$&AK=%QR&'==d>N>NO !1#5%i1A4H"9mTB K> K>  /++'    r&imagestextaudiovideosrGreturnc J !||| | td|jtfd|jji|}t |t r|g}n.t |tst |dt s tdi}|F|jj|}t|}|j|fi|d}|jd} |s5|Dcgc]*}dj|jgt|z,}}t|t|k7r$tdt|d t|d | D cgc]+} |j |j| z|j"-} } t%|  t'j(|j} |D cgc]} t'j*| fd | }} i}||j,d'd |i|d }|jd}|j/ds|jd}n|d}g!t1||D]\}}|j2t4j7d|j2dn |j2|_|j8Dcgc]#}t;|dzddt;|dzd%}}!j=dj|Dcgc].}|d|j |j>|z|j"0c}t%!!t'j(|j>} |D cgc]} t'j*| !fd| }} i}|W|j@|jB |jD tdt |tFjHr|jJdk(r|g}|s|j@gt|z}|j/di}|jL|fi|}|jLjN}|Dcgc]}|jQ||}}|D cgc]+} |jB|j@| z|jD-} } t%|  t'j(|j@}|D cgc]} t'j*| fd| }} |djdd}|djdd}|jd'd|i|d}g}|j=d ||j=d!||j=d"|r|jS|||#|r|jU|d$|d%<tWi|||||&Scc}wcc} wcc} wcc}wcc}wcc} wcc}wcc} wcc} w)(Nz?Provide at least one of `text`, `images`, `audio`, or `videos`.tokenizer_init_kwargsrzAInvalid input text. Please provide a string, or a list of stringsrnum_soft_tokens_per_image z1Received inconsistently sized batches of images (z ) and text (z).ctSNnext_replacements_iters r'z*Gemma4Processor.__call__..s d3D.Er&rLrnum_soft_tokens_per_videorvideo_metadataaGemma 4 requires frame timestamps to construct prompts, but the `fps` of the input video could not be inferred. Probably `video_metadata` was missing from inputs and you passed pre-sampled frames. Defaulting to `fps=24`. Please provide `video_metadata` for more accurate results.<02d:ctSrSrT)rWvideo_replacementss r'rYz*Gemma4Processor.__call__..s d3E.Fr&zUAudio inputs were provided, but the tokenizer does not have an `audio_token` defined.rrctSrSrTrVs r'rYz*Gemma4Processor.__call__..s D9J4Kr&rreturn_tensorsrFrJimagevideorK) modalities input_idsmm_token_type_ids)data tensor_typer%), ValueError _merge_kwargsrr8 init_kwargs isinstancestrlist TypeErrorr7 fetch_imagesrpopjoinr>lenr<r=iterreescapesubr9getzipfpslogger warning_once timestampsintappendr@r3r4r5npndarrayndimr6 sampling_rate_compute_audio_num_tokens_check_special_mm_tokenscreate_mm_token_type_idsr)"rFrIrJrKrLrG output_kwargs image_inputsbatched_imagesnum_soft_tokensn replacementspatternprompt video_inputsnum_video_tokensr[metadatan_tokensseconds timestamp_strt audio_inputsrranum_audio_tokens audio_patternrcr text_inputsactive_modalitiesrXras" @@r'__call__zGemma4Processor.__call__ns F7?N/4//Y-:XYL*../JKOQ_`v$"2"2!3c&k!AB``>"c$i/ GNH[G\\hilmqirhssuv`ooZ[t~~.t/?/?!/C.DT^^DTUoLo $\ 2 ii 0 01G]abSYBFF7$EvNbDb  /4//`v`A_`L+//0KL ::/0!-!1!12B!C!-.>!?!# &).:J&K "(<<''', &.\\%9rx|| X`WjWj!LSs7b=)#.aGbL0A#/FG! !#))HHgtubcA3a/0@0@80K/LT^^L\]u &"&&8!9 ii 0 01G^bcTZBFF7$FOcDc  '4>>+AT^^E[ k %,q(()CJ6),,^R@L1411%H<HL 22@@MZ_`UV > >q- P` `_opZ[t~~.t/?/?!/C.DT^^DTUpLp $\ 2 IId&6&67McghY_BFF=*KVThDh&}599:JDQ#0#?#C#CD^`e#f $dnnO$O- 2NO     $ $W -    $ $W -    $ $W -   ) )$ HY ) Z #/3/L/L[YdMe/fK+ ,PKP<P<P<P&  Gap c2! v d4 apis6/U8 0U=-!V(V 3V 9!VV50V!V rc0t|}tt|dzdz }tt|dzdz }|dz}|dz}||z}||z |zdz} | dkry| } tdD]} | dz} | dz dzdz} t | |j S)aQCompute the number of audio soft tokens for a single waveform. Replicates the exact sequence-length arithmetic of the audio encoder so that the processor inserts the correct number of placeholder tokens. The computation mirrors: 1. Mel framing via ``_unfold`` in ``Gemma4AudioFeatureExtractor`` 2. Two ``Conv2d`` subsampling layers in ``Gemma4AudioSubSampleConvProjection`` (each: kernel=3, stride=2, semicausal padding top=1, bottom=1) The result is capped at ``self.audio_seq_length`` (the configured maximum). Args: audio_waveform: A 1-D numpy array or list containing the raw audio samples. sampling_rate: The sampling rate of the audio waveform in Hz. Returns: The number of audio soft tokens to insert as placeholders. g4@g@@g$@rrr)rurroundrangeminr.) rFaudio_waveformr num_samples frame_length hop_lengthframe_size_for_unfoldpad_leftpadded_samplesnum_mel_framesrrWt_paddeds r'rz)Gemma4Processor._compute_audio_num_tokenss(.) 5!5!>?@ }t3f<=> ,q 0  1$$x/(+@@ZORSS Q  q (A1uHA!#a'A ( 1d++,,r&c &tjjdi}|j||jddxs|jj }|jddxs|jj }|jddxs|jj}||dzz}i} |ig} |D]?} t| d| d||| \} } | |z}| |z}| j||z|dzzAdgt|z}| j| |d |\t|jd d }|Dcgc]'}|jtj||)}}| jd |it!di| Scc}w)av Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. audio_lengths (`list[int]`, *optional*): The lengths of audio inputs in number of samples. Used to dynamically compute per-audio token counts. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. r patch_sizeNpooling_kernel_sizemax_soft_tokensrrr)heightwidthr max_patchesr)num_image_tokensnum_image_patchesri>rr%)rr$rzupdater7rrrrrrurCr6rrzerosr)rF image_sizes audio_lengthsrGrrrrr vision_datar image_sizetarget_htarget_w patch_height patch_widthrrlengthrs r'_get_num_multimodal_tokensz*Gemma4Processor._get_num_multimodal_tokens&s .77;;ORP V$"&&|T:]d>R>R>]>]   3T : fd>R>R>f>f (++,=tDlH\H\HlHl%(;Q(>>   "! ) ^ %E%a=$Q-) +(; &"( (:5 &*4  '' {(BFY[\F\(\] ^"#c+&6 6    4D[lm n  $$D$:$:OVTM^k TZ..rxx/?O       24DE F, ,,  s,Fct|}|Dcgc]}|dvr| }}|j:|jj}|j|Dcgc] }||vs| c}|dgzScc}wcc}w)N)rPrZrh)rDmodel_input_namesr6extend)rFrnamefeature_extractor_input_namesrHs r'rz!Gemma4Processor.model_input_names]s!G5* UU     ! ! -,0,B,B,T,T )  $ $7T%vtX\duXud%v w $7#888 &ws A+ A0A0)Nii()NNNN)NN)r r!r"rrErrr r rprrr rrrrrpropertyr __classcell__)rHs@r'r,r,3s # #"$6 6 6  6 p%)Z^#'$( F T!F ++d9o=EV@WWF D F T! F ./ F  F F P--s--s--^5-n 9 9r&r,)#rwnumpyr audio_utilsrimage_processing_utilsr image_utilsrrprocessing_utilsrr r r tokenization_utils_baser r utilsrrrutils.import_utilsr video_utilsrimage_processing_pil_gemma4rr get_loggerr r}rr,__all__r%r&r'rs %2AXXCAA*%i   H % ,E  ;v9nv9 v9r  r&