iY U d Z ddlZddlZddlmZ ddlmZ ddlmZ ddl m
Z
m Z ddl m
Z
dd lmZ dd
lmZ dd lmZ dd lmZmZmZmZmZ dd
lmZ ddl m Z ddl m Z m Z m!Z!m"Z" ddl#m$Z$ ddl%m&Z& ddl'm(Z( ddl)m*Z* ejV e, Z-er e Z.ee/e/dz f e0d< n
eg d Z. e e e. Z1de/fdZ2 G d d Z3ddgZ4y) zAutoProcessor class. N) OrderedDict)
TYPE_CHECKING )PreTrainedConfig) get_class_from_dynamic_moduleresolve_trust_remote_code)FeatureExtractionMixin)ImageProcessingMixin)ProcessorMixin)TOKENIZER_CONFIG_FILE)FEATURE_EXTRACTOR_NAMEPROCESSOR_NAMEVIDEO_PROCESSOR_NAME cached_filelogging)BaseVideoProcessor )_LazyAutoMapping)CONFIG_MAPPING_NAMES
AutoConfigmodel_type_to_module_name!replace_list_option_in_docstrings)AutoFeatureExtractor)AutoImageProcessor)
AutoTokenizer)AutoVideoProcessorPROCESSOR_MAPPING_NAMES))aimv2
CLIPProcessor)alignAlignProcessor)altclipAltCLIPProcessor)aria
AriaProcessor)audioflamingo3AudioFlamingo3Processor)
aya_visionAyaVisionProcessor)bark
BarkProcessor)blip
BlipProcessor)zblip-2Blip2Processor) bridgetowerBridgeTowerProcessor) chameleonChameleonProcessor) chinese_clipChineseCLIPProcessor)clap
ClapProcessor)clipr )clipsegCLIPSegProcessor)clvp
ClvpProcessor)cohere2_visionCohere2VisionProcessor)
cohere_asrCohereAsrProcessor)colmodernvbertColModernVBertProcessor)colpaliColPaliProcessor)colqwen2ColQwen2Processor) deepseek_vlDeepseekVLProcessor)deepseek_vl_hybridDeepseekVLHybridProcessor)dia DiaProcessor)edgetam
Sam2Processor)emu3
Emu3Processor)ernie4_5_vl_moeErnie4_5_VLMoeProcessor)evollaEvollaProcessor)flavaFlavaProcessor) florence2Florence2Processor)fuyu
FuyuProcessor)gemma3Gemma3Processor)gemma3nGemma3nProcessor)gemma4Gemma4Processor)git GitProcessor)glm46vGlm46VProcessor)glm4vGlm4vProcessor) glm4v_moere ) glm_imagere )glmasrGlmAsrProcessor)got_ocr2GotOcr2Processor)granite_speechGraniteSpeechProcessor)zgrounding-dinoGroundingDinoProcessor)groupvitr )higgs_audio_v2HiggsAudioV2Processor)hubertWav2Vec2Processor)ideficsIdeficsProcessor)idefics2Idefics2Processor)idefics3Idefics3Processor) instructblipInstructBlipProcessor)instructblipvideoInstructBlipVideoProcessor)internvlInternVLProcessor)janusJanusProcessor)zkosmos-2Kosmos2Processor)z
kosmos-2.5Kosmos2_5Processor)kyutai_speech_to_textKyutaiSpeechToTextProcessor)lasr_ctc
LasrProcessor) lasr_encoderr )
layoutlmv2LayoutLMv2Processor)
layoutlmv3LayoutLMv3Processor) layoutxlmLayoutXLMProcessor)lfm2_vlLfm2VlProcessor) lighton_ocrLightOnOcrProcessor)llama4Llama4Processor)llavaLlavaProcessor)
llava_nextLlavaNextProcessor)llava_next_videoLlavaNextVideoProcessor)llava_onevisionLlavaOnevisionProcessor)markuplmMarkupLMProcessor)
metaclip_2r )zmgp-strMgpstrProcessor)mistral3PixtralProcessor)mllamaMllamaProcessor)zmm-grounding-dinorn ) modernvbertry ) moonshiners )moonshine_streamingMoonshineStreamingProcessor)
musicflamingoMusicFlamingoProcessor)z omdet-turboOmDetTurboProcessor) oneformerOneFormerProcessor)ovis2Ovis2Processor)owlv2Owlv2Processor)owlvitOwlViTProcessor) paddleocr_vlPaddleOCRVLProcessor) paligemmaPaliGemmaProcessor)
perception_lmPerceptionLMProcessor)phi4_multimodalPhi4MultimodalProcessor)pi0 PI0Processor)
pix2structPix2StructProcessor)pixtralr ) pop2pianoPop2PianoProcessor)pp_chart2tablePPChart2TableProcessor) qwen2_5_omniQwen2_5OmniProcessor)
qwen2_5_vlQwen2_5_VLProcessor) qwen2_audioQwen2AudioProcessor)qwen2_vlQwen2VLProcessor)qwen3_5Qwen3VLProcessor) qwen3_5_moer )qwen3_omni_moeQwen3OmniMoeProcessor)qwen3_vlr ) qwen3_vl_moer )sam SamProcessor)sam2rM )sam3
Sam3Processor)sam_hqSamHQProcessor) seamless_m4tSeamlessM4TProcessor)sewrs )zsew-drs ) shieldgemma2ShieldGemma2Processor)siglipSiglipProcessor)siglip2Siglip2Processor)smolvlmSmolVLMProcessor)speech_to_textSpeech2TextProcessor)speecht5SpeechT5Processor)t5gemma2r[ )t5gemma2_encoderr[ )trocrTrOCRProcessor)tvp TvpProcessor)udop
UdopProcessor) unispeechrs )z
unispeech-satrs )
vibevoice_asrVibeVoiceAsrProcessor) video_llavaVideoLlavaProcessor)vilt
ViltProcessor)vipllavar )zvision-text-dual-encoder VisionTextDualEncoderProcessor)voxtralVoxtralProcessor)voxtral_realtimeVoxtralRealtimeProcessor)wav2vec2rs )z
wav2vec2-bertrs )zwav2vec2-conformerrs )wavlmrs )whisperWhisperProcessor)xclipXCLIPProcessor
class_namec t j D ]< \ }}| |v s t | }t j d| d } t || c S t j j D ] }t |dd | k( s|c S t j d }t || r t || S y # t $ r Y w xY w)N.ztransformers.models__name__ transformers) r itemsr importlib
import_modulegetattrAttributeErrorPROCESSOR_MAPPING_extra_contentvalueshasattr)r module_name
processorsmodule processor main_modules y/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/transformers/models/auto/processing_auto.pyprocessor_class_from_namer s #:#@#@#B Z #3K@K ,,q
->@UVF
vz22
'55<<> 9j$ /: = )).9K{J'{J// "
s B99 CCc N e Zd ZdZd Ze ee d Ze dd Z
y)
AutoProcessora
This is a generic processor class that will be instantiated as one of the processor classes of the library when
created with the [`AutoProcessor.from_pretrained`] class method.
This class cannot be instantiated directly using `__init__()` (throws an error).
c t d )Nz}AutoProcessor is designed to be instantiated using the `AutoProcessor.from_pretrained(pretrained_model_name_or_path)` method.)OSError)selfs r __init__zAutoProcessor.__init__ s
_
c T |j dd }|j dd }d|d< d}d}d}|D ci c]
}||v s||| } }| j dddd t |t fi | }
|
Ft j
|fi |\ } } | j
d }d
| j
d i v r| d d
}|t |t fi | }
|
Gt j |fi |\ } } | j
d d }d
| j
d i v r| d d
}|
Zt |t fi | }
|
Gt j |fi |\ } } | j
d d }d
| j
d i v r| d d
}|
\t |t fi | }
|
I|Gt j |fi |\ } } | j
d d }d
| j
d i v r| d d
}|lt |t fi | }|Yt! |d
5 }t# j$ | } ddd j
d d }d
| j
d i v r| d d
}|_ t' |t( st+ j, |fd|i|}t/ |d d }t1 |d r d
|j2 v r|j2 d
}| t7 | }|du}|duxs t9 | t: v }|xr0 |xs t: t9 | j< j? d }|r*d|v r|jA d d }nd}tC ||||| }|rH|rF|sDtE ||fi |}|j dd } |jG |j, |fd|i|S | |j, |fd|i|S t9 | t: v r"t; t9 | j, |fi |S tH tJ tL tN fD ] } |j, |fd|i|c S t5 d| d c c}w # 1 sw Y xY w# t4 $ r Y {w xY w# tP $ r Y Xw xY w)a3
Instantiate one of the processor classes of the library from a pretrained model vocabulary.
The processor class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible):
List options
Params:
pretrained_model_name_or_path (`str` or `os.PathLike`):
This can be either:
- a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on
huggingface.co.
- a path to a *directory* containing a processor files saved using the `save_pretrained()` method,
e.g., `./my_model_directory/`.
cache_dir (`str` or `os.PathLike`, *optional*):
Path to a directory in which a downloaded pretrained model feature extractor should be cached if the
standard cache should not be used.
force_download (`bool`, *optional*, defaults to `False`):
Whether or not to force to (re-)download the feature extractor files and override the cached versions
if they exist.
proxies (`dict[str, str]`, *optional*):
A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',
'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.
token (`str` or *bool*, *optional*):
The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
when running `hf auth login` (stored in `~/.huggingface`).
revision (`str`, *optional*, defaults to `"main"`):
The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
identifier allowed by git.
return_unused_kwargs (`bool`, *optional*, defaults to `False`):
If `False`, then this function returns just the final feature extractor object. If `True`, then this
functions returns a `Tuple(feature_extractor, unused_kwargs)` where *unused_kwargs* is a dictionary
consisting of the key/value pairs whose keys are not feature extractor attributes: i.e., the part of
`kwargs` which has not been used to update `feature_extractor` and is otherwise ignored.
trust_remote_code (`bool`, *optional*, defaults to `False`):
Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
should only be set to `True` for repositories you trust and in which you have read the code, as it will
execute code present on the Hub on your local machine.
kwargs (`dict[str, Any]`, *optional*):
The values in kwargs of any keys which are feature extractor attributes will be used to override the
loaded values. Behavior concerning key/value pairs whose keys are *not* feature extractor attributes is
controlled by the `return_unused_kwargs` keyword parameter.
Passing `token=True` is required when you want to use a private model.
Examples:
```python
>>> from transformers import AutoProcessor
>>> # Download processor from huggingface.co and cache.
>>> processor = AutoProcessor.from_pretrained("facebook/wav2vec2-base-960h")
>>> # If processor files are in a directory (e.g. processor was saved using *save_pretrained('./test/saved_model/')*)
>>> # processor = AutoProcessor.from_pretrained("./test/saved_model/")
```configNtrust_remote_codeT
_from_auto) cache_dirforce_downloadproxiestokenrevisionlocal_files_only subfolder repo_type
user_agentF) _raise_exceptions_for_gated_repo%_raise_exceptions_for_missing_entries'_raise_exceptions_for_connection_errorsprocessor_classr auto_mapzutf-8)encodingz
transformers.z--r
code_revisionz!Unrecognized processing class in z. Can't instantiate a processor, a tokenizer, an image processor, a video processor or a feature extractor for this model. Make sure the repository contains the files of at least one of those processing classes.))popupdater r r get_processor_dictgetr
r
get_image_processor_dictr r get_video_processor_dictr get_feature_extractor_dictr openjsonload
isinstancer r from_pretrainedr
r r2
ValueErrorr typer
__module__
startswithsplitr r register_for_auto_classr r r r Exception)cls pretrained_model_name_or_pathkwargsr" r# r1 processor_auto_map_hub_valid_kwargskeycached_file_kwargsprocessor_config_file config_dict_preprocessor_config_filetokenizer_config_filereaderhas_remote_codehas_local_codeexplicit_local_code
upstream_repoklasss r r@ z AutoProcessor.from_pretrained so D Hd+ "JJ':D A #| !
;L ]3sV\}c6#; . ] ]!!499>;@
!,,I> p]o p , + > >?\ g`f gNK )oo.? @O +//*b"AA%0%<_%M" "'2-/E(I[( $ (3!5!N!NOl!wpv!w Q"-//2CT"J"kooj"&EE)4Z)@ )Q& (/+613G,K],( ,7%7%P%P5&9?&NK '2oo6G&NO&+//*b*II-8-D_-U*'/+613I,M_,( ,7O 0f >
*ZZ021 )) 2 8 8 >q A
$
9!#@.Racp! 09L ;"$A EK O
?D1A 3 3 52?22-ARV\
(2?22-ARV\
&\ .
.$T&\2BBC`kdjk k $%79KMab E
,u,,1EVZ` /0M/N Og
g
G ^x4 4,
\
s; O9O9/O>>A P P>P PP P'&P'c 4 t j | || y)a
Register a new processor for this class.
Args:
config_class ([`PreTrainedConfig`]):
The configuration corresponding to the model to register.
processor_class ([`ProcessorMixin`]): The processor to register.
)exist_okN)r register) config_classr1 r[ s r r\ zAutoProcessor.register s ""< 8"Tr N)F) r rC __qualname____doc__r classmethodr r r@ staticmethodr\ r r r r sH
&'>?[
@ [
z U Ur r r )5r_ r r= collectionsr typingr configuration_utilsr dynamic_module_utilsr r feature_extraction_utilsr image_processing_utilsr
processing_utilsr tokenization_pythonr utilsr
r r r r video_processing_utilsr auto_factoryr configuration_autor r r r feature_extraction_autor image_processing_autor tokenization_autor video_processing_autor
get_loggerr loggerr str__annotations__r r r __all__rb r r rx s # 4 \ > : . 8 g g 8 * : 5 , 5
H % =HM[cDj 9I )G
I V %%9;RS # 0wU wUt
0r