idZddlmZddlmZddlmZddlmZm Z e je Z edeGd d eZ edeGd d eZd gZy )zEvolla model configuration)strict)PreTrainedConfig)RopeParameters)auto_docstringloggingzwestlake-repl/Evolla-10B-hf) checkpointc eZdZUdZdZeed<dZeed<dZeed<dZ eed <d Z eed <d Z eed <dZ eed<dZ eezed<dZeezed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed <y!)" SaProtConfigat mask_token_id (`int`, *optional*, defaults to 4): The id of the *mask* token in the protein sequence model. position_embedding_type (`str`, *optional*, defaults to `"rotary"`): The type of position embedding to use in the protein sequence model. Currently only `"rotary"` is supported. emb_layer_norm_before (`bool`, *optional*, defaults to `False`): Whether to apply layer normalization before the position embedding in the protein sequence model. token_dropout (`bool`, *optional*, defaults to `True`): Whether to apply dropout to the tokens in the protein sequence model. i vocab_size mask_token_id pad_token_idi hidden_size!num_hidden_layersnum_attention_headsiintermediate_size皙?hidden_dropout_probattention_probs_dropout_probimax_position_embeddings{Gz?initializer_rangeh㈵>layer_norm_epsrotaryposition_embedding_typeFemb_layer_norm_beforeT token_dropout is_decoderadd_cross_attentionN)__name__ __module__ __qualname____doc__r int__annotations__rrrrrrrfloatrrrrr strr!boolr"r#r$/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/transformers/models/evolla/configuration_evolla.pyr r s JM3L#Ks!!!s!'**03 %#+3#'S'#u#!NE!#+S+"'4'M4J %%r/r cpeZdZUdZdZdeiZdZdZe e zdze d<dZ e e d<dZe e d <d Ze e d <d Ze e d <d Ze e d<dZe dze d<dZee d<dZe e d<dZee d<dZee zdze d<dZee d<dZee zdze d<dZee d<dZe dze d<dZedze d <d!Z edze d"<dZ!e dze d#<d$Z"e dze d%<d&Z#e dze d'<dZ$e dze d(<d&Z%e dze d)<dZ&e dze d*<d+Z'ee d,<dZ(e dze d-<d.Z)e dze d/<d0Z*e e+e zdze d1<dZ,ee d2<dZ-ee d3<dZ.edze d4<dZ/edze d5<fd6Z0xZ1S)7 EvollaConfiga protein_encoder_config (`dict`, *optional*): Dictionary of configuration options used to initialize [`SaProtConfig`]. aligner_ffn_mult (`int`, *optional*, defaults to 4): The FFN multiplier for the aligner layer. aligner_enable_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in the aligner layer. aligner_attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities in the aligner layer. aligner_num_add_layers (`int`, *optional*, defaults to 8): The number of additional layers for the aligner layer. resampler_depth (`int`, *optional*, defaults to 6): The depth of the resampler layer in the llama model. resampler_dim_head (`int`, *optional*, defaults to 64): The dimension of the heads in the resampler layer in the llama model. resampler_heads (`int`, *optional*, defaults to 8): The number of heads in the resampler layer in the llama model. resampler_num_latents (`int`, *optional*, defaults to 64): The number of latents in the resampler layer in the llama model. resampler_ff_mult (`int`, *optional*, defaults to 4): The FFN multiplier for the resampler layer. Example: ```python >>> from transformers import EvollaModel, EvollaConfig >>> # Initializing a Evolla evolla-10b style configuration >>> configuration = EvollaConfig() >>> # Initializing a model from the evolla-10b style configuration >>> model = EvollaModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` EvollaModelprotein_encoder_configgANir iri8r rrnum_key_value_headssilu hidden_acti rr rms_norm_epsrope_parametersFattention_biasgattention_dropoutmlp_biasr aligner_ffn_multTaligner_enable_biasr$aligner_attention_probs_dropout_probaligner_num_add_layersresampler_depth@resampler_dim_headresampler_headsresampler_num_latentsresampler_ff_multrrri bos_token_idi  eos_token_id use_cachetie_word_embeddingsr#r$c |j%t|_tjdn4t |jt rtdi|j|_t |di|y)NzX`protein_encoder_config` is `None`. Initializing the `SaProtConfig` with default values.r.)r4r loggerinfo isinstancedictsuper __post_init__)selfkwargs __class__s r0rTzEvollaConfig.__post_init__s\  & & .*6.D ' KKr s 33T :*6*U9T9T*UD ' ''r/)2r%r&r'r( model_typer sub_configs default_thetar4rRrr*r r)rrrrr7r9r,rr:r+r;rr<r-r=r>r?r@rArBrDrFrGrHrIrrrJrKlistrLrMr#r$rT __classcell__)rWs@r0r2r2;s# JJ+\:KM=AD#33d:AJK"s"s!!&'t'J#'S'L%48O^d*T18 ND ,/us{T)/Hd#$cDj$'++9<(%$,<)*C$J*"#OS4Z#%'d '"#OS4Z#(*3:*$%sTz%#u##L#*#%L#*%+1L#S /D(1It %%#Jt #',,((r/r2N)r(huggingface_hub.dataclassesrconfiguration_utilsrmodeling_rope_utilsrutilsrr get_loggerr%rOr r2__all__r.r/r0rcs!.31,   H %89&#&:&>89P(#P(:P(f  r/