it dZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z ddlmZdd lmZmZdd lmZdd lmZmZdd lmZd dlmZej8eZGddej>Z ejBjDdZ#dZ$Gddej>Z%d1dejLde'de(dejLfdZ)Gddej>Z*Gddej>Z+Gddej>Z,Gd d!ej>Z-d"Z.d#Z/Gd$d%eZ0Gd&d'ej>Z1eGd(d)eZ2eGd*d+e2Z3ed,-Gd.d/e e2Z4gd0Z5y)2zPyTorch ViTDet backbone.N)nn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)GradientCheckpointingLayer)BackboneOutputBaseModelOutput)PreTrainedModel)auto_docstringlogging)can_return_tuple) VitDetConfigc`eZdZdZfdZdZdejdejfdZxZ S)VitDetEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) to be consumed by a Transformer. cpt||j|j}}|j|j }}t |tjjr|n||f}t |tjjr|n||f}|d|dz|d|dzz}||_ ||_||_||_ |jr?|dz}tjtj d||j |_nd|_tj$|||||_y)Nrr) kernel_sizestride)super__init__pretrain_image_size patch_size num_channels hidden_size isinstance collectionsabcIterable image_size num_patches use_absolute_position_embeddingsr Parametertorchzerosposition_embeddingsConv2d projection) selfconfigr!rrrr" num_positions __class__s {/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/transformers/models/vitdet/modeling_vitdet.pyrzVitDetEmbeddings.__init__*s !'!;!;V=N=NJ $*$7$79K9Kk #-j+//:R:R#SZZdfpYq #-j+//:R:R#SZZdfpYq !!} 1 5*Q-:VW=:XY $$(&  2 2'!OM')||EKK=RXRdRd4e'fD $'+D $))L+:^hic|r |ddddf}|jd}ttj|}||z|k7r t dt j js ||k7s||k7r]tjj|jd||djdddd||fdd }|jddddS|jd||dS) a Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token dimension for the original embeddings. Args: abs_pos_embeddings (`torch.Tensor`): Absolute positional embeddings with (1, num_position, num_channels). has_cls_token (`bool`): If true, has 1 embedding in abs_pos_embeddings for cls token. height (`int`): Height of input image tokens. width (`int`): Width of input image tokens. Returns: Absolute positional embeddings after processing with shape (1, height, width, num_channels) Nrz5Absolute position embeddings must be a square number.rrbicubicF)sizemode align_corners) shapeintmathsqrt ValueErrorr%jit is_tracingr functional interpolatereshapepermute)r*abs_pos_embeddings has_cls_tokenheightwidth num_positionr4new_abs_pos_embeddingss r.get_absolute_positionsz'VitDetEmbeddings.get_absolute_positions@s$ !3AqrE!: )//2 499\*+ $;, &TU U 99   !dfn %']]%>%>"**1dD"=EEaAqQe_# &?& "*11!Q1= =%--aC Cr/ pixel_valuesreturncz|jd}||jk7rtd|jd|d|j|}|jc|j dddd}||j |jd|jd|jdz}|j dddd}|S) NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .rr2rT)r7rr;r)r'rArH)r*rIr embeddingss r.forwardzVitDetEmbeddings.forwardfs#))!, 4,, ,!../yaI __\2  # # /#++Aq!Q7J#d&A&A(($ 0@0@0CZEUEUVWEX'J$++Aq!Q7Jr/) __name__ __module__ __qualname____doc__rrHr%TensorrN __classcell__r-s@r.rr$s0 j,$DLELLU\\r/rcTtdt||zdz }|jd|k7rttjj |j d|jddjddd|d}|j d|jdd}n|}tj|dddft||z dz}tj|dddft||z dz}||z |dz t||z dzz}||jS) a Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (`int`): Size of query q. k_size (`int`): Size of key k. rel_pos (`torch.Tensor`): Relative position embeddings (num_embeddings, num_channels). Returns: Extracted positional embeddings according to relative positions. r2rrr1linear)r4r5Ng?) r8maxr7rr>r?r@rAr%arangelong)q_sizek_sizerel_pos max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r. get_rel_posrc|s% q3vv..23L}}Q<'--33 OOAw}}Q/ 4 < bhwkzbhwc,wkc->bhwkN)rcr7r@r%einsumview)attnqueries rel_pos_h rel_pos_wr[r\queries_height queries_width keys_height keys_widthrelative_heightrelative_width batch_size_dimr_qrelative_weights r.!add_decomposed_relative_positionsrvs,%+!NM$K!.+yIO  IFN J3 //*nmS ICll#3S/JOll#3S.IO *nm[*U !Q1d* + , !Q4* + , d:~ 5{Z7OP  Kr/c,eZdZdZdfd ZddZxZS)VitDetAttentionz=Multi-head Attention block with relative position embeddings.c t||j}|j}||_||z}|dz|_t j||dz|j|_ t j|||_ |j|_ |jrot jtjd|dzdz ||_t jtjd|dzdz ||_yy)z Args: config (`VitDetConfig`): Model configuration. input_size (`tuple[int]`, *optional*): Input resolution, only required in case relative position embeddings are added. grbiasr2rrN)rrrnum_attention_heads num_headsscalerLinearqkv_biasqkvproj use_relative_position_embeddingsr$r%r&rirj)r*r+ input_sizersr}head_dimr-s r.rzVitDetAttention.__init__s   .. ")#t^ 99S#'@IIc3' 060W0W-  0 0\\%++a*Q-6G!6KX*VWDN\\%++a*Q-6G!6KX*VWDN 1r/c 0|j\}}}}|j|j|||zd|jdj ddddd}|jd||jz||zdj d\}} } ||j z| jddz} |jr(t| ||j|j||f||f} | jd} | | z}|j||j||d}|j ddddd}|j|||d}|j|}|r>| j||j| jd| jd} || f} | S|f} | S) Nrr1r2rr)rs)r7rr@r}rAunbindr~ transposerrvrirjsoftmaxrfr)r* hidden_stateoutput_attentionsrqrDrErrrrhkeysvaluesattention_scoresattention_probsoutputss r.rNzVitDetAttention.forwards'3'9'9$ FE1hh|$,,Z%DNN\^_gghiklnoqrtuv # AzDNN/JFUZN\^ _ f fgh iv#djj0DNN2r4JJ  0 0@ '4>>4>>FTY?]cej\k  +22r2:&/ #((T^^VUTVW #++Aq!Q: #++JrJ yy. -55DNNO,A,A",EG\G\]_G`O$_5G$oGr/NFrOrPrQrRrrNrTrUs@r.rxrxsGX4r/rxinput drop_probtrainingrJc|dk(s|s|Sd|z }|jdfd|jdz zz}|tj||j|j z}|j |j||z}|S)zc Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). rr)r)dtypedevice)r7ndimr%randrrfloor_div)rrr keep_probr7 random_tensoroutputs r. drop_pathr s Cx I I [[^  Q 7 7E 5 ELL YYM YYy !M 1F Mr/cxeZdZdZd dedzddffd ZdejdejfdZde fdZ xZ S) VitDetDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).NrrJc0t|||_yr)rrr)r*rr-s r.rzVitDetDropPath.__init__s "r/ hidden_statescDt||j|jSr)rrr)r*rs r.rNzVitDetDropPath.forward s FFr/c d|jS)Nzp=)rr*s r. extra_reprzVitDetDropPath.extra_repr#sDNN#$$r/r) rOrPrQrRfloatrr%rSrNstrrrTrUs@r.rrsGb#%$,#$#GU\\GellG%C%r/rc*eZdZdZdfd ZdZxZS)VitDetLayerNormaL A LayerNorm variant, popularized by Transformers, that performs point-wise mean and variance normalization over the channel dimension for inputs that have shape (batch_size, channels, height, width). https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119 ct|tjt j ||_tjt j||_||_ |f|_ yr) rrrr$r%onesweightr&r{epsnormalized_shape)r*rrr-s r.rzVitDetLayerNorm.__init__.sT ll5::.>#?@ LL-=!>? !1 3r/c|jdd}||z jdjdd}||z tj||jzz }|j ddddf|z|j ddddfz}|S)NrT)keepdimr2)meanpowr%r:rrr{)r*xuss r.rNzVitDetLayerNorm.forward5s FF1dF # UKKN  4  0 UejjTXX. . KK4 & *TYYq$}-E Er/)gư>rrUs@r.rr's 4r/rc(eZdZdZfdZdZxZS)VitDetResBottleneckBlockz The standard bottleneck residual block without the last activation layer. It contains 3 conv layers with kernels 1x1, 3x3, 1x1. ct|tj||dd|_t ||_t|j|_ tj||ddd|_ t ||_ t|j|_ tj||dd|_ t ||_y)ar Args: config (`VitDetConfig`): Model configuration. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. bottleneck_channels (`int`): Number of output channels for the 3x3 "bottleneck" conv layers. rFrzr)paddingr{N)rrrr(conv1rnorm1r hidden_actact1conv2norm2act2conv3norm3)r*r+ in_channels out_channelsbottleneck_channelsr-s r.rz!VitDetResBottleneckBlock.__init__Cs YY{,?O $%89 6,,- YY24GTU\ab $%89 6,,- YY2L!%P $\2 r/cN|}|jD] }||} ||z}|Sr)children)r*routlayers r.rNz VitDetResBottleneckBlock.forward[s5]]_ E*C #g r/rrUs@r.rr=s 30r/rcdeZdZdededdffd Zdej dej fdZxZS) VitDetMlp in_featureshidden_featuresrJNct|tj|||_t |j |_tj|||_tj|j|_ yr) rrrrfc1rractfc2Dropout dropout_probdrop)r*r+rrr-s r.rzVitDetMlp.__init__esZ 99[/:&++,99_k:JJv223 r/rc|j|}|j|}|j|}|j|}|j|}|Sr)rrrr)r*rs r.rNzVitDetMlp.forwardlsH HHQK HHQK IIaL HHQK IIaLr/) rOrPrQr8rr%rSrNrTrUs@r.rrds84C4#4$4%,,r/rc `|j\}}}}|||zz |z}|||zz |z}tjj|ddd|d|f}||z||z} }|j |||z|| |z||}|j ddddddj j d|||} | || ffS)a  Partition into non-overlapping windows with padding if needed. Args: hidden_state (`torch.Tensor`): Input tokens with [batch_size, height, width, num_channels]. window_size (`int`): Window size. Returns: `tuple(torch.FloatTensor)` comprising various elements: - windows: windows after partition with [batch_size * num_windows, window_size, window_size, num_channels]. - (padded_height, padded_width): padded height and width before partition rrrr2rr1)r7rr>padrfrA contiguous) r window_sizerqrDrEr pad_height pad_width padded_height padded_widthwindowss r.window_partitionrvs/;.@.@+J| 44 CJu{22kAI==$$\Aq!Y:3VWL"(:"5uy7H}dN^N^._ +M+!% /"0" /q1 (,   a .}d>N>NP`cikpbqrM!4>>-#@@ %txx =@Y7Z([[ %--aAq9  " " MM-8M "W,r/)rrFr)rOrPrQrRrrr8boolrr%rSrrNrTrUs@r.rrsMqv!"!49!LO!im! !L#('||' ' u||U\\) *U5<<-@ @ 'r/rc feZdZdeddffd Z d dej dedededee zf d Z xZ S) VitDetEncoderr+rJNc t|||_|j}t j d|j |dDcgc]}|j}}g}t|D]I}|jt|||||jvr |jnd||jvKtj||_d|_ycc}w)Nrcpu)r)rrrF)rrr+num_hidden_layersr%linspaceritemrangeappendrwindow_block_indicesrresidual_block_indicesr ModuleListrgradient_checkpointing)r*r+depthrrlayersir-s r.rzVitDetEncoder.__init__s  ((-2NN1f>S>SUZch,ijq!&&(jju A MM#1!#4676;V;V6V 2 2\]'(F,I,I'I   ]]6* &+#ksCrroutput_hidden_states return_dictc|rdnd}|rdnd}t|jD]'\}}|r||fz}|||} | d}|s|| dfz})|r||fz}|std|||fDSt|||S)Nrrc3&K|] }|| ywrr).0vs r. z(VitDetEncoder.forward..5smq_`_lmslast_hidden_stater attentions) enumeraterrr ) r*rrrrall_hidden_statesall_self_attentionsr  layer_module layer_outputss r.rNzVitDetEncoder.forwards#7BD$5b4(4 POA|#$58H$H!(8IJM)!,M &9]1=M>> from transformers import VitDetConfig, VitDetModel >>> import torch >>> config = VitDetConfig() >>> model = VitDetModel(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 768, 14, 14] ```Nz You have to specify pixel_values)rrrrrr) r+rrrr;rMr:r rr) r*rIrrrkwargsembedding_outputencoder_outputssequence_outputs r.rNzVitDetModel.forwardqs82C1N-TXT_T_TqTq$8$D $++JjJj &1%r r%rSrrr rNrTrUs@r.r8r8bs|*&6*-1)-,0#' 5 llT)5  $;5 #Tk 5 D[ 5   5 5 r/r8zF ViTDet backbone, to be used with frameworks like Mask R-CNN. ) custom_introceZdZfdZdefdZeee d de jde dzde dzde dzde f d Z xZS) VitDetBackbonect||t||_t ||_t |jdzDcgc]}|jc}|_ |jycc}w)Nr) rrrrMrr:rrr num_featuresr;)r*r+rrr-s r.rzVitDetBackbone.__init__sd  *62$V, 9>v?W?WZ[?[9\]AV//] ^s A7rJc.|jjSrr=rs r.r>z#VitDetBackbone.get_input_embeddingsr?r/NrIrrrc ||n|jj}||n|jj}||n|jj}|j |}|j |d||}|r |j n|d}d} t|j|D]\} } | |jvs| | fz } |s|r | f|ddz} | S| f|ddz} | St| |r |j nd|jS)a Examples: ```python >>> from transformers import VitDetConfig, VitDetBackbone >>> import torch >>> config = VitDetConfig() >>> model = VitDetBackbone(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```NT)rrrrrr2) feature_mapsrr) r+rrrrMr:rzip stage_names out_featuresr r) r*rIrrrrArBrrrLstagerrs r.rNzVitDetBackbone.forwards5<&1%rrr r%rSrr rNrTrUs@r.rGrGs *&6* -1)-#' < ll< #Tk<  $; < D[ <  < !< r/rG)r8rrG)rF)6rRcollections.abcrr9r%rrr( activationsrbackbone_utilsrrmodeling_layersr modeling_outputsr r modeling_utilsr utilsr r utils.genericrconfiguration_vitdetr get_loggerrOloggerModulerr<script_if_tracingrcrvrxrSrrrrrrrrrrrrr8rG__all__rr/r.r`s &!H9?-,-.   H %UryyUp!3!3H&R;bii;~ U\\ e T V[VbVb  %RYY %bii,$ryy$N $2@>M,M`5 BII5 p!+O!+!+HD 'D D N M ]$9M  M ` Er/