iS JUddlZddlZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z ddl m Z mZmZmZmZmZmZmZmZmZmZddlZddlmZmZmZddlmZddlm Z dd l!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+dd l!m,Z-dd l!m.Z/dd l!m0Z0e d dZ1e e2e3d <ddl m4Z4ddl5m6Z6ddl6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>dhdZ?dZ@dZAdideBddfdZCde de2fdZDde de2fdZEde de2fdZFde de2fdZGde de2fd ZHde de2fd!ZIde de2fd"ZJde de2fd#ZKde de2fd$ZLde de2fd%ZMde de2fd&ZNd'ZOd(eBdd)fd*ZPde2fd+ZQdid(eBde2fd,ZRdid(eBde2fd-ZSd.e8d/e8ddfd0ZT djd1d2d3e:d4eeBd5eUde9fd6ZVd7d8d9ee8eZZd?ee ge2fd@ee ge fde de fdAZ[d?ee ge2fde de fdBZ\ dldCe8dDe2dEedddFfdGZ]ddHdIdFdJedKde8fdLZ^ dmdCe8dDe2dMe2ddNfdOZ_ddHdPdNdJedKde8fdQZ` dndCe8dDe2ddRfdSZaddHdTdRdJedKde8fdUZbedVZcdWedXecfdYe dZe dedXecffd[ZdGd\d]eeZfd^egdWee e e2ge fd9e d3e ddf d_Zhejdod`ZjejdaZkejdpdbegdceBfddZle GdedfZmgdgZny)qN) ContextVar) dataclass) AnyCallableDictListMappingOptionalSequenceTupleTypeVarUnioncast)SchemaValidationError validate_type)Version)table) cupycupy_from_dlpackhas_cupy has_cupy_gpuhas_gpu has_mxnethas_tensorflow has_torchhas_torch_cuda_gpu has_torch_mps)mxnet) tensorflow)torchDATA_VALIDATIONF)default) TYPE_CHECKING)types) ArgsKwargsArrayXdFloatsXdIntsXdPaddedRagged)Opsreturn torch.devicecFt tdddlm}ddlm}ddlm}|}t||r6tjj}tjd|St||rtjdStjdS) Nz'9#'>>rBcPt|xrt|dxr |jS)Nis_mps)rahasattrrirYs r@rfrfs" #  H73#9 HcjjHrBcHtsyt|tjryyr_)rr:tfr`rYs r@is_tensorflow_arrayrms  C #rBc8t|xrd|jvS)NzGPU:)rmr=rYs r@is_tensorflow_gpu_arrayros s # <#**(<s r@ require_cpurs2 %.CC rBc2tr t|tS)z?Use GPU if it's available. Returns True if so, False otherwise.r~)r require_gpurs r@ prefer_gpurs6" NrBcTddlm}m}m}t j dk(r"t str tdtdt j dk7rts tdts tdtr||t|y||y) Nr)r2r3rDarwinz6Cannot use GPU, installed PyTorch does not support MPSz(Cannot use GPU, PyTorch is not installedz%Cannot use GPU, CuPy is not installedzNo GPU devices detectedT) r7r2r3rplatformsystemrrr6rrrr)r~r2r3rs r@rrs::H$] UV VCDD  h &x@AA 233 "v  ! rBdstsrcct|tjr t|tjr||ddyt|r.t j |d}t j ||ytj ||y)NF)copy)r:rEr\rFrr{copyto)rrs r@ copy_arrayrsZ#u}}%*S%--*HA s jj5) C S#rB)label_smoothingY n_classesrc |!ttj|dz}|dkr td|dk(r|dk(r tdd}n|dkDstd|d||dz z }|dz |z }|dkDr||k\rtd|d |d |d t |}|j ||f|d }|j |d|z ||S)Nrrz>Label-smoothing parameter has to be greater than or equal to 0rzn_classes should be at least 1zGn_classes should be greater than 1 when label smoothing is enabled,but z was provided.zFor z7 classes label_smoothing parameter has to be less than z , but found .float32)dtype)intrEmaxr6rIfull fill_diagonal)rrr nongold_prob max_smoothxp label_distrs r@to_categoricalrs  ! q()  L  # >=> > 1} k1 ')a-8 a-9,J1}J69+=l,&7q :  ! B''9i0,i'PK[!o"56 q>rBdimXrc*t|trt|j|St|trt|j|St |drxt |drlt t|}t|jdk(ryt|jdk(rt|jdzS|j|St|ttfrt|dk(ryt|d|Sd}t|)zrInfer the 'width' of a batch of data, which could be any of: Array, Ragged, Padded or Sequence of Arrays. rshapendimrrz=Cannot get width of object: has neither shape nor __getitem__)r:r, get_widthr|r+rjrr(lenrrrlisttupler6)rrerrs r@rr&s !VS)) Av S)) G F!3 !  qww<1  \Q quuw=2.0.0,<2.6.0)pkgN)r ImportErrorformat)templates r@assert_tensorflow_installedrBs)RH (//.H/IJJ rBc&ts tdy)z/Raise an ImportError if MXNet is not installed.zjMXNet support requires mxnet: pip install thinc[mxnet] Enable MXNet support with thinc.api.enable_mxnet()N)rrrLrBr@assert_mxnet_installedrIs  z   rBc&ts tdy)z1Raise an ImportError if PyTorch is not installed.z8PyTorch support requires torch: pip install thinc[torch]N)rrrLrBr@assert_pytorch_installedrQs TUU rBis_match convert_itemc|r|St|tr9tt|j }tj |St|t r;i}|j D]$\}}t|}t|}|||<&|St|tr|Dcgc]}t|c}St|trtfd|DS|Scc}w)zEither convert a single value if it matches a given function, or recursively walk over potentially nested lists, tuples and dicts applying the conversion, and returns the same type. Also supports the ArgsKwargs dataclass. c38K|]}t|ywrK)convert_recursive).0itemrrs r@ z$convert_recursive..nsU&xtDUs)r:r'rritems from_itemsdictr)rrrW convertedkeyvaluers`` r@rrWs}C  C $%h d399;>OP $$Y// C  ))+ #JC#HlC@C%h eDE"IcN # C LOPD!(L$?PP C UQTUUU Qs;C9c#K||r|yt|tr,t|t|j Ed{yt|t rA|j D]-\}}t||Ed{t||Ed{/yt|tst|t r|D]}t||Ed{yy77Z7H7w)zEither yield a single value if it matches a given function, or recursively walk over potentially nested lists, tuples and dicts yielding matching values. Also supports the ArgsKwargs dataclass. N)r:r'iterate_recursiverrrr)rrWrrrs r@rrss } C $$XtCIIK/@AAA C ))+ :JC(37 7 7(59 9 9 : C *S%"8 9D(48 8 8 9#9 B 8 9 9sHAC"C:C"CC"C;C"C  C"C"C" C" xp_tensor requires_gradr=z torch.Tensorct| t}t|dr:|j}tj j j|}nKt|dr*tj j j|}nt j|}|j|}|r|j|S)z3Convert a numpy or cupy tensor to a PyTorch tensor.toDlpack __dlpack__) rrArjrr"utilsdlpack from_dlpack from_numpytorequires_grad_)rrr= dlpack_tensor torch_tensors r@xp2torchrs  ~)+y*%!**, {{))55mD L ){{))55i@ '' 2 ??6*L##% rB)r>rr>r-cddlm}tt|rjt ||r,|j j jSttjjj|St ||s|,|j j jStj|S)zConvert a torch tensor to a numpy or cupy tensor depending on the `ops` parameter. If `ops` is `None`, the type of the resultant tensor will be determined by the source tensor's device. rNumpyOps)apirrrdr:detachr5rErr"rr to_dlpackrasarray)rr>rs r@torch2xprs <( c8 $&&(,,.446 6#EKK$6$6$@$@$NO O c8 $ &&(,,.446 6<< - -rB as_variablez tf.Tensorctt|dr:|j}tjj j |}n[t|dr:|j}tjj j |}ntj|}|r?tj|j5tj||}ddd|durC|dur?tj|j5tj|}ddd|S|S#1swYRxYw#1swY|SxYw)zAConvert a numpy or cupy tensor to a TensorFlow Tensor or Variablerr) trainableNF) rrjrrl experimentalrrrconvert_to_tensorr=Variable stop_gradient)rrrr tf_tensors r@ xp2tensorflowrs !y*%!**, OO**66}E L )!,,. OO**66}E ((3 YYy'' ( H IGI H+"6YYy'' ( 4((3I 4 9 H H  4 s D6E6D?E rcZddlm}tt|rPt ||r|j St jjj|}t|St ||s||j Stj|j S)zConvert a Tensorflow tensor to numpy or cupy tensor depending on the `ops` parameter. If `ops` is `None`, the type of the resultant tensor will be determined by the source tensor's device. rr) rrrror:rErlrrrrrr)rr>rrs r@ tensorflow2xprs !y) c8 $??$ $OO22<rs r@mxnet2xprs)$ c8 $##%--/ /#I$A$A$CD D c8 $ ##%--/ /<< 1 1 34 4rBPartialTfunc.argskwargscZtj|g|i|}|j|_|S)znWrapper around functools.partial that retains docstrings and can include other workarounds if needed. ) functoolspartial__doc__)rrr partial_funcs r@rr s0 $$T;D;F;L< locmsgz  N)rGjoinrzstrappendrr6__init__) selfrrrrmessage type_infor|errorperr_locresults r@r zDataValidationError.__init__s/tfA6$q'$tAwi0  5Ekk599UB3G"Ha3q6"HIG KK%))E"23 4 59eDk2D&499V+<"<=#IsB; ) __name__ __module__ __qualname__r rrr r rrr rLrBr@rrsc LN >> >  > hwsCx014S#X3GGH >  >rBrrctj|}t|j}t |dk7r6t |ddj |d}d|}t |||d|ig|j|d}|j|d} g} |a|jturOt|trt |d kDr|d d }t||j}|r| jd |d |`tj|j} | tjj ur%t|d f| }|r| jd|d | rt |||| y )zValidate the input and output of a forward function against the type annotations, if available. Used in Model.initialize with the input and output samples as they pass through the network. z (z, )zIInvalid forward function. Expected 3 arguments (model, X, is_train), got rrN)r)rrc|SrKrL)xs r@z+validate_fwd_input_output..BsarB)r)r from_functionr model_fieldsrrr annotationrr:rr inspect signaturereturn_annotation Signatureempty) rrrrschemafields bad_paramsrx_fieldy_fieldrrets r@validate_fwd_input_outputr,)sf ! !$ 'F &%% &F 6{aF }Btyy'8&9; YZdYef!$1s|n==!!&),G!!&),G F}++36 a 3q6A:"1AAw112  MM&5 6}%77 g''-- -K 0#6C fS9: !$1f55rBc#Ktj|d}||jtj|j yw)NF)modedelete)tempfileNamedTemporaryFilecloseosremover)r.fs r@ make_tempfiler6Is6##es r@use_nvtx_rangerCZs=   (3  ! sA#A%cpeZdZUdZej ed<ejed<ede fdZ de fdZ y) ArrayInfoz4Container for info for checking array compatibility.rrrHc>||j|jS)NrrrG)clsrHs r@ from_arrayzArrayInfo.from_arrayms#))44rBc|j|jk7r%td|jd|j|j|jk7r%td|jd|jy)NzShape mismatch in backprop. Y: z, dY: zType mismatch in backprop. Y: )rr6r)r rHs r@check_consistencyzArrayInfo.check_consistencyqss 99 "1$**VCII;O  99 "0 F399+N  #rBN) rrrrr&Shape__annotations__DTypes classmethodr(rIrKrLrBr@rErEfs>> ;; <<5W55WrBrE)rIrArVrFrDrrrrrrrrrrr,rr6rCrErr)r.r/)rrK)r.N)FN)FF)F)r)r)o contextlibrr!r3rrPr0r8 contextvarsr dataclassesrtypingrrrrr r r r r rrrEconfection.validationrrrpackaging.versionrwasabircompatrrrrrrrrrrr rqr!rlr"r#boolrMr%r&r'r(r)r*r+r,rr-rArIrMrrVrZrFrDrardrgrfrmrortrxr}rrrrrfloatrrrrrrrrrrrrrrrr6rr r,contextmanagerr6r=rCrE__all__rLrBr@r^s  "!     HH%    $$./@%$PD!P HH$  3#3d3(5S5T5 st/S/T/?C?D?ICIDIST===DCDDD! 3 #5 Ts4D*G'd $%! % %}% %  %RIK Wffhw&7 78BE8K V ud{#3;SE3J3GNQ89# 59C9C9( '+ ^ $ 8=A. .*25/. .,JO'+BF67;33$,UO3 3./4  '+   ;?55(05 5( :  3= !*-9< c8m>*>&6 6sC.3469<6AD6 6@  "" C3  , rB