i4ddlZddlmZddlmZddlZddlZddlmZddl m Z ddl m Z ddl mZmZd d lmZmZmZmZmZmZmZej0d ed d ed dded deddddeddddeddddedddd ed!d"d#d$d%edd&d'd(d)ed*d+d,d-f d.ed/ed0ed1ed2ed3ed4eed5ed6eed7efd8Zd9e d6ed:dfd;Zej0d9ddd<d=ed d>dd?ed d@eddAdBdCed!d"d#d$d%eddDdEdFfdGej<dHedIedJeed5edKef dLZej0dMddd<Ned d>dd?ed dOeddAdBdCed!d"d#d$d%eddDdEdFfdGej<dHedIedJeed5edKef dPZ dQZ!y)RN)Path)Optional)msg)util)Language)convert_vectorsinit_nlp)ArgOpt import_codeinit_cliparse_config_overrides setup_gpushow_validation_errorvectors.z(The language of the nlp object to create)helpzVectors file in Word2Vec formatT)rexistszPipeline output directoryz--prunez-pz&Optional number of vectors to prune toz --truncatez-tzFOptional number of vectors to truncate to when reading in vectors filedefaultz--modez-mzVectors mode: default or floretz--namez-nz?Optional name for the word vectors, e.g. en_core_web_lg.vectorsFz --verbosez-Vz-VVz/Display more information for debugging purposesz--lexemes-jsonlz-jz+Location of JSONL-formatted attributes file)rhiddenORTHz--attrz-az?Optional token attribute to use for vectors, e.g. LOWER or NORMlang vectors_loc output_dirprunetruncatemodenameverbose jsonl_locattrc |r-tjjtjt j d|dtj|} | t| |t| |||||| t jdt| jjd| j|t jd|jy)zConvert word vectors for use with spaCy. Will export an nlp object that you can use in the [initialize] block of your config to initialize a model with vectors. z(Creating blank nlp object for language ''N)rrr rr#zSuccessfully converted z vectorszSaved nlp object with vectors to output directory. You can now use the path to it in your config as the 'vectors' setting in [initialize].)rloggersetLevelloggingDEBUGrinfoget_lang_classupdate_lexemesr goodlenvocabrto_diskresolve) rrrrrrr r!r"r#nlps h/var/www/vps2.regionflexible.com/Desarrollo/venv/lib/python3.12/site-packages/spacy/cli/init_pipeline.pyinit_vectors_clir4s` W]]+HH7vQ ?@ #$  d # %CsI&    HH&s399+<+<'=&>h GHKK HH Nr2returnctj|}|D]+}d|vr|j|d}|jdi|-y)Nsettingsorth)srsly read_jsonlr/ set_attrs)r2r" lex_attrsattrslexemes r3r,r,_sQ  +I"   5=)!5! "r5)allow_extra_argsignore_unknown_options)context_settingsrzPath to config file)rr allow_dashz&Output directory for the prepared dataz--codez-czNPath to Python file with additional code (registered functions) to be importedz--gpu-idz-gzGPU ID or -1 for CPUctx config_path output_path code_pathuse_gpuc|r-tjjtjt |j }t|t|t|5tj||}dddtd5t|}dddj|tjd|y#1swYUxYw#1swY?xYw)N overridesF hint_fillrIzSaved initialized pipeline to )rr&r'r(r)rargsrrr load_configr r0rr- rErFrGrHr!rIrLconfigr2s r3init_pipeline_clirTis8 W]]+&sxx0I  g { +D!!+CD  /0vw/0KK HH-k] ;< DD00s&CCCC'labels)rCzOutput directory for the labelsc|r-tjjtj|j s|j dt|j}t|t|t|5tj||}dddtd5t|}dddt|y#1swY8xYw#1swY"xYw)zGenerate JSON files for the labels in the data. This helps speed up the training process, since spaCy won't have to preprocess the data to extract the labels.T)parentsrKNFrMrO)rr&r'r(r)rmkdirrrPrrrrQr _init_labelsrRs r3init_labels_clirZs< W]]+    $'&sxx0I  g { +D!!+CD  /0vw/0k" DD00sC4C#C #C,c|jD]o\}}t|ddD||dz }tj||jt j d|d|Wt jd|dqy)N label_dataz.jsonz!Saving label data for component 'z' to z#No label data found for component 'r%)pipelinegetattrr; write_jsonr\rr-r*)r2rGr component output_files r3rYrYs<<Di 9lD 1 =%4&6K   [)*>*> ? HH8eK=Q R HH:4&B C Dr5)"r(pathlibrtypingrr;typerwasabirrlanguagertraining.initializer r _utilr r rrrrrcommandstrintboolr4r,ContextrTrZrYr:r5r3rosD ;)CHIC&GPTU3%@A It"J  U  Ix4UV  N     > !$  : ! N GD DD D  DD D 3-D*+D8~9DF GDDN""T"d" *.$O  'C&NO #  ] !    > r:t2HI)!= !=!=  !=~!=!=()!=  !=H *.$O 'C&GH #  ] !    > r:t2HI)%# %#%#  %#~%#%#()%# %#PDr5