o)j ddlZddlZddlmZmZddlmZddlmZm Z m Z m Z ddl m Z ejeZeGddZe dfd eezd ed ed efd Zd ededed efdZy)N) dataclassfield)Path)chunkerembedderloaderstore)DEFAULT_COLLECTIONcdeZdZUeed<dZeed<dZeed<dZeed<e e Z e eed<y) IndexResult collectionrdocs_processed chunks_addedchunks_skipped)default_factoryerrorsN) __name__ __module__ __qualname__str__annotations__rintrrrlistr//var/www/vps2.regionflexible.com/rag/indexer.pyr r s8ONCL#NCd3FDI3rr Fsourcer forcereturnc  t|}|rtj|tj|}|st j d||S|D]M} tj|j}|s#t j d|jItj|}t|D cgc]\}} t|j|| } }} tt!|Dcgc]}|j|d|j"} }tj$|| ||| \} } |xj&dz c_|xj(| z c_|xj*| z c_t j-d|jd| d| d P|Scc} }wcc}w#t.$rO}|jd |}|j0j3|t j d |Yd}~d}~wwxYw) N)r z!No se encontraron documentos en: zSin chunks utilizables en: )r chunk_indexz Indexado 'z': z chunks nuevos, z omitidos.z: zError indexando )r r resetrloadloggerwarningrsplittextrrembed enumerate _chunk_idrangelenmetadataupsertrrrinfo Exceptionrappend)rr rresultdocsdocchunks embeddingsir(ids metadatasaddedskippedexcmsgs rindex_documentsr?s J /F  J ;;v D :6(CD 5 5]]388,F!