最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

python - Add new field in existing FAISS vector store - Stack Overflow

programmeradmin5浏览0评论

I have a csv file containing multiple fields and from that I am only adding the first_name & last_name and making unique id using uuid to the vectorstore. I want to add one more field from my csv to this vector only.

This is my code which I wrote in order to make the vector.

def proc_f(df_chunk, faiss_module, chunk_idx):

    faiss_mod = FAISSModule()


    df_chunk.fillna("NA", inplace=True)
    df_chunk['vectortext'] = df_chunk['first_name'] + df_chunk['last_name']

    df_chunk['_id'] = df_chunk.apply(
        lambda x: base64.b64encode(str.encode(x['vectortext'])).decode(), axis=1)

    df_chunk['doctext'] = [
        Document(page_content=vectext, metadata={'id': _id})
        for vectext, _id in zip(df_chunk['vectortext'], df_chunk['_id'])
    ]

    documents = df_chunk['doctext'].values.tolist()
    faiss_mod.add_documents(documents)

    faiss_mod.save_index(f".//faiss_chunk_{chunk_idx}.index")
    print(f"Chunk {chunk_idx} processed and FAISS index saved.")

    return df_chunk

Is it possible to update the current vectorstore or I need to rerun the whole process?

发布评论

评论列表(0)

  1. 暂无评论