I have a csv file containing multiple fields and from that I am only adding the first_name & last_name and making unique id using uuid to the vectorstore. I want to add one more field from my csv to this vector only.
This is my code which I wrote in order to make the vector.
def proc_f(df_chunk, faiss_module, chunk_idx):
faiss_mod = FAISSModule()
df_chunk.fillna("NA", inplace=True)
df_chunk['vectortext'] = df_chunk['first_name'] + df_chunk['last_name']
df_chunk['_id'] = df_chunk.apply(
lambda x: base64.b64encode(str.encode(x['vectortext'])).decode(), axis=1)
df_chunk['doctext'] = [
Document(page_content=vectext, metadata={'id': _id})
for vectext, _id in zip(df_chunk['vectortext'], df_chunk['_id'])
]
documents = df_chunk['doctext'].values.tolist()
faiss_mod.add_documents(documents)
faiss_mod.save_index(f".//faiss_chunk_{chunk_idx}.index")
print(f"Chunk {chunk_idx} processed and FAISS index saved.")
return df_chunk
Is it possible to update the current vectorstore or I need to rerun the whole process?