I need to count the number of files of multiple files .csv located in a server (250 files x 30 folders) and I would like some advice on how to do this fast, because i see its taking me a lot of time doing this for each file.

After connecting with Paramiko to a sftp, my code looks like:

with sftp.open('file.csv', r) as f: 
    csv_file = csv.reader(f)
    sum(1 for _ in csv_file)

I have also tried to get the file and then read it directly from a local folder as a dataframe (this is faster, but still slow). I have also saw there are some post in which they break the file into chunks, but I am not sure how to implement that option with a file located in a server.