I have a problem: some operations (uploading a huge file by chunks) occupy pretty large chunks of RAM (2-3 mb each). During that uploading process, my app occasionally suffers from some freezes. Debugging has lead me to a conclusion, that my code isn't a problem. That freeze happens when literally no calculations are performed.
That chunks of data become unneeded one by one, so I might garbage-collect them when needed, without any freezes. But I haven't found any links about how to garbage-collect objects manually.
In C++ for example, manual garbage-collecting isn't a problem. In JavaScript there is an option for that case too.
Is there any opportunity to make Dart garbage-collector to remove a specific object from memory? Ideally, I need some api like VM.removeObjectFromHeap(myObjectInstance)
.
So, the code where I can see the freeze:
final createChunkFromBytesStart = DateTime.now();
final currentChunk = _createFileChunkFromBytesChunks(currentChunkIndex, isFileFinished: isFileFinished);
final createChunkFromBytesEnd = DateTime.now();
// 26-50 mks average. Sometimes a freeze occurs (> 10000 mks, while I see that whole function takes 20mks to execute)
myLogger.d(
'_createFileChunkFromBytesChunks time in mks, ${createChunkFromBytesEnd.difference(createChunkFromBytesStart).inMicroseconds}');
That _createFileChunkFromBytesChunks function:
Chunk _createFileChunkFromBytesChunks(int chunkIndex, {bool isFileFinished = false}) {
final functionStart = DateTime.now();
final totalChunksLength = utils.calculateTotalListsLength(_streamChunksToSend);
final extraBytesLength = chunkSizeBytes >= totalChunksLength ? 0 : totalChunksLength - chunkSizeBytes;
// так как handleChunkBytes вызывается последовательно после каждого чтения стрима, extraBytes гарантированно может быть только в последнем чанке.
late final List<int> extraBytes;
if (extraBytesLength != 0) {
extraBytes = _streamChunksToSend.last.sublistToLength(_streamChunksToSend.last.length - extraBytesLength);
_streamChunksToSend[_streamChunksToSend.length - 1] =
_streamChunksToSend.last.sublistToLength(0, _streamChunksToSend.last.length - extraBytesLength);
} else {
extraBytes = [];
}
final bytesChunkData = _streamChunksToSend;
_streamChunksToSend = extraBytes.isNotEmpty ? [extraBytes] : [];
final chunk =
Chunk(chunkIndex: chunkIndex, bytesData: bytesChunkData, isFinishChunk: isFileFinished, uploaderId: uploaderId);
final functionEnd = DateTime.now();
// 20 mks average, never jumps to 50+mks. So the source of that freeze is inbetween returning the value from this function, and receiving the function result
myLogger.d('_createFileChunkFromBytesChunks time in mks, ${functionEnd.difference(functionStart).inMicroseconds}');
return chunk;
}
As the source of that freeze is somewhere "between" my code, I suppose that it's VM that causes that freeze. No other code is executing while file uploading: no syncronizations, no serialization. Also, If some data serialization was the problem, then I think that the freeze would occur at every chunk (because the amount of data to serialize is the same).
The uploading is done via Stream. The code:
final formData = FormData.fromMap({
'videoType': videoType,
'fileId': _fileId,
'file': MultipartFile.fromStream(() => Stream.fromIterable(chunk.bytesData), chunk.chunkSize,
filename: 'file_${_fileId}_${chunk.chunkIndex}.${fileExt}'),
});
var localTotalChunks = chunk.chunkIndex + 10;
if (chunk.isFinishChunk) {
localTotalChunks = chunk.chunkIndex + 1;
}
var headers = {
'uploader-videotype': videoType,
'uploader-file-id': _fileId,
'uploader-chunks-total': localTotalChunks,
'uploader-chunk-number': chunk.chunkIndex,
'uploader-filename': 'file_${_fileId}_${chunk.chunkIndex}.${fileExt}',
};
final beforeSend = DateTime.now();
final result = await dioSendObject(url,
bodyData: formData,
connectTimeout: connectTimeout,
receiveTimeout: receiveTimeout,
extraParsing: true,
onSendProgress: (int count, int total) => chunk.bytesUploaded = count,
cancelToken: cancelToken,
options: Options(headers: headers));
final afterSend = DateTime.now();
myLogger.d('chunk send time ${afterSend.difference(beforeSend).inMilliseconds}');
chunk.bytesData
is List<List<int>>
.
In release, the freeze is minor, but still present (user can see it as occasional short fps drops from 60+ fps to 20fps)