I am sending 11-byte data with random value from raspberry pi to the computer. I encoded the data with Start Byte ($) and End Byte (#). And when I try to receive and parse the data from my computer using C# application the length of sensor data is varying for every read. I try to parse and read individual data from the packet received but due to varying data length I can't be able to read the data correctly.

Raspberry Pi Code

import serial
from time import sleep
import random
import struct
port = serial.Serial("/dev/ttyUSB0", baudrate=115200, timeout=3.0)

while True:
    num = random.randrange(0,101)
    port.write(chr(36).encode()) # $ - Head
    port.write(chr(79).encode()) #
    port.write(chr(10).encode()) #Data0 
    port.write(chr(20).encode()) #Data1
    port.write(struct.pack('B', num)) #Data2 
    port.write(chr(127).encode()) #Data1 
    port.write(chr(75).encode()) #Data4
    port.write(chr(30).encode()) #Data5
    port.write(chr(20).encode()) #Data6
    port.write(chr(10).encode()) #Data7
    port.write(chr(35).encode()) # # - Tail
    sleep(5) 

C# Data Receiving Code

String s;
void SerialOnReceivedHandler(object sender, SerialDataReceivedEventArgs e)
{     
        
    String str = Serial.ReadExisting();

    s = Encoding.GetEncoding("ISO-8859-1").GetString(str);
    Console.WriteLine("String Length" + s.Length);
    if ((s.Length == 11) && (s.StartsWith("$")) && (s.EndsWith("#")))
    {
        Console.WriteLine("String Data" + s); 
    }
    else
    {
        Console.WriteLine("No Data Received........");
    }

Sometimes it reads 11-byte data correctly but most of the times it show No Data Received. The data length received is keep on varying.

I am sending 11-byte data with random value from raspberry pi to the computer. I encoded the data with Start Byte ($) and End Byte (#). And when I try to receive and parse the data from my computer using C# application the length of sensor data is varying for every read. I try to parse and read individual data from the packet received but due to varying data length I can't be able to read the data correctly.

Raspberry Pi Code

import serial
from time import sleep
import random
import struct
port = serial.Serial("/dev/ttyUSB0", baudrate=115200, timeout=3.0)

while True:
    num = random.randrange(0,101)
    port.write(chr(36).encode()) # $ - Head
    port.write(chr(79).encode()) #
    port.write(chr(10).encode()) #Data0 
    port.write(chr(20).encode()) #Data1
    port.write(struct.pack('B', num)) #Data2 
    port.write(chr(127).encode()) #Data1 
    port.write(chr(75).encode()) #Data4
    port.write(chr(30).encode()) #Data5
    port.write(chr(20).encode()) #Data6
    port.write(chr(10).encode()) #Data7
    port.write(chr(35).encode()) # # - Tail
    sleep(5) 

C# Data Receiving Code

String s;
void SerialOnReceivedHandler(object sender, SerialDataReceivedEventArgs e)
{     
        
    String str = Serial.ReadExisting();

    s = Encoding.GetEncoding("ISO-8859-1").GetString(str);
    Console.WriteLine("String Length" + s.Length);
    if ((s.Length == 11) && (s.StartsWith("$")) && (s.EndsWith("#")))
    {
        Console.WriteLine("String Data" + s); 
    }
    else
    {
        Console.WriteLine("No Data Received........");
    }

Sometimes it reads 11-byte data correctly but most of the times it show No Data Received. The data length received is keep on varying.

• c#
• python-3.x
• raspberry-pi
• serial-port

• 1 Just curious: There are dedicated codes for message boundaries (STX, ETX, ... ) why not use them? – Fildor Commented Mar 17 at 7:10
• As for stable message length: I would guess it is caused by port.write(struct.pack('B', num)) #Data2 . You'll probably need to align that data point to whatever the biggest value is you are expecting / that the API documentation is specifying. (Just a guess because I am not versed in python, this might as well not be the cause) – Fildor Commented Mar 17 at 7:13
• 1 Also: Never expect to receive the complete message in one Read operation. (A very common mistake) Accumulate data, until it contains a complete message, then take that from your buffer. Maybe consider flushing after the message is complete on sender side, too. – Fildor Commented Mar 17 at 7:17
• Another one: If you cannot avoid a non-static length in one of the fields in your message, you need to accomodate for that. Either send along the length (in a constant length field itself, of course, that precedes the variable-length field) or sub-delimiters. – Fildor Commented Mar 17 at 7:22
• 1 Serial port standards only determine the electrical and physical aspects, and do not define software protocols. There is no concept of packets like in TCP/IP, and reading will end regardless of the number of bytes depending on the conditions at the time, and this will often not match the number of bytes on the sending device. You will need to add multiple received data pieces to a buffer, and then check whether the data matches your protocol and extract it. Fundamentals of RS-232... – kunif Commented Mar 17 at 7:23

1 Answer 1

I figured, my comments actually should be an answer, so here we go:

A few things to address here, I'll go through them one by one.

Serial I/O is not message-based

It is stream-based. That is: what you read is a stream of bytes.

This implies that you cannot expect to read complete messages according to some application specific protocol. You need to make sure you have complete messages by explicitly coding to your protocol.

Usually this is done by reading into an accumulating buffer until one (or more) complete messages are detected. If and when so, the message is to be cropped out and passed on while the buffer is cleaned (not too much, though) and then back to reading again.

BTW: You could increase the probability to reading a full message in one go by flushing the sender after the message is complete.

Constant-Length vs Variable-Length Message fields

If a field in the message can have a variable number of bytes, you can deal with it in several different ways:

• If the difference is small (like a few bytes) you can "align" the data to the longest possible value specified in the protocol.

• If the variation is significantly large enough, you may want to either have a second field that ecodes the length of the variable-length field or introduce sub-delimiters.

Delimiter bytes

You are using "$" and "#". This is not inherently wrong, but I'd like to make you aware that there are message delimiters in ASCII (which extends to unicode and also iso-8859-1) already. If that protocol is being written by you, you can take advantage of those without giving up the "$" and "#" characters as content.

Message length variance in your case

The behavior you are seeing is either probably caused by point one of this answer , or - but due to my lack of python expertise I may be wrong here - it may be that the line

port.write(struct.pack('B', num)) #Data2 

introduces different sizes of that field. I did not check Python docs, though.

EDIT: I checked - should be an unsigned integer with constant size. See https://docs.python./3/library/struct.html