in this code:

if __name__ == "__main__":
    soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    soc.bind(("127.0.0.1", 4000))

    while True:
        soc.listen(5)
        conn, address = soc.accept()

        data = conn.recv(32)
        print(f"got data: {data}")
        conn.sendall(response.encode())
        conn.shutdown(socket.SHUT_WR)

when replacing shutdown with close and try use curl

curl http://localhost:4000

response:

curl: (56) Recv failure: Connection was reset

but using shutdown normally works fine, what I also noticed use close + time.sleep or print after it, it will work fine

what is the reason for such behavior? I understand shutdown is half-close, meaning it can receive more data from connection

but new curl means new connection, right?

details

1- socket start 2- send curl 3- connection.close 4- send new curl 5- got connection reset

in this code:

if __name__ == "__main__":
    soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    soc.bind(("127.0.0.1", 4000))

    while True:
        soc.listen(5)
        conn, address = soc.accept()

        data = conn.recv(32)
        print(f"got data: {data}")
        conn.sendall(response.encode())
        conn.shutdown(socket.SHUT_WR)

when replacing shutdown with close and try use curl

curl http://localhost:4000

response:

curl: (56) Recv failure: Connection was reset

but using shutdown normally works fine, what I also noticed use close + time.sleep or print after it, it will work fine

what is the reason for such behavior? I understand shutdown is half-close, meaning it can receive more data from connection

but new curl means new connection, right?

details

1- socket start 2- send curl 3- connection.close 4- send new curl 5- got connection reset

• sockets

2 Answers 2

Connection reset is triggered if close is called even though not all data are read yet. In your case the server only reads 32 bytes from the client, even though the request is likely larger.

When using shutdown or deferring the close (with sleep or print) it is likely that the client has read the response and has already exit, so a reset would have no visible effect.

I'll refer to your code as the server and curl as the client.

As pointed out by @Steffen Ullrich, the curl: (56) Recv failure: Connection was reset message looks as though it is a result of your server code not fully reading its socket's receive buffer before closing the socket connection. This results in a TCP RST,ACK being sent to the curl client. This is the correct behaviour according to Section 4.2.2.13 of RFC 1122 [Credit to Berth Hubert].

When I send the same curl request, I see the received data is 77 bytes in length, so if you are reading only 32 bytes, there are still 45 bytes to be read by your server before closing the connection.

Increasing the receive buffer size to 77 or more:

data = conn.recv(77)

would prevent the client generating the curl: (56) Recv failure: Connection was reset message.

As you mentioned, shutting down the write-side of the server's socket connection with:

conn.shutdown(socket.SHUT_WR)

also prevents the client generating this error response. This tells the client I have nothing else to write, you can shutdown. The client exits but your server still generates the TCP RST,ACK. You don't see the error message because the curl client process has exited before receiving it.

If you shutdown() the write side of the connection on your server and immediately close() the connection like this:

conn.shutdown(socket.SHUT_WR)
conn.close()

you could end up with a race condition between whether the curl client exits first or the server closes the connection.

If you add a sleep() between the shutdown() and the close():

conn.shutdown(socket.SHUT_WR)
time.sleep()
conn.close()

it is highly likely that the curl client will exit before the server closes the connection and again you won't see the error.

The best way to handle the situation would be:

# Shutdown thw write side of the socket
conn.shutdown(socket.SHUT_WR)
# Read anything remaining in the read buffer
while True:
    try:
        buffer = conn.recv(32, socket.MSG_DONTWAIT)
        print("Socket: Emptying read buffer")
        print("Socket: Reading {} bytes".format(len(buffer)))
        if len(buffer) == 0:
            break
    except BlockingIOError:
        print("Blocking error")
        break
# Close the connection
conn.close()

Generally, a http server doesn't need to close a connection and could expect the client to do it.

You can use the following code to experiment with different approaches:

#!/user/env/python

import os
import socket
import time

response = """HTTP/1.1 200 OK\r\n
Content-Length: 50\r\n
Content-Type: text/html\r\n
Connection: keep-alive\r\n
Last-Modified: Tue, 19 Nov 2024 02:11:40 GMT\r\n
Accept-Ranges: bytes\r\n
Date: Tue, 19 Nov 2024 02:11:40 GMT\r\n
\r\n
ABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJ1\r\n"""

# Define processing options
send_shutdown = True
sleep = False
emptyreadbuffer = True
close = False

if __name__ == "__main__":
    try:
        print("socket: create()")
        soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        
        print("socket: bind({})".format("(127.0.0.1, 4000)"))
        soc.bind(("127.0.0.1", 4000))

        print("socket: listen(5)")
        soc.listen(5)

        while True:
            print("socket: accept()")
            conn, address = soc.accept()
            print("connection: {} address: {}".format(conn, address))

            print("Receiving data")
            data = conn.recv(32)
            print("Received data length: {}".format(len(data)))
            print("Data:\r\n{}".format(data))

            print("Sending response")
            conn.sendall(response.encode())
            print("Sent response length: {}".format(len(response)))
            print("Response:\r\n{}".format(response))
            if send_shutdown:
                print("Connection: shutdown(SHUT_WR)")
                conn.shutdown(socket.SHUT_WR)
            if sleep:
                print("sleep(): Zzzzz")
                time.sleep(1)
            if emptyreadbuffer:
                while True:
                    try:
                        buffer = conn.recv(32, socket.MSG_DONTWAIT)
                        print("Socket: Emptying read buffer")
                        print("Socket: Reading {} bytes".format(len(buffer)))
                        if len(buffer) == 0:
                            break
                    except BlockingIOError:
                        print("Blocking error")
                        break
            if close:
                print("Connection: close()")
                conn.close()
    except Exception as err:
        print("Error: {}".format(err))

If you sniff your network connection with something like Wireshark, you will see something like this when you do not flush the read buffer:

And you will see something like this when you do flush the read buffer: