I want to make multiple HTTP requests in only one connection with using concurrency, but my code makes about four or five. But when I don't use concurrency, it will only have one connection.

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "sync"
    "time"
)

func main() {
    proxyURL, err := url.Parse("http://localhost:9099")//In the traffic sniffer tool I see this program makes 5 requests
    if err != nil {
        fmt.Println(err)
        return
    }
    
    var wg sync.WaitGroup
    client := &http.Client{
        Transport: &http.Transport{
            MaxIdleConns:        10,
            IdleConnTimeout:     90 * time.Second,
            DisableKeepAlives:   false,
            Proxy: http.ProxyURL(proxyURL),
        },}

    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(index int) {
            defer wg.Done()
            resp, err := client.Get(";)
            if err != nil {
                fmt.Println(err)
                return
            }
            err = resp.Body.Close()
            if err != nil {
                fmt.Println(err)
                return 
            }
            fmt.Println(index)
        }(i)
    }

    wg.Wait()
    fmt.Println("done")
}

Edit: Thanks to you, but I found another alternative solution: just establish a connecition before running goroutine, it's faster than setting "MaxConnsPerHost: 1" when there are lot of requests

I want to make multiple HTTP requests in only one connection with using concurrency, but my code makes about four or five. But when I don't use concurrency, it will only have one connection.

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "sync"
    "time"
)

func main() {
    proxyURL, err := url.Parse("http://localhost:9099")//In the traffic sniffer tool I see this program makes 5 requests
    if err != nil {
        fmt.Println(err)
        return
    }
    
    var wg sync.WaitGroup
    client := &http.Client{
        Transport: &http.Transport{
            MaxIdleConns:        10,
            IdleConnTimeout:     90 * time.Second,
            DisableKeepAlives:   false,
            Proxy: http.ProxyURL(proxyURL),
        },}

    for i := 0; i < 5; i++ {
        wg.Add(1)
        go func(index int) {
            defer wg.Done()
            resp, err := client.Get("https://www.httpbin./get")
            if err != nil {
                fmt.Println(err)
                return
            }
            err = resp.Body.Close()
            if err != nil {
                fmt.Println(err)
                return 
            }
            fmt.Println(index)
        }(i)
    }

    wg.Wait()
    fmt.Println("done")
}

Edit: Thanks to you, but I found another alternative solution: just establish a connecition before running goroutine, it's faster than setting "MaxConnsPerHost: 1" when there are lot of requests

• go
• http

• go fmt.Println(index) ? – Николай Лубышев Commented Jan 19 at 7:52
• If you want the requests to be concurrent, then you will have multiple connections. You can’t send multiple requests at the same time over the same connection. You are also not reading the response, and if the body isn’t drained the connection can’t be reused – Mr_Pink Commented Jan 20 at 19:12

2 Answers 2

You can consider using MaxConnsPerHost

// MaxConnsPerHost optionally limits the total number of
// connections per host, including connections in the dialing,
// active, and idle states. On limit violation, dials will block.
//
// Zero means no limit.
MaxConnsPerHost int

usage

Transport: &http.Transport{
   MaxConnsPerHost: 1,
   ..rest of config...
}

A note: if the remote server doesn't support keep-alive, you might not reuse the same connection but open a new one every time

`Body 'is still open until the function is finished:

defer resp.Body.Close()

instead of try:

resp.Body.Close()