I have a problem with stopping/starting VF RX/TX queues from our DPDK application. I am using a patched DPDK 22.11.1 and RedHat 4.14 kernel.

Traffic flows correctly until the first time we stop/start the VF RX/TX queues. But after we restart RX/TX queues traffic stops flowing from the NIC to the application, I don't see any DPDK extended stats increment after that either. I am unable to understand what I am missing.

The PF is using an ice kernel driver, and the DPDK application is using one of the VFs on this PF.

sh-5.1# ethtool -i ens1f0
driver: ice
version: 5.14.0-284.66.1.rt14.351.el9_2.
firmware-version: 4.22 0x80019c88 1.3346.0
expansion-rom-version:
bus-info: 0000:10:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

These are the list of DPDK calls we make to IAVF PMD, and at step 13 we expect that packet flow resumes from the VF, but no traffic is received. I can see that none of the DPDK extended stats increase after restarting the queue.

 1. rte_eth_dev_configure
 2. rte_eth_rx_queue_setup(rx_deferred_start)
 3. rte_eth_tx_queue_setup(tx_deferred_start)
 4. rte_eth_dev_start
 5. rte_eth_dev_rx_queue_start
 6. rte_eth_dev_tx_queue_start
 7. **Traffic flows as normal**
 8. rte_eth_dev_rx_queue_stop
 9. rte_eth_dev_tx_queue_stop
10. **Traffic stops**
11. rte_eth_dev_rx_queue_start
12. rte_eth_dev_tx_queue_start
13. **No traffic**

UPDATE

I see the same behavior when using test-pmd as well, after initial port configuration and queue start packets are received correctly.

port stop all

port config all rxq 1
port config all txq 1
port config all rxd 1024
port config all txd 1024
port 0 rxq 0 deferred_start on
port 0 txq 0 deferred_start on
port 0 rxq 0 setup
port 0 txq 0 setup
port start 0

port 0 rxq 0 start
port 0 txq 0 start
clear port stats 0
show port stats 0

testpmd> show port stats 0
iavf_handle_virtchnl_msg(): adminq response is received, opcode = 15

  ######################## NIC statistics for port 0  ########################
  RX-packets: 58         RX-missed: 0          RX-bytes:  3972
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 0          TX-errors: 0          TX-bytes:  0

  Throughput (since last show)
  Rx-pps:           12          Rx-bps:         7224
  Tx-pps:            0          Tx-bps:            0
  ############################################################################

Then I stop/start the queues and I see rx_missed_errors start increasing and no packets are received in the application

port 0 rxq 0 stop
port 0 txq 0 stop

Wait 5 seconds

port 0 rxq 0 start
port 0 txq 0 start
clear port stats 0

testpmd> show port stats 0
iavf_handle_virtchnl_msg(): adminq response is received, opcode = 15

  ######################## NIC statistics for port 0  ########################
  RX-packets: 0          RX-missed: 38         RX-bytes:  2768
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 0          TX-errors: 0          TX-bytes:  0

  Throughput (since last show)
  Rx-pps:            0          Rx-bps:         7824
  Tx-pps:            0          Tx-bps:            0
  ############################################################################
testpmd> show port stats 0
iavf_handle_virtchnl_msg(): adminq response is received, opcode = 15

  ######################## NIC statistics for port 0  ########################
  RX-packets: 0          RX-missed: 8262       RX-bytes:  620741
  RX-errors: 0
  RX-nombuf:  0         
  TX-packets: 0          TX-errors: 0          TX-bytes:  0

  Throughput (since last show)
  Rx-pps:            0          Rx-bps:         9184
  Tx-pps:            0          Tx-bps:            0
  ############################################################################

Another update, with a newer kernel ice driver, the problem is not seen

sh-5.1# ethtool -i ens1f0
driver: ice
version: 5.14.0-427.50.1.el9_4.x86_64+rt
firmware-version: 4.22 0x80019c88 1.3346.0
expansion-rom-version: 
bus-info: 0000:10:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes