Sun V890: Forcing Full Duplex Under Solaris 10

Sun V890: Forcing Full Duplex Under Solaris 10

Abstract:

When a server boots, the network card will negotiate with the network switch how to communicate. Various settings such as speed and duplex are often part of the auto-negotiation protocol. Occasionally, a server will not negotiate with the network switch the highest possible throughput. The duplex setting may improperly negotiate to half, which could result in collisions and errors, but this can be resolved through manually setting the port characteristics.

Background:

The Sun V890 offers various ethernet ports out-of-the box when purchased from Sun. Some of these port types include: ge, eri, qfe. To see which ports are installed on a platform, one can use the "dladm show-dev" command.

Symptoms:

Bad negotiation is often determined by poor performance.

sunv890/user$ netstat -in
Name Mtu  Net/Dest      Address       Ipkts Ierrs Opkts Oerrs Collis Queue
lo0  8232 127.0.0.0     127.0.0.1      8854     0 8854    0   0      0
qfe0 1500 192.127.224.0 192.127.224.33 6258     0 1460    0   0      0
qfe1 1500 172.39.0.0    172.39.5.33    73075033 0 1774172 0   0      0
qfe2 1500 192.127.254.0 192.127.254.9  10190603 0 6153115 176 546372 0 

Determining Bad Negotiation:

If a network switch negotiates incorrectly, the results are clear with Solaris 10.

sunv890/root# dladm show-dev
ge0  link: down speed: 0   Mbps duplex: unknown
eri0 link: down speed: 0   Mbps duplex: unknown
qfe0 link: up   speed: 100 Mbps duplex: full
qfe1 link: up   speed: 100 Mbps duplex: full
qfe2 link: up   speed: 100 Mbps duplex: half
qfe3 link: down speed: 0   Mbps duplex: unknown

Correcting Bad Negotiation:

The duplex can be fixed dynamically. To correct instance "2" of the "qfe" interface:

sunv890/root# ndd -set /dev/qfe instance 2
sunv890/root# ndd -set /dev/qfe adv_100T4_cap 0
sunv890/root# ndd -set /dev/qfe adv_100fdx_cap 1
sunv890/root# ndd -set /dev/qfe adv_100hdx_cap 0
sunv890/root# ndd -set /dev/qfe adv_10fdx_cap 0
sunv890/root# ndd -set /dev/qfe adv_10hdx_cap 0
sunv890/root# ndd -set /dev/qfe adv_autoneg_cap 0

Verifying Corrected Negotiation:

Duplex correction can be easily verified.

sunv890/root# dladm show-dev
ge0  link: down speed: 0   Mbps duplex: unknown
eri0 link: down speed: 0   Mbps duplex: unknown
qfe0 link: up   speed: 100 Mbps duplex: full
qfe1 link: up   speed: 100 Mbps duplex: full
qfe2 link: up   speed: 100 Mbps duplex: full
qfe3 link: down speed: 0   Mbps duplex: unknown

Other Interfaces:

If the problem was experienced on instance "0" of the "eri" interface, the following commands could be used to dynamically adjust the interface to 100 Mbs and full duplex.

sunv890/root# ndd -set /dev/eri instance 0
sunv890/root# ndd -set /dev/eri adv_100T4_cap 0
sunv890/root# ndd -set /dev/eri adv_100fdx_cap 1
sunv890/root# ndd -set /dev/eri adv_100hdx_cap 0
sunv890/root# ndd -set /dev/eri adv_10fdx_cap 0
sunv890/root# ndd -set /dev/eri adv_10hdx_cap 0
sunv890/root# ndd -set /dev/eri adv_autoneg_cap 0

If the problem was experienced on instance "0" of the "ge" interface, the following commands could be used to dynamically adjust the interface to 1000 Mbs and full duplex with 802.3 transmit and receive flow control.

sunv890/root# ndd -set /dev/ge instance 0
sunv890/root# ndd -set /dev/ge adv_1000fdx_cap 1
sunv890/root# ndd -set /dev/ge adv_1000hdx_cap 0
sunv890/root# ndd -set /dev/ge adv_pauseTX 1
sunv890/root# ndd -set /dev/ge adv_pauseRX 1
sunv890/root# ndd -set /dev/ge adv_1000autoneg_cap 0

Kernel Inquiry:

The configuration and statistics regarding the network interfaces can also be retrieved through a kernel inquiry. This can be used for operating systems earlier than Solaris 10.

sunv890/root# kstat qfe
module: qfe                   instance: 0
name:   qfe0                  class:    net
                                  
        align_errors          0
        allocbfail            0
        babble                0
        brdcstrcv             36739
        brdcstxmt             493
        buff                  0
        carrier_errors        0
...

Solaris 9: Missing dladm show-dev

Solaris 9: Missing dladm show-dev

Abstract:
Solaris 10 has included a new feature referred to as the Data Link Admin tool. This tool provides a simple way to configure and check the status of the layer 2 ethernet interfaces. Some of the information commonly used in dladm under Solaris 10 can be derived in Solaris 9.

Solaris 10: dladm show-dev
The Data Link Administration tool under Solaris 10 has some very nice features, including quickly seeing the interface name, speed, and duplex.

sunt2000# dladm show-dev
ipge0 link: unknown speed: 100 Mbps duplex: full
ipge1 link: unknown speed: 100 Mbps duplex: full
ipge2 link: unknown speed: 100 Mbps duplex: half
ipge3 link: unknown speed: 0 Mbps duplex: unknown

Solaris 9: kstat & nawk
A simple nawk script can be used on a Solaris 9 platform, to perform similar output.

sunt2000# kstat -p | nawk '/duplex/ || /speed/ { split($1,Array,":") ; Dev=Array[3] } /link_duplex/ && $2=="2" { Duplex[Dev]="full" } /link_duplex/ && $2=="1" { Duplex[Dev]="half" } /link_speed/ { if ( Duplex[Dev] == "" ) Duplex[Dev]="unknown" ; Speed[Dev]=$2 ; print Dev "\tlink: unknown\tspeed: " Speed[Dev] "\tMbit\tduplex: " Duplex[Dev] }'
ce0 link: unknown speed: 100 Mbit duplex: full
ce1 link: unknown speed: 1000 Mbit duplex: full
ce2 link: unknown speed: 1000 Mbit duplex: full
ce3 link: unknown speed: 1000 Mbit duplex: full
ce4 link: unknown speed: 0 Mbit duplex: unknown
ce5 link: unknown speed: 0 Mbit duplex: unknown