Documente Academic
Documente Profesional
Documente Cultură
I/O Connectivity Options for the Dell PowerEdge M1000e Blade Enclosure
December 8, 2014
Send feedback to: trieu_phan@dell.com
Contents
Quick Reference Guides
- Ethernet switching
- Fibre Channel switching
- Cisco and Infiniband switching
Infiniband Blades
3
4
5
9
11
13
15
17
19
23
25
27
28
32
34
36
39
41
Fabric Topologies
44
51
55
Interoperability Guide
69
76
86
92
Change Revision
96
Dell Inc.
Converged Ethernet
Fibre Channel
MXL
PowerEdge M I/O Aggregator
M8024-k
M8428-k
10 Gb Pass-Through
Cisco B22DELL FEX
Brocade M6505
Brocade M5424
Pass Through FC8/4
1Gb Ethernet
InfiniBand
M6348
M6220
1Gb Pass-Through
Cisco Catalyst Blade
Mellanox M4001F
Mellanox M4001T
MXL
External Ports:
(2) 10/40GbE QSFP+
(2) Optional Modules
40Gb
1Gb
Performance / Bandwidth
10Gb
Cisco Nexus
B22DELL (FEX)
External Ports:
(8) SFP+ 10GbE
M8428-k
External Ports:
(8) SFP+ 10GbE
(4) 8Gb FC SAN
M8024-k
External Ports:
(4) SFP+ 1/10GbE
(1) Optional Module
SFP+, or Base-T
I/O Aggregator
External Ports:
(2) QSFP+ ports in
4x10GbE mode
(2) Optional Modules
QSFP+, SFP+, Base-T, or FC
FCF
Integration w/ ToR
FCoE Transit / FSB
IEEE 802.1Q DCB Ethernet (for iSCSI or FCoE)
Cisco 3130G/3130X
External Ports:
(4) RJ45 GbE
(2) Optional Modules
TwinGig
M6220
External Ports:
(4) RJ45 GbE
(2) Optional Modules
SFP+, CX4, Base-T, or stacking
M6348
External Ports:
(16) RJ45 GbE
(2) SFP+ 10GbE
(2) CX4 10GbE
Server Ports
7
16
16
32
Converged
Ethernet
10/40Gb
Switch
MXL
8
10Gb
Plug & Play
M-IOA
10Gb
Basic
M8024-k
Ethernet &
FC Switch
M8428-k
10Gb
Cisco
PassB22DELL FEX
Through
Converged
Two
FlexIO
Modules
4-port FC module
Stack up to 6 devices
VLT
PVST+ protocol for easy integration into Cisco environments
Converged
Native Fibre Channel uplinks with FC FlexIO module (FCoE on internal ports to the
servers)
FCoE transit to top of rack switch with IOM acting as a FIP Snooping Bridge
Two
QSFP+
ports
Converged
Adapters
11G
-Broadcom 57712-k
-Brocade BR1741M-k
-Intel X520-x/k
-QLogic QME8242-k
Optical Transceivers
SFP+ 10Gb: SR, LR
SFP 1GbE: SX, LX
SFP to RJ45 converter
1000Base-T (only capable of 1Gbps)
12G
-Broadcom 57810S-k
-Broadcom 57840S-k
-Brocade BR1741M-k
-Intel X520-x/k
-Qlogic QME8262-k
More details in
Adapter Portfolio section
4 port FC Module
2 port QSFP+
Module
A2
A1
B2
B1
C2
C1
B1/B2
C1/C2
10
A1/A2
Optical Transceivers
SFP+ 8Gbps (will connect at 4/2 Gbps)
SW 150m, LW 4km
Optical Transceivers
SFP+ 40Gb: SR only
QSFP+ to QSFP+
Fiber Cables
QSFP+ to 4xSFP+ Fiber
Breakout Cables
Converged
Easy Deployment
DCB and FCoE settings detected from top of rack switch through DCBx protocol
(Note: CMC GUI will not function if the IOA is stacked. IOA must be managed through CLI when stacked.
Maximum stacking capability is 2)
Two
FlexIO
Modules
2-port 40GbE QSFP+ module (converts to 8-port 10GbE SFP+ using breakout cables)
4-port 10GbE SFP+ module
(If running Base-T module then second IO slot must be of different type due to power
constraints)
4-port FC module
Converged
Native Fibre Channel uplinks with FC FlexIO module (FCoE on internal ports to the servers)
FCoE transit to top of rack switch with IOM acting as a FIP Snooping Bridge
11
Two
QSFP+
ports
Adapters
11G
-Broadcom 57712-k
-Brocade BR1741M-k
-Intel X520-x/k
-QLogic QME8242-k
Optical Transceivers
SFP+ 10Gb: SR, LR
SFP 1GbE: SX, LX
SFP to RJ45 converter
1000Base-T (only capable of 1Gbps)
12G
-Broadcom 57810S-k
-Broadcom 57840S-k
-Brocade BR1741M-k
-Intel X520-x/k
-Qlogic QME8262-k
More details in
Adapter Portfolio section
4 port FC Module
2 port QSFP+
Module
A2
B2
B1
C2
C1
C1/C2
A1
B1/B2
A1/A2
Optical Transceivers
SFP+ 8Gbps (will connect at 4/2 Gbps)
SW 150m, LW 4km
Optical Transceivers
SFP+ 40Gb: SR only
QSFP+ to QSFP+
Fiber Cables
QSFP+ to 4xSFP+ Fiber
Breakout Cables
M8024-k
Fully modular full wire-speed 10GbE managed
Layer 2/3 Ethernet switching
Converged
One
FlexIO
Module
2-port
10GBASE-T
13
Four
SFP/SFP+
ports
4-port
SFP+
M8024-k
Adapters
Converged
Uplinks
11G
-Broadcom 57712-k
-Brocade BR1741M-k
-Intel X520-x/k
-QLogic QME8242-k
Cables
RJ45 / Cat6a
Uplinks
12G
-Broadcom 57810S-k
-Brocade BR1741M-k
-Intel X520-x/k
-Qlogic QME8262-k
-Broadcom 57840S-k (will
only link with 2 ports)
The M8024-k switch supports
connectivity to 10Gb-KR adapters,
all of which are notated with -k.
It does not provide connectivity to
legacy 10Gb-XAUI NICs/CNAs
If connected to 1Gb Ethernet
Mezzanine cards or LOMs, device
will auto-negotiate individual
internal ports to 1Gb.
10GBASE-T
Copper
Module
(supports autonegotiation to
100Mb/1Gb)
10GbE SFP+
Module
(10Gb only)
SFP+ Direct Attach Cable (Twinax)
(0.5m, 1m, 3m, 5m, 7m available)
Can operate at 10Gb and 1Gb
More details in
Adapter Portfolio section
B1
C2
C1
B1/B2
A1
A1/A2
C1/C2
Secondary
Management
Serial Port
(Cable included)
4 external
SFP/SFP+ ports
(multi-speed 1/10Gb)
14
M8428-k
Converged
15
Pre-installed 8Gbps short-wave SFP+ optical transceivers enable quick and easy cableand-go connections
M8428-k
Converged
Adapters
11G
-Broadcom 57712-k
-Brocade BR1741M-k
-Intel X520-x/k
-QLogic QME8242-k
12G
-Broadcom 57810S-k
-Brocade BR1741M-k
-Intel X520-x/k
-Qlogic QME8262-k
-Broadcom 57840S-k (will
only link with 2 ports)
Supports connectivity to
10Gb-KR adapters, all of which
are notated with -k. It does
not provide connectivity to
legacy 10Gb-XAUI NICs/CNAs
Cables
Secondary
Management
Serial Port
More details in
Adapter Portfolio section
B1
C2
C1
B1/B2
A1
A1/A2
C1/C2
16
Cables
Converged
17
Converged
Adapters
11G
-Broadcom 57712-k
-Brocade BR1741M-k
-Intel X520-x/k
-QLogic QME8242-k
12G
-Broadcom 57810S-k
-Brocade BR1741M-k
-Intel X520-x/k
-Qlogic QME8262-k
-Broadcom 57840S-k (will
only link with 2 ports)
Supports connectivity to
10Gb-KR adapters, all of which
are notated with -k. It does
not provide connectivity to
legacy 10Gb-XAUI NICs/CNAs
1Gb Ethernet mezzanine cards
and LOMs are not supported.
More details in
Adapter Portfolio section
B1
C2
C1
B1/B2
A1
A1/A2
C1/C2
18
Cables
Converged
19
Converged
12G
-Broadcom 57810S-k
-Brocade BR1741M-k
-Intel X520-x/k
-Qlogic QME8262-k
-Broadcom 57840S-k (will
only link with 2 ports)
More details in
Adapter Portfolio section
B2
C2
C1
C1/C2
20
FET-10Gb Optic
(Distance up to 100m with OM3 fiber)
A FET is a new optic provided by Cisco. A FET can
only be used on FEX devices and Nexus switch
ports that connect to a FEX.
FET optics are sold with FEX at time of purchase.
You CANNOT purchase these optics separately
A2
B1
B1/B2
A1
A1/A2
The minimum Cisco Nexus software versions to support the B22DELL FEX are
-5.2(1)N1(3)
-6.0(2)N1(2)
Dell PowerEdge M
I/O Aggregator
Cisco Nexus
B22DELL FEX
Dell M8024-k
Dell M8428-k
10/40GbE Switch
10GbE Extender
10GbE Basic
Ethernet / FC
32 (10GbE)
32 (10GbE)
16 (10GbE)
16 (10GbE)
16 (10GbE)
2 Fixed 6 Total
None
None
None
Model
Overview
Server Ports Supported
2 Fixed 6 Total
None
None
No
Yes
Yes
1QSFP+
Stacking
n/a
(All traffic is forwarded
to Nexus Top-of-Rack
/ End-of-Row)
No
East-west traffic
support
Yes
Yes
Yes
Yes
Yes
Yes
(Enabled via CLI)
Yes
No
No
Yes
Yes
No
No
No
Not in a
Not in a
Not in a
redundant manner redundant manner redundant manner
Global Marketing
1Gb Ethernet
1/10Gb
High-density
M6348
22
1/10Gb
Basic
M6220
1Gb
PassThrough
Cisco
Catalyst
3130X & 3130G
M6348
1/10GbE
1/10GbE
M6348
Adapters
Works with all 1Gb Mezzanine
cards and LOMs. Optimal use is
with quad-port 1Gb adapters.
Cables
CAT 5
More details in
Adapter Portfolio section
B1
C2
C1
B1/B2
A1
A1/A2
C1/C2
24
Cables
CX4 Cables
for 10Gb uplinks or 32Gb M6348 stacking
(with other M6348 or rack-mount PC 7000 series switches)
(1m or 3m available)
Secondary
Management
Serial Port
(Cable included)
M6220
1/10GbE
25
4 x fixed
10/100/1000Mb
(RJ-45)
48Gb Stacking
Module
2 x 10Gb Optical
SFP+ Uplinks
2 x 10GBASE-T
Copper Uplinks
2 x 10Gb Copper
CX-4 Uplinks
1/10GbE
M6220
Adapters
Uplinks
Cables
Stacking Cable
(1m included;
3m available)
Uplinks
A2
A1
B2
B1
C2
C1
B1/B2
C1/C2
26
Uplinks
Cables
RJ45 / Cat6a
PowerConnect
6xxx SFP+ Module
Uplinks
Cables
10GBase-T
(Copper)
Uplink Module
CAT 5
Cables
SR & LR
Stacking Module,
48Gbps
A1/A2
Cables
CX4 Cable for
10GbE Uplink, 12m
Gb Ethernet Pass-Through
1GbE
Adapters
Cables
CAT 5
B1
C2
C1
B1/B2
A1
A1/A2
C1/C2
27
1/10GbE
Software
1/10GbE
B2
C2
C1
C1/C2
29
CAT5
Cable
Fibre
Two TwinGig
converters ship
by default in
each switch
module
A2
B1
B1/B2
CAT 5
Cables
Software Upgrades
IP Services Upgrade Available
(for 3130X)
A1
A1/A2
Secondary
Management
Serial Port
SFP+
10GBASE-SR
X2 Module
or
10GBASE-LRM
X2 Module
(3130X only)
3130X Modules
OneX SFP+
Converter
Module
CVR-X2-SFP10G
Not sold by Dell
purchase elsewhere
Copper
Fibre
Fibre Channel
16Gb
M6505
8Gb
M5424
8Gb
PassThrough
31
M5424
M6505
Model Choices
12-port, 24-port
24-port (Ent Perf Pk)
12-port, 24-port
24-port (Ent Perf Pk)
2 of 8 - 4 of 8 - 8 of 8
2 of 8 - 4 of 8 - 8 of 8
Qlogic &
Emulex - 8Gb & 4Gb
Qlogic &
Emulex - 16Gb & 8Gb
Brocade ISL-Trunking
(License option)
Optional
Available a-la-carte
Optional
Included
Not
Supported
Included
Brocade M6505
Fibre Channel
16Gb switch
16 Gb Fibre
Channel ports*
Secondary
Management
port
32
Dynamic Ports on Demand (PoD) and pay-as-yougrow port upgrades for 12-port configurations
Brocade M6505
Fibre Channel
16Gb switch
Adapters
11G
- Qlogic QME2572 FC8
- Emulex LPe1205-M FC8
12G
- Qlogic QME2662 FC16
- Emulex LPm16002 FC16
- Qlogic QME2572 FC8
- Emulex LPe1205-M FC8
Cables
Brocade Transceivers
Brocade SWL, LWL or ELWL 16Gb SFP+ Optics
Brocade SWL, LWL or ELWL 8Gb SFP+ Optics
Brocade SWL, LWL or ELWL 4Gb SFP+ Optics
Note: Requires SFP LC connector
Secondary
Management
Serial Port
More details in
Adapter Portfolio section
B2
B1
C2
C1
B1/B2
C1/C2
33
(16) internal and (8) SFP+ external FC16 ports with (8) FC16 SWL transceivers and Enterprise
Performance Pack
(16) internal and (8) SFP+ external FC16 ports with (4) FC16 SWL transceivers
(8) internal and (4) SFP+ external FC16 ports with (2) FC16 SWL transceivers
(12 port model expands to 24 ports with on-demand license)
Brocade M5424
8Gb switch
Fibre Channel
Brocade M5424
Fibre Channel
8Gb switch
Adapters
Cables
Brocade Transceivers
Brocade SWL or LWL 8Gb SFP+ Optics
Brocade SWL, LWL or ELWL 4Gb SFP+ Optics
11G
- Qlogic QME2572
- Emulex LPe1205
12G
- Qlogic QME2572
- Emulex LPe1205-M
- Qlogic QME2662 (at FC8
speeds)
- Emulex LPm16002 (at FC8
speeds)
FC4 Mezzanine cards are
also supported with this
switch at 4Gbps.
More details in
Adapter Portfolio section
B2
B1
C2
C1
B1/B2
C1/C2
35
Secondary
Management
Serial Port
(16) internal and (8) SFP+ external FC8 ports with (8) FC8 SWL transceivers and
Enterprise Performance Pack
(16) internal and (8) SFP+ external FC8 ports with (4) FC8 SWL transceivers
(8) internal and (4) SFP+ external FC8 ports with (2) FC8 SWL transceivers
(12 port model expands to 24 ports with on-demand license)
36
Fibre Channel
Fibre Channel
Adapters
11G
- Qlogic QME2572,
- Emulex LPe1205
12G
- Qlogic QME25722
- Emulex LPe1205-M
Cables
B2
B1
C2
C1
B1/B2
C1/C2
37
Brocade Transceivers
16 pre-installed 8Gbps SWL SFP+ transceivers
(one per port)
InfiniBand
56Gb
M4001F
FDR
38
40Gb
M4001T
FDR10
Mellanox Blades
Infiniband
Models
M4001F
M4001T
Speed
FDR
FDR10
56Gbps
40Gbps
Data rate
Total ports
39
Infiniband
Adapters
Cables
Cables
or
or
Not a Management
Port. Debug port only
B2
B1
C2
C1
B1/B2
Not a Management
Port. Debug port only
C1/C2
40
M4001F FDR
M4001T FDR10
Fabric
Topologies
41
FCoE transit
Topology Diagram
Topology / Configuration
Topology
Fabric Inside Chassis: FCoE
Blade models: MXL, IOA, M8024-k
Top-of-Rack switch: Dell S5000
as well as the Cisco Nexus 5000
Configuration
All FCoE traffic moves from the
adapters, to the IOM, then to
the Top-of-Rack switch
FC is broken out at the Top-ofRack switch and moves to the
SAN or directly to the storage
array
42
SAN A
LAN
LAG
SAN B
LAG
FCoE
LAN
FC
Inside Chassis
FCoE from
Adapter to
Blade Switch
Topology / Configuration
Topology
Fabric Inside Chassis: FCoE
Blade model: MXL, IOA, M8428-k
Top-of-Rack switch: Dell S5000,
S4810, S4820T
Configuration
FCoE inside chassis (from adapter
to blade switch) and Native FC
outside the chassis
LAN
LAN
SAN A
SAN B
LAG
LAG
LAN
FC
Inside Chassis
FCoE from
Adapter to
Blade Switch
43
Topology / Configuration
LAN
Topology
Fabric Inside Chassis: Converged iSCSI
Blade models: MXL or IOA
Top-of-Rack switch: S5000, S4810,
S4820T
Storage: iSCSI External Array
VLT
LAG
LAG
Configuration
Converged iSCSI traffic (LAN and
iSCSI) up to the Top-of-Rack switch
44
Inside Chassis
Topology / Configuration
Configuration
Converged iSCSI to the blades and
up to the Top-of-Rack switch
Blade IOMs are using VLT so that
array to array traffic can stay inside
the M1000e chassis
45
LAN
VLT
LAG
M1000e chassis
Topology
Fabric Inside Chassis: Converged iSCSI
Blade model: MXL, IOA
Top-of-Rack switch: S5000, S4810,
S4820T
Storage: PS4410 storage blade
Optional Storage: EqualLogic External
Array
VLT
iSCSI
& LAN
LAN
iSCSI
Dual Port
Adapter
Storage
Blade
Topology Diagram
VLT
Topology
Blade models: MXL, M8024-k, M6348,
M6248, IOA (using CLI)
Configuration
Blade switches are stacked vertically
so that there are two independent
stacking rings. Switches on the left
of the chassis form a ring and
switches on the right side of the
chassis form a ring. Independent
stack rings allow each ring to be
upgraded independently.
Note that IOA is limited to a two unit
stack. IOA has a simplified CLI
command for stacking and IOA
must be managed via CLI when
stacked.
46
Stack Ring 1
Stack Ring 2
LAN
Benefits of Stacking
Topology Diagram
Benefits of Stacking
Single point of management for each stack
Increase of East/West traffic so less traffic
goes to Top of Rack
Save on Top of Rack ports
Reduced Cables
Less Congestion at Top of Rack
VLT
Core
Core
Topology / Configuration
Topology
Stacked blade switches connected
directly to the Network Core switches
Stacked IOMs
Configuration
Stacked blade switches act as the
aggregation layer. No need for Top of
Rack switches.
47
Stacked IOMs
Stack Ring 1
Stack Ring 2
LAN
Automation &
Management
48
CMC features
Inventory of servers, I/O modules, & iDRAC cards
Perform configuration and monitoring tasks
Back up, clone settings and apply BIOS profiles
Remotely power on or off blades
Configure power and thermal settings
Receive email or alert notifications if errors arise
CMC software provides configuration of:
Network and security settings of the M1000e
Power redundancy & power ceiling settings
I/O switches and iDRAC network settings
First boot device on the server blades
User access security
49
FlexAddress Plus
FlexAddressassigned MACs
Reduce involvement of network admin in blade deployments by eliminating the need to understand
STP (Spanning Tree Protocol), VLANs (Virtual Local Area Networks), & LACP (Link Aggregation
Control Protocol) groups
For an overview demo of Simple Switch mode, visit:
http://www.delltechcenter.com/page/PowerEdge+Blade+Demos (English only)
51
Fabrics and
Port Mapping
52
Fabric A2
A1
A2
Fabric B1
1/10/40 GbE,
4/8/16Gb FC,
20/40/56Gb IB
Fabric C1
1/10/40 GbE,
4/8/16Gb FC,
20/40/56Gb IB
Reserved for
1/10GbE LOMs or
Select Network
Adapters
Fabric B2
B1
B2
C1
C2
1/10/40 GbE,
4/8/16Gb FC,
20/40/56Gb IB
Fabric C2
1/10/40 GbE,
4/8/16Gb FC,
20/40/56Gb IB
Colors chosen to
facilitate whiteboard
discussions.
The capabilities of the enhanced midplane (1.1) are shown above
53
C2
OR
B1 B2
C1 C2
Half
Height
B1 B2
C1 C2
B1
Full
Height
B2
C1 C2
B1 B2
54
Dell Inc.
Fabric A
LOM
2 lanes
2 lanes
Fabric B
4 lanes
4 lanes
Mezzanine
Fabric C
4 lanes
4 lanes
Front of Chassis
Mezzanine
Fabric A2
Ethernet
I/O Module
Fabric B2
I/O Module
Fabric C2
I/O Module
Fabric C1
I/O Module
Fabric A
LOM
2 lanes
2 lanes
Fabric B
4 lanes
4 lanes
Mezzanine
Fabric C
Mezzanine
4 lanes
4 lanes
8.4Tb Mid-plane
Fabric B1
I/O Module
Fabric A1
Ethernet
I/O Module
55
Fabric A Capabilities:
Fabric
& C Capabilities:
Up to B
4 lanes
to each IOM
CPU
1
Fabric A
LOM
8 lane
PCIe
Mezzanine
8 lane
PCIe
Fabric A2
Ethernet
I/O Module
Fabric B
Fabric B2
I/O Module
Fabric C
Mezzanine
Front of Chassis
CPU
2
4-8 lane
PCIe
M1000e Mid-plane
Fabric C2
I/O Module
Fabric C1
I/O Module
4-8 lane
PCIe
Fabric A
LOM
8 lane
PCIe
Fabric B
8 lane
PCIe
Fabric B1
I/O Module
Mezzanine
Fabric A1
Fabric C
Mezzanine
8.4Tb Mid-plane
Ethernet
I/O Module
56
Fabric A:
Fabric B & C:
Slot2
Slot3
Slot4
Slot5
Slot6
Slot7
Slot8
IOM1
P1
IOM1
P2
IOM1
P3
IOM1
P4
IOM1
P5
IOM1
P6
IOM1
P7
IOM1
P8
IOM2
P1
IOM2
P2
IOM2
P3
IOM2
P4
IOM2
P5
IOM2
P6
IOM2
P7
IOM2
P8
Slot9
Slot10
Slot11
Slot12
Slot13
Slot14
Slot15
Slot16
IOM1
P9
IOM1
P10
IOM1
P11
IOM1
P12
IOM1
P13
IOM1
P14
IOM1
P15
IOM1
P16
IOM2
P9
IOM2
P10
IOM2
P11
IOM2
P12
IOM2
P13
IOM2
P14
IOM2
P15
IOM2
P16
C2, B2, A2
IOM1
IOM2
Slot2
Slot3
Slot4
Slot5
Slot6
Slot7
Slot8
IOM1
P1
IOM1
P2
IOM1
P3
IOM1
P4
IOM1
P5
IOM1
P6
IOM1
P7
IOM1
P8
IOM2
P1
IOM2
P2
IOM2
P3
IOM2
P4
IOM2
P5
IOM2
P6
IOM2
P7
IOM2
P8
IOM1
P17
IOM1
P18
IOM1
P19
IOM1
P20
IOM1
P21
IOM1
P22
IOM1
P23
IOM1
P24
IOM2
P17
IOM2
P18
IOM2
P19
IOM2
P20
IOM2
P21
IOM2
P22
IOM2
P23
IOM2
P24
Slot9
Slot10
Slot11
Slot12
Slot13
Slot14
Slot15
Slot16
IOM1
P9
IOM1
P10
IOM1
P11
IOM1
P12
IOM1
P13
IOM1
P14
IOM1
P15
IOM1
P16
IOM2
P9
IOM2
P10
IOM2
P11
IOM2
P12
IOM2
P13
IOM2
P14
IOM2
P15
IOM2
P16
IOM1
P25
IOM1
P26
IOM1
P27
IOM1
P28
IOM1
P29
IOM1
P30
IOM1
P31
IOM1
P32
IOM2
P25
IOM2
P26
IOM2
P27
IOM2
P28
IOM2
P29
IOM2
P30
IOM2
P31
IOM2
P32
A1,B1,C1
C2, B2, A2
IOM1
IOM2
CPU
1
Fabric A
LOM
8 lane
PCIe
Mezzanine
8 lane
PCIe
Fabric A2
Ethernet
I/O Module
Fabric B
Fabric B2
I/O Module
Fabric C
Mezzanine
Front of Chassis
CPU
2
4-8 lane
PCIe
M1000e Mid-plane
Fabric C2
I/O Module
Fabric C1
I/O Module
CPU
3
CPU
4
4-8 lane
PCIe
Fabric A
LOM
8 lane
PCIe
Fabric B
8 lane
PCIe
Fabric B1
I/O Module
Mezzanine
Fabric A1
Fabric C
Mezzanine
8.4Tb Mid-plane
Ethernet
I/O Module
59
Fabric A:
Ethernet Only
Dual port 1Gb and 10Gb
Ethernet adapters
Quad port 1Gb Ethernet and
capable of quad port 10Gb
Ethernet adapters
Fabric B & C:
Slot2
Slot3
Slot4
Slot5
Slot6
Slot7
Slot8
IOM1
P1
IOM1
P2
IOM1
P3
IOM1
P4
IOM1
P5
IOM1
P6
IOM1
P7
IOM1
P8
IOM2
P1
IOM2
P2
IOM2
P3
IOM2
P4
IOM2
P5
IOM2
P6
IOM2
P7
IOM2
P8
IOM1
P9
IOM1
P10
IOM1
P11
IOM1
P12
IOM1
P13
IOM1
P14
IOM1
P15
IOM1
P16
IOM2
P9
IOM2
P10
IOM2
P11
IOM2
P12
IOM2
P13
IOM2
P14
IOM2
P15
IOM2
P16
A1,B1,C1
C2, B2, A2
IOM1
IOM2
Slot2
Slot3
Slot4
Slot5
Slot6
Slot7
Slot8
IOM1
P1
IOM1
P2
IOM1
P3
IOM1
P4
IOM1
P5
IOM1
P6
IOM1
P7
IOM1
P8
IOM2
P1
IOM2
P2
IOM2
P3
IOM2
P4
IOM2
P5
IOM2
P6
IOM2
P7
IOM2
P8
IOM1
P17
IOM1
P18
IOM1
P19
IOM1
P20
IOM1
P21
IOM1
P22
IOM1
P23
IOM1
P24
IOM2
P17
IOM2
P18
IOM2
P19
IOM2
P20
IOM2
P21
IOM2
P22
IOM2
P23
IOM2
P24
IOM1
P9
IOM1
P10
IOM1
P11
IOM1
P12
IOM1
P13
IOM1
P14
IOM1
P15
IOM1
P16
IOM2
P9
IOM2
P10
IOM2
P11
IOM2
P12
IOM2
P13
IOM2
P14
IOM2
P15
IOM2
P16
IOM1
P25
IOM1
P26
IOM1
P27
IOM1
P28
IOM1
P29
IOM1
P30
IOM1
P31
IOM1
P32
IOM2
P25
IOM2
P26
IOM2
P27
IOM2
P28
IOM2
P29
IOM2
P30
IOM2
P31
IOM2
P32
A1,B1,C1
C2, B2, A2
IOM1
IOM2
Fabric A
LOM
CPU
2
8 lane
PCIe
Mezzanine
Ethernet
I/O Module
Fabric C
Fabric B2
I/O Module
Quarter-Height (Slot b)
Front of Chassis
Fabric A2
CPU
1
8 lane
PCIe
Fabric A
LOM
CPU
2
8 lane
PCIe
Fabric B
Fabric C2
I/O Module
Mezzanine
Quarter-Height (Slot c)
CPU
1
8 lane
PCIe
Fabric A
LOM
CPU
2
8 lane
PCIe
Mezzanine
Fabric C1
I/O Module
Fabric C
Fabric B1
I/O Module
Quarter-Height (Slot d)
CPU
1
8 lane
PCIe
Fabric A
LOM
CPU
2
8 lane
PCIe
Mezzanine
Fabric A1
Fabric B
8.4Tb Mid-plane
Ethernet
I/O Module
CPU
1
M1000e Mid-plane
Fabric A:
Fabric B & C:
62
Slot2a
Slot3a
Slot4a
Slot5a
Slot6a
Slot7a
Slot8a
A1 P1
A1 P2
A1 P3
A1 P4
A1 P5
A1 P6
A1 P7
A1 P8
Slot2b
Slot3b
Slot4b
Slot5b
Slot6b
Slot7b
Slot8b
IOM A1 and A2
A2 P2
A2 P3
A2 P4
A2 P5
A2 P6
A2 P7
A2 P8
Slot1c
Slot2c
Slot3c
Slot4c
Slot5c
Slot6c
Slot7c
Slot8c
A1 P9
Slot2d
Slot3d
Slot4d
Slot5d
Slot6d
Slot7d
Slot8d
A1
A2
Slot2a
Slot3a
Slot4a
Slot5a
Slot6a
Slot7a
Slot8a
A1 P1
A1 P2
A1 P3
A1 P4
A1 P5
A1 P6
A1 P7
A1 P8
Slot1b
Slot2b
Slot3b
Slot4b
Slot5b
Slot6b
Slot7b
Slot8b
A2 P1
A2 P2
A2 P3
A2 P4
A2 P5
A2 P6
A2 P7
A2 P8
Slot1c
Slot2c
Slot3c
Slot4c
Slot5c
Slot6c
Slot7c
Slot8c
A1 P9
Slot1d
Slot2d
A2 P9
Slot3d
Slot4d
Slot5d
Slot6d
Slot7d
Slot8d
A1
A2
Slot2a
Slot3a
Slot4a
Slot5a
Slot6a
Slot7a
Slot8a
C1 P1
C1 P2
C1 P3
C1 P4
C1 P5
C1 P6
C1 P7
C1 P8
C2 P1
C2 P2
C2 P3
C2 P4
C2 P5
C2 P6
C2 P7
C2 P8
Slot1b
Slot2b
Slot3b
Slot4b
Slot5b
Slot6b
Slot7b
Slot8b
B1 P1
B1 P2
B1 P3
B1 P4
B1 P5
B1 P6
B1 P7
B1 P8
B2 P1
B2 P2
B2 P3
B2 P4
B2 P5
B2 P6
B2 P7
B2 P8
Slot1c
Slot2c
Slot3c
Slot4c
Slot5c
Slot6c
Slot7c
Slot8c
C1 P9
C2 P9
Slot1d
Slot2d
B1 P9
B2 P9
Slot3d
Slot4d
Slot5d
Slot6d
Slot7d
Slot8d
C1
C2
B2
66
Adapters
Pass-Through
67
M6348
M6220
Cisco
3032
Cisco
3130G
Cisco
3130X
Broadcom 5709
LOM/Mezz
Broadcom 5709
4-port NDC/Mezz
Intel ET
4-port Mezz
Broadcom 5719
4-port Mezz
Broadcom 5720
4-port Mezz
PowerEdge
M I/O
M8024-k
M8024
M8428-k
Aggregator
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not Compatible
Not Compatible
Not Compatible
Not Compatible
10Gb
Pass-Through
(original model)
10Gb
Pass-Through
-k
B22DELL
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
*
*
*
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
*
*
*
Not
Compatible
Not
Compatible
*
*
*
*
*
*
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
N/A
N/A
N/A
N/A
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
*
*
Not
Compatible
Not
Compatible
N/A
N/A
Not
Compatible
Not
Compatible
N/A
N/A
Not
Compatible
Not
Compatible
N/A
N/A
Not
Compatible
Not
Compatible
Not
Compatible
Not Compatible
Not
Compatible
*
*
*
*
*
*
*
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
Not
Compatible
10Gb
PassThrough
II
10GBe on fabric A with original mid-plane (1.0) will shift down to 1Gb. Note: fabrics B & C remain 10Gb with original mid-plane (1.0)
N/A: This combination is not possible
Not Compatible: This combination will not link ]
*: In Fabric A with original mid-plane (1.0), this combination will not link
68
Mezzanine Cards
I/O Modules
M2401G
Mellanox DDR
M3601Q
Mellanox QDR
M4001Q
Mellanox QDR
M4001T
Mellanox FDR10
M4001F
Mellanox FDR
DDR
DDR
Not Supported
Not Supported
Not Supported
DDR
QDR
QDR
QDR
QDR
Not Supported
QDR
QDR
QDR*
QDR
Not Supported
QDR
QDR
FDR10
FDR10
Not Supported
QDR
QDR
FDR10
FDR**
70
Mellanox DDR
Connect-X
Mellanox QDR
Connect-X2
Mellanox QDR
Connect-X3
Mellanox FDR10
Connect-X3
Mellanox FDR
Connect-X3
M420
Not
Supported
Not
Supported
Not
Supported
Not
Supported
M520
Not
Supported
Not
Supported
Not
Supported
M620
Not
Supported
Not
Supported
M630
Not
Supported
Not
Supported
M820
Not
Supported
Not
Supported
Not
Supported
Not
Supported
M830
TBD
TBD
TBD
TBD
TBD
M910
Blade Servers
Mezzanine Cards
Not
Supported
Mezzanine Cards
I/O Modules
M4424
Brocade FC4
FC8
Passthrough
Dell 8/4Gbps FC
SAN Module
Brocade FC8
M6505
Brocade FC16
Emulex FC4
FC4
FC4
FC4
FC4
FC4
Not
compatible
QLogic FC4
FC4
FC4
FC4
FC4
FC4
Not
compatible
FC4
FC4
FC8
FC8
FC8
FC8
FC4
FC4
FC8
FC8
FC8
FC8
Not
compatible
Not
compatible
FC8
FC8
FC8 FC16*
Not
compatible
Not
compatible
FC8
FC8
FC8 FC16*
71
M5424
FC4
Passthrough
72
QLogic
QME2662
FC8
Emulex
LPm16002
FC16
M420
Not
Supported
Not
Supported
M520
Not
Supported
Not
Supported
M620
M630
M820
M830
TBD
TBD
TBD
TBD
M910
Blade Servers
QLogic
QME2572
FC16
Server Adapter
Portfolio
73
10Gb Converged
Ethernet
10Gb Converged
Ethernet
1Gb Ethernet
Broadcom 5720 4P LOM
Broadcom 5719 4P Mezz
Intel I350 4P Mezz
QDR/FDR
InfiniBand
Mellanox CX3 FDR Mezz
Mellanox CX3 FDR10 Mezz
74
Global Marketing
Broadcom
57810S-k NDC
Broadcom
57840S-k NDC
QLogic QMD8262-k
Intel X520-k NDC
NDC (Will EOL @
end of 12G)
Emulex
OCm14102U2-D NDC
(12G only)
Emulex
OCm14102U4-D NDC
(13G only)
(Default choice)
Ports x Link Speed
Supported Speed
Chipset
Interface
ISCSI HBA
ISCSI Boot
FCoE
FCoE Boot
Switch Independent
Partitioning
DCB
SR-IOV
WOL
PXE
EEE
Multi-queue2 (per port)
Supported Servers
Strengths
2x10Gb
1Gb, 10Gb
57810S
KR
Yes
Yes
Yes
Yes
4x10Gb
1Gb, 10Gb
57810S
KR
Yes
Yes
Yes
Yes
2x10Gb
1Gb,10Gb
X520/82599
KR
No
Yes
Yes
Yes
2x10Gb
10Gb
P3+
KR
Yes
Yes
Yes
Yes
2x10Gb
10Gb
Skyhawk
KR
Yes
Yes
Yes
Yes
2x10Gb
10Gb
Skyhawk
KR
Yes
Yes
Yes
Yes
Yes 4
Yes 5
No
Yes
Yes 4
Yes 3
Yes
Yes
Yes
Yes
No
128 TX, 128 RX
M620, M820
Yes
Yes
Yes
Yes
No
128 TX, 128 RX
M620, M820
Yes
Yes 1
Yes
Yes
No
64 TX, 64 RX
M620, M820
Yes
No
Yes
Yes
No
64 TX, 64 RX
M620, M820
Yes
Yes
Yes
Yes
No
128 TX, 128 RX
M620, M820
Yes
Yes
Yes
Yes
No
128 TX, 128 RX
M630, M830
Convergence
features like ISCSI
HBA, FCoE and
NPAR
NPAR EP , RoCE
Support
Convergence features
FCoE, iSCSI HBA and
NPAR
Convergence features
FCoE, iSCSI HBA and
NPAR
Convergence
features like ISCSI
HBA, FCoE and
NPAR
1: Citrix Xenserver 6.0 and Linux KVM only. 63 VFs per port
2: No. of queues will vary depending upon hypervisor memory limitations
3: 8 Partitions per 10Gb port
4: 4 partitions per 10Gb port
5: 2 partitions per 10Gb port
75
Global Marketing
76
Broadcom 57810S-k 2p
10Gb LOM
2x10Gb
1Gb, 10Gb
57810S
KR
Yes
Yes
Yes
Yes
Broadcom 5720 4p
1Gb LOM
4x1Gb
1Gb
5720
Serdes
No
Yes
No
No
Yes2
No
Yes
Yes
Yes
Yes
No
128 TX, 128 RX
M420
No
No
Yes
Yes
Yes
8 TX, 8 RX
M520
Global Marketing
4x1Gb
1Gb
Chipset
I350
5719
Interface
ISCSI HBA
iSCSI Boot
FCoE
FCoE boot
Serdes
No
Yes
No
No
Serdes
No
Yes
No
No
Switch Independent
Partitioning
No
No
DCB
SR-IOV
WOL
PXE
EEE
No
No
Yes
Yes
Yes
No
No
Yes
Yes
Yes
Multi-queue1 (per
port)
8 TX, 8 RX
8 TX, 8 RX
Supported Servers
Great for
77
Global Marketing
Broadcom
(Qlogic) 57810Sk DP 10Gb
QLogic
QME8262-k
(EOL @ end of
12G)
2x10Gb
2x10Gb
2x10Gb
2x10Gb
2x10Gb
2x10Gb
Supported Speed
1Gb,10Gb
1Gb,10Gb
10Gb
1Gb, 10Gb
1Gb, 10Gb
1Gb, 10Gb
Chipset
57810S
X520
P3+
Catapult I
Skyhawk
Skyhawk
Interface Type
KR
XAUI/KR
KR
KR
KR
KR
ISCSI HBA
Yes
No
Yes
No
Yes
Yes
iSCSI Boot
Yes
Yes
Yes
No
Yes
Yes
FCoE
Yes
Yes
Yes
Yes
Yes
Yes
FCoE boot
Yes
Yes
Yes
Yes
Yes
Yes
Switch Independent
Partitioning
Yes3
No
Yes
No
Yes3
Yes4
DCB
Yes
Yes
Yes
Yes
Yes
Yes
SR-IOV
Yes
Yes1
No
No
Yes
Yes
WOL
Yes
Yes
Yes
No
No
No
PXE
Yes
Yes
Yes
Yes
Yes
Yes
EEE
No
No
No
No
No
No
RoCE
No
No
No
No
No
No
Multi-queue2 (per
port)
64 TX, 64 RX
64 TX, 64 RX
Supported Servers
M630, M830
Great for
Convergence features
like ISCSI HBA, FCoE and
NPAR
78
Emulex
OCm14102-U3-D
(12G Only)
Emulex
OCm14102-U5-D
(13G Only)
NPAR EP
1) Citrix Xenserver 6.0 and Linux KVM only. 63 VFs per port 2) No. of queues will vary depending upon hypervisor
memory limitations 3) 4 partitions per 10Gb port 4) 8 partitions per 10Gb port
Convergence features
like ISCSI HBA, FCoE and
NPAR
Global Marketing
79
Features
QLogic
QLE2572 FC8
Emulex 1205-M
FC8
Qlogic
QME2662 FC16
Emulex
LPm16002 FC16
Ports x Link
speed
2x8Gb
2x8Gb
2x16Gb
2x16Gb
Supported
Speed
4Gb, 8Gb
4Gb, 8Gb
8Gb, 16Gb
8Gb, 16Gb
Chipset
2500
LightPulse
2600
LightPulse
FC Boot
Yes
Yes
Yes
Yes
Supported
Servers
M420, M520,
M620, M820,
M630, M830
M420, M520,
M620, M820,
M630, M830
M620, M820,
M910, M915,
M630, M830
M620, M820,
M910, M915,
M630, M830
Global Marketing
80
Features
Ports x Link
2 x 40Gb
2 x 56Gb
Chipset
CX-3
CX-3
Supported
Protocols
IB
IB
Supported
servers
M620
Great for
HFT, co-located
investment banks,
algorithmic trading, low
latency applications
Global Marketing
Form
Factor
11G
1G
b
Blade
NDC
Broadcom 5709 4P
1Gb Blade NDC
(M710HD, M915 only)
12G
Broadcom 57810S-k 2P
10Gb NDC
10
Gb
82
Blade
NDC
Broadcom 57712-k 2P
10Gb KR NDC
(M710HD, M915 only)
13G
Broadcom 57840S-k 4P
10Gb NDC
Broadcom 57840S-k 4P
10Gb NDC
QLogic QMD8262-k 2P
NDC
QLogic QMD8262-k 2P
NDC
1Gb
10G
b
Form
factor
Blade Mezz
Blade Mezz
11G
12G1
Broadcom 5709 4P
Adapter Mezz
Broadcom 5719 4P
Adapter Mezz
Broadcom 5719 4P
Adapter Mezz
Intel ET 4P Adapter
Mezz
Emulex OCm10102-F-M
2P XAUI Mezz
Emulex OCm14102-U3-D
Emulex OCm14102-U5-D
Broadcom 57810S-k 2P
Mezz
Broadcom 57810S-k 2P
Mezz
QLogic QME8242-k 2P
Mezz
QLogic QME8262-k 2P
Mezz
QLogic QME8262-k 2P
Mezz
Brocade BR1741M-k 2P
Mezz
Brocade BR1741M-k 2P
Mezz
Brocade BR1741M-k 2P
Mezz
13G
Form factor
Blade Mezz
11G
12G, 13G
16Gb
Blade Mezz
Emulex LPm16002 FC16
84
Vendor/Chipsets
Speed
LC
configuration
and update
Monitoring
Support
Blade
NDC
10GbE
10GbE
10GbE
10GbE
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Blade
LOM
10GbE
1GbE
Yes
Yes
Yes
Yes
Blade
Mezz
Broadcom 57810S-k
Broadcom 5719 Serdes
Intel I350 Serdes
Intel X520 x/k
QLogic QME8262-k
Brocade BR1741M-k
10GbE
1GbE
1GbE
10GbE
10GbE
10GbE
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
No
XAUI-KR
Transition
Midplane
Enhancement
86
Confidential
Global Marketing
*This date is confirmed and was based on factory supply transition from original to enhanced midplane.
87
Confidential
Global Marketing
All 10GbE mezz cards and LOMs launched in 2011 or later are KRbased
M710HD 10Gb LOM risers
Brocade BR1741M-k, QLogic QME8242-k, etc.
88
Confidential
Global Marketing
Q: Will I be able to tell whether a chassis has the standard or enhanced midplane?
A: Yes, via CMC on Chassis Health Summary screen. Back-of-chassis IOM Bay labels will also change
to reflect 10Gb-Fabric-A support.
Q: Can I use KR-based mezz cards and switches on fabrics B and C of my existing chassis?
A: Yes, fabrics B & C fully support 10GbE KR on any midplane.
Q: Do these midplane and XAUI-KR changes impact any other currently shipping I/O modules?
A: No. Gigabit Ethernet Switches, FC4/8 switches, and QDR/DDR IB switches are not affected by the
XAUI to KR transition or the midplane transition. Note that these changes also dont impact
support for M710HD when configured with 4x1GbE LOMs.
89
Confidential
Global Marketing
Upgrade via CusKit would require new midplane + Service installation (custom service offering), new Fabric A
10Gb switches; would result in unused 10Gb capability on ports linked to existing 1Gb LOMs
Discard / Re-purpose:
8x M710HD
10Gb LOMs
8x M610
1Gb LOMs
1x M1000e
Chassis
Add:
2x Gigabit
Enet IOMs
1x M1000e
Enhanced
Midplane
1x M1000e
Standard
Midplane
2x 10GbE
IOMs
(CusKit)
90
Confidential
2x Gigabit
Enet IOMs
Considerations:
Half the internal ports of 10GbE IOM will run at
1Gb (with installed M610 1Gb LOMs), i.e.
diminished benefit of 10Gb upgrade
Enhanced midplane can be replaced by customer
on-site, but will require chassis downtime
(including all installed servers)
Global Marketing
Unseat the power supplies, CMC, KVM, FANs and switches (they can stay in the chassis just pull them out a bit)
Remove the blades from the chassis
Remove the 4 screws holding in the blade cage and slide the cage completely out and place face down on cart
of anti-static mat on floor.
Unseat the ribbon cable and remove the 4 screws holding the backplane into the blade cage.
Remove and replace the backplane with the new unit, plug in ribbon cable, new screws should come in the kit to
secure mid-plane to cage.
Replace the blade cage and secure the cage with 4 screws
Plug in blades, CMC, KVM, FANs, switches and swap power supplies if new ones were ordered
91
Confidential
Global Marketing
via CLI:
racadm getsysinfo # search/grep for Midplane Revision
92
Confidential
enhanced midplane
(supports 10Gb on Fabric A)
Global Marketing
Deployment and
Technical Guides
93
Document Title
Link
M6220
http://del.ly/m6220stacking
http://del.ly/m6200mstp
http://del.ly/m6220vlan
M6220, M6348
Sizing and Best Practices for Deploying VMware with Dell EqualLogic Storage
http://del.ly/vmwareoneql
CLI Transition Guide for Dell 7000, 8024, M8024, M6348, M6220 switches
http://del.ly/cli_transition
http://del.ly/portaggregator
Deployment of Dell M6348 Blade Switch With Cisco 4900M Catalyst Switch (using Simple Mode)
http://del.ly/m6448tociscocatalyst
SAN Design Best Practices for the M1000e Blade Enclosure and EqualLogic PS Series Storage (1GbE) http://del.ly/bladeeqlintegration
http://del.ly/m8024kend2endsip
End-to-end deployment using SIP and M8024-k
http://del.ly/m8024kstacking
http://del.ly/m8024kfipsnooping
Deployment of Dell M8024-k Blade Switch with Cisco Nexus 5000 Series Switch (in Simple Mode)
http://del.ly/m8024kcisconexussimple
Deployment of Dell M8024-k Blade Switch with Cisco Nexus 5000 Series Switch
http://del.ly/m8024kcisconexus
MXL
http://del.ly/mxlstacking
MXL
http://del.ly/mxlfipsnooping
Dell EqualLogic
http://del.ly/eqlcompatmatrix
Dell EqualLogic
http://del.ly/eqlconfigguide
Dell EqualLogic
http://del.ly/eqlconfigportal
Best Practices for Dell EqualLogic SANs Using Cisco Nexus 2248TP 1Gb Fabric Extender
http://del.ly/eqlciscofex
94
http://del.ly/sandesignbestpractices
http://del.ly/ioaconfigquickref
95
Feedback
We encourage readers of this publication to provide feedback
on the quality and usefulness of this information by sending an
email to BladeInterconnects@Dell.com.
96
Change Revision
Version
Changes
January 9, 2014
December 8, 2014
97
Dell Inc.