Module 13: How Teradata uses NCR Systems

After completing this module, you will be able to:

Describe the fundamental relationship between UNIX,
logical units, and disk array controllers.
Describe the fundamental relationship between Vdisks,
Pdisks, slices, and LUNs.
Specify the maximum disk space that an AMP can manage.
Given a new disk array and a set of configuration
parameters, determine the number of LUNs and the size of
each LUN as configured by PUT and pdeconfig.

Teradata and the Processing Node

FSG (File Segment Cache) - managed by PDE

PE vproc

PE vproc

AMP

AMP

AMP

AMP

AMP

AMP

AMP

AMP

AMP

AMP

vproc

vproc

vproc

vproc

vproc

vproc

vproc

vproc

vproc

vproc

Teradata TPA S/W

Teradata PDE (BaseVproc)
UNIX SVR4 MP-RAS or Windows 2000
Process Control
CPUs
Pentium IV
3.06 GHz

Memory Mgmt.
Memory

I/O Mgmt. (Device Drivers)

BIC4M

QFC

Eth.

Pentium IV
3.06 GHz

PDE - Parallel Data Extensions

QFC - Quad Fibre Channel

BYNET Disk
Arrays

LANs

SMP Memory and Teradata

Free Memory
O.S. 80 MB
13 Vprocs @ 40 MB each
= 520 MB
20% Remaining Space
= 290 MB
2048 MB

80 MB

520 MB

290 MB
FSG Cache
1158 MB
Examples of objects that are
memory resident:
Hash Maps
Configuration Maps
Master Indexes
RTS - Request-to-Steps Cache
D/D - Data Dictionary Cache

20% of remaining space 290 MB available to O.S.

FSG (File Segment Cache)

(Examples of use Data Blocks & Cylinder Indexes)
Managed by PDE Software

80% of remaining space 1158 MB available for FSG

40 MB

40 MB 40 MB

40 MB

40 MB

40 MB

40 MB

PE
Vproc

AMP
PE
Vproc Vproc

AMP
Vproc

AMP
Vproc

AMP
Vproc

AMP
Vproc

RTS
D/D
Cache

RTS
D/D
Cache

Master
Index

Master
Index

Master
Index

Master
Index

PDE (BaseVproc) 40 MB

..

Hash Maps
Configuration Maps

Operating System 60 to 100 MB

Ex. 2048 MB 2 GB Memory

Master
Index

SMP Shared Memory

When the operating
system boots, it
allocates approximately
60 to 80 MB for itself and
40 MB for each vproc.
FSG - pool of memory
managed by PDE and
each AMP uses what it
needs.

100% - 1448 MB
90% - 1303 MB
80% - 1158 MB

Managed by
PDE FSG
software.

FSG Cache
13 Vprocs @ 40 MB each
520 MB
Operating System 80 MB

Memory
managed by
O.S.

Ex. 2048 MB - 2 GB
Memory

Xctl Parameter - FSG Cache Percent - usually set between 80 and 90%

Memory Managed by the Operating System

Memory managed and used by the operating system and the vprocs is
sometimes called free memory. Examples of how this memory is used:

Vprocs for non-Teradata file system activity

AMP worker tasks
Parser and dispatcher tasks
Messages - communication between vprocs
Dictionary cache
Request to Steps cache
Temporary work space for vprocs
Administrative programs such as:
program text and data
message buffers (ex., TCP/IP)
kernel resources
other applications (ex., FastLoad)

FSG Cache
FSG Cache is primarily used by the AMPs to access memory resident
database segments. When Teradata needs to read a database block, it
checks FSG Cache first.

FSG is also used to backup AMP updates on other SMPs. This is referred
to as buddy node backup.

Uses of FSG Cache

Permanent data blocks
Cylinder Indexes
Spool data blocks
Transient Journals
Permanent Journals
Synchronized scan (sync scan) data blocks
Buddy Backup data blocks

Buddy Node Backup (UNIX MP-RAS only)

A buddy node is another SMP in the same clique which is used to
backup AMP updates.

Updates to data blocks and/or cylinder indexes are written immediately

to disk based on following parameters:

Data Blocks only - DBSControl WriteDBsToDisk set to TRUE

Data Blocks & Cylinder Indexes - Xctl Write All to Disk set to ON
BYNET
FSG

FSG
Data Block

SQL UPDATE

AMP

Data Block

AMP
Buddy Node

Vproc Number Assignment

Each PDE, AMP, and PE is assigned a unique Vproc # in the system.
PDE Vproc #s (start at 16384 and increment by 1)
PDE on first SMP - 16384
Appear in utilities such
PDE on second SMP - 16385
as xgtwglobal.
PDE on third SMP - 16386
AMP Vproc #s (start at 0 and increment by 1)
First AMP - 0
Second AMP - 1
Third AMP - 2

Appear in utilities such

as vprocmanager,
qryconfig, etc.

PE Vproc #s (start at 16383 and decrement by 1)

First PE - 16383
Appear in utilities such
Second PE - 16382
as vprocmanager,
Third PE - 16381
qryconfig, etc.

Vproc Number Assignment Example

SMP001-4

PE
16383

PE
16382

AMP
0

AMP
4
PDE

SMP001-5

PE
16381

PE
16380

AMP
1

AMP
5
PDE

SMP001-6

PE
16379

PE
16378

AMP
2

AMP
6
PDE

SMP001-7

PE
16377

PE
16376

AMP
3

AMP
7
PDE

AMP
8

AMP
12

AMP
16

AMP
20

AMP
24

AMP
13

AMP
17

AMP
21

AMP
25

AMP
14

AMP
18

AMP
22

AMP
26

AMP
15

AMP
19

AMP
23

AMP
27

16384

AMP
9
16385

AMP
10
16386

AMP
11
16387

Disk Arrays from an O.S. Perspective

A logical unit (LUN) or Volume is a single disk to UNIX.
For UNIX systems, standard UNIX utilities that are used to slice physical
disks are also used to slice LUNs.

The operating system does not know or care about the specific RAID
technology being used for a LUN or Volume.

Operating System

LUN
or
Volume

LUN
or
Volume

LUN
or
Volume

LUN
or
Volume

The operating system (e.g., UNIX MP-RAS or Windows 2000) thinks it is

reading and writing to 4 individual disks.

Logical Units and Slices

With UNIX MP-RAS, a logical unit (LUN) will have a single UNIX partition which
is typically divided into multiple slices.
How are slices typically used?

Hold a UNIX file system (e.g., vxfs).

Provides raw data storage area (raw disk slice) for databases (e.g., Teradata).
Slice 7 (Boot Slice) contains VTOC which has starting location (physical block #) and
size of each slice on the disk.

7 - Boot

7 - Boot

1 - Root FS

UNIX MP-RAS
System Disk

2 - Swap

8 - Raw slice
assigned
to a Pdisk

Teradata
0 Pdisk space

LUN and Slicing Guidelines

Each LSI array has a maximum of 32 LUNs or Volumes.
UNIX 3.01 and above (currently on UNIX 3.02)
Maximum LUN size = 1 TB
Maximum slice size = 1 TB
Teradata sizing issues (maximum space that one AMP can address)
Teradata V2 Releases 1 and 2 43 GB
(61000 cylinders x 1488 sectors/cylinder x 512 bytes/sector)

Teradata V2R3, V2R4, and Windows NT 112 GB

(61000 cylinders x 3872 sectors/cylinder x 512 bytes/sector)

Teradata V2R4.1, V2R5.0 1.26 TB

(700,000 cylinders x 3872 sectors/cylinder x 512 bytes/sector)

Teradata, the O.S., and Disk Arrays

User

Teradata Pdisk = UNIX Slice

= Windows 2000 Partition

AMP

Vdisk
Pdisk 0

PE

File
System

Single Disk
Pdisk 1

PDE
Disk Array Controller

O.S.

Logical
Disks

LUN 0

LUN 1

Pdisk 0

Pdisk 1

Teradata and a 1456 Disk Array (Logical View)

SMP

SMP

AMP
vproc
4

AMP
vproc
6

AMP
vproc
8

AMP
vproc
10

AMP
vproc
12

AMP
vproc
1

AMP
vproc
3

AMP
vproc
5

AMP
vproc
7

AMP
vproc
9

AMP
vproc
11

AMP
vproc
13

Vdisk
0

Vdisk
2

Vdisk
4

Vdisk
6

Vdisk
8

Vdisk
10

Vdisk
12

Vdisk
1

Vdisk
3

Vdisk
5

Vdisk
7

Vdisk
9

Vdisk
11

Vdisk
13

}
}
}
}
}
}
}

AMP
vproc
2

}
}
}
}
}
}
}

AMP
vproc
0

6840-1456 Disk Array with 56 Disks - Logical View

Teradata and a 1456 Disk Array (Detail View)

QFC
QFC

AMP
vproc
0

AMP
vproc
2

AMP
vproc
4

AMP
vproc
6

AMP
vproc
8

AMP
vproc
10

AMP
vproc
12

Vdisk
0

Vdisk
2

Vdisk
4

Vdisk
6

Vdisk
8

Vdisk
10

Vdisk
12

SMP

DAC

DAC

6840-1456
Disk Array
QFC - Quad Fibre
Channel adapter

QFC
QFC

SMP

Vdisk
1

Vdisk
3

Vdisk
5

Vdisk
7

Vdisk
9

Vdisk
11

Vdisk
13

AMP
vproc
1

AMP
vproc
3

AMP
vproc
5

AMP
vproc
7

AMP
vproc
9

AMP
vproc
11

AMP
vproc
13

Logical Units and Vdisks Example

AMP
vproc
0

AMP
vproc
1

LUN 0

Vdisk 0

Pdisk 0

36 GB

36 GB

Pdisk 0
Pdisk 1

72 GB of
Max
PERM
space

LUN 2

Vdisk 1

Pdisk 2

36 GB

36 GB

Pdisk 2
LUN 1

36 GB

Pdisk 1
36 GB

Pdisk 3

72 GB of
Max
PERM
space

Assumes 36 GB Disks & RAID 1

LUN 3

36 GB

Pdisk 3
36 GB

Example of 72 GB Vdisk
Teradatas File System
software divides the
Vdisk into logical
cylinders; each
cylinder is 3872
sectors in size.

Vdisk

Pdisk 0

Physical Disks

72 GB

36 GB

36 GB

Cylinder 0

Cylinder 0

1
2

AMP
3

Disk ID
1,0
36 GB

18327

Disk ID
2,0

36 GB

36 GB

18328

Disk ID
3,0
36 GB

36653

36653
Pdisk 1

Disk ID
4,0
Assumes RAID 1 Mirroring

Teradata File System Concepts

Vdisk
With V2R4 and
V2R5, the cylinder
size is 3872
sectors.

Cylinder Index
Cylinder
1

Data Block with rows

Data Block with rows

SMP Memory
Cylinder
2

Cylinder Index
Data Block with rows

Master Index
Entry for CI #1
Entry for CI #2

Data Block with rows

Entry for CI #700,000

For a 72 GB Vdisk,
there will be
approximately
36,650 cylinders.

Max # of
Cylinders
is approx.
700,000

Cylinder Index

Size of each Cylinder Index:

V2R4.1 8 KB
V2R5.0 12 KB

Full Cylinder Read

The Full Cylinder Read feature is available with V2R5. Multiple pre-load
reads to a cylinder may be retrieved with a single large read, rather than
individual reads.
CYLINDER

Data
Block

DB

DB

DB

DB

DB

~2 MB

Enables efficient use of disk & CPU performance resources for the following
table scan operations under specific conditions:

aggregates: sum, avg, min, max, count

joins: merge joins, product joins, inner/outer joins:
large selects
merge delete
merge insert/select: empty or populated tables
full file update/deletes

Additional operations will use cylinder read in future releases .

PUT, pdeconfig, and Disk Arrays

The PUT and pdeconfig utilities are used to configure a Teradata database
environment.

One of its key functions is to scan a system for disk arrays and configure
those disk arrays for use with Teradata.

Depending on the type of array, RAID Level, size and number of disks, these
utilities will configure arrays differently.

This chart lists configuration defaults.

Summary
Memory managed and used by the operating system and the vprocs is
sometimes called free memory.

PDE software manages FSG Cache.

FSG Cache is primarily used by the AMPs to access memory resident
database segments.

The operating system and Teradata does not know or care about the RAID
technology being used.

A LUN or Volume looks like a single disk to the operating system.

With UNIX MP-RAS, a LUN or Volume is divided into slices and raw disk
slices are assigned to Teradata Pdisks.

With Windows 2000, a LUN or Volume is considered a partition and the raw
partition is assigned to a Teradata Pdisk.

PUT and pdeconfig are utilities that are used to configure a Teradata
database environment.

Review Questions
1.

Describe the fundamental relationship between the O.S., logical units, and disk array controllers.
________________________________________________________________________________
________________________________________________________________________________

2. Describe the fundamental relationship between Vdisks, Pdisks, and LUNs or Volumes.
________________________________________________________________________________
________________________________________________________________________________
3. Specify the maximum disk space that an AMP can manage.
Teradata V2R3 - ________________
Teradata V2R5.0 - ________________
4. Given a new disk array and the following parameters, complete the following chart as configured by
pdeconfig.
RAID
Level

#
of Disks

Disk
Size

RAID 1

56

36 GB

RAID 1

56

73 GB

RAID 5

40

18 GB

RAID 5

56

36 GB

Total #
of LUNs

Size of
each LUN

Module 13: Review Question Answers

1. Describe the fundamental relationship between the O.S., logical units, and disk array controllers.
Disk array controllers manage a set of logical units (implemented across a set of disks) and make
these available to the operating system.
2. Describe the fundamental relationship between Vdisks, Pdisks, and LUNs or Volumes.
Each AMP is assigned to a Vdisk which is made up of 1 or more Pdisks. A Pdisk is assigned to a
logical unit or volume (in a disk array).
3. Specify the maximum disk space that an AMP can manage.
Teradata V2R3 - 112 GB
Teradata V2R5.0 - 1.26 TB
4. Given a new disk array and the following parameters, complete the following chart as configured by
pdeconfig.
RAID
Level

#
of Disks

Disk
Size

Total #
of LUNs

Size of
each LUN

RAID 1

56

36 GB

28

36

RAID 1

56

73 GB

28

73

RAID 5

40

18 GB

10

54

RAID 5

56

36 GB

14

108