Documente Academic
Documente Profesional
Documente Cultură
LASCON STORAGE
HOM E
BACKUP AND DR
Subsections
FDRABR
HARDWARE
TSM
M AINFRAM E
Snapshot Backups
Find
CONTACT US
POWERED BY FREEFIND
OPEN SYSTEM S
Instant Backup
DATABASES
STRATEGY
Remote Mirroring
This Page
Managing the TSM database and
log files.
Managing TSM
Effe ct
O ld command
EXT END
DBSPACE
QUERY DBVOLUME
SET
Used to define the database backup device class and must be run before the first
DEFINE
DBRECOVERY
backup.
DBBACKUPT RIGGER
SET
A new command used to decide how much diagnostic information to report.
DBREPORT MODE Options are; NONE, FULL, PART IAL.
None
back to top
www.lascon.co.uk/tsm-database-and-log.php#db6base
8/16/13
When a TSM database is initially created on multiple file systems, that database will be spread eaqually over all
the file systems. However if you add an extra file system to the database space using the 'extend dbs' command,
DB2 will not rebalance the database to spread the data equally. This means that if some of the original file
spaces were 100% full, they will still be 100% full after the new filespace is added and this could cause the TSM
to stop.
If you are running TSM Server V6.2 or above you can rebalance the database dynamically using DB2 commands.
I suggest that you look up the IBM technote about this, and also contact IBM for advice before trying this.
The database can be sized anywhere between 2.2GB and 1TB. The DB2 database will be between 35 and 50%
bigger than the equivalent legacy database, partly becuase it hold sort space for SQL queries. The DB2 database
is largely self tuning, so there is no requirement for DB2 tuning skills. A new parameter, DBMEMPERCENT,
replaces the old BUFFPOOLSIZE. This set of buffers contains much more data than the old buffer so the
recommendation is to set its size to unlimited. In fact, TSM/DB2 will try to change it to unlimited on startup.
Two other legacy features are not required now; database audits and indexed tables.
The database uses DB2 relational consistency rules to prevent incorrect data from entering, and is self auditing.
The database will aslo run automatic 'runstats' from time to time. This is a DB2 feature that optimises storage
paths through the database to improve performance.
The database also uses relational indices, so it does not require special index tables to speed up SQL queries.
2013
added
February 2013
Lasconet Forum
Visit the Lasconet Forum
back to top
www.lascon.co.uk/tsm-database-and-log.php#db6base
2/12
8/16/13
High end disk subsystems such as the EMC DMX, the HDS VSP and the IBM DS8000 have very large front end
cache to speed up performance, and stripe data in a way that makes it difficult to separate data by physical
spindle. The IBM XIV takes this virtualisation to a higher level again. To get the best performance from these
devices you want enough LUNs to spread the IO and get readahead cahce benefit, but not so many that they
become difficlult to manage. For the XIV, consider using a queue depth of 64 per HBA to get best advantage of the
parallelism capabilities.
Don't stripe your data using logical volumes, let the hardware do the striping. As a rule of thumb, consider using
50GB volumes for DISK pools and 25GB volumes for file pools. Define the same number of volumes per LUN as
the RAID type to make up the LUN, so for example with 4+1 RAID5, define 4*50GB volumes per LUN, then each
LUN will use 250GB, with effective capacity of 200GB.
The Unix Tips section contains some detail on how to use VMSTAT and IOSTAT commands to investigate
potential disk bottlenecks.
back to top
Userid TSM_ADMIN can now be used to stop and start the TSM services
Recovering from a full archive log
Under tsm 6.x, the archive and active log directories can fill up, and if they do, the server will shut down. To
prevent this, you need to make sure you trigger a FULL database backup once the archive log hits a threshold,
but if the worst happens and the log files do fill up, you need a recovery process.
If this happens then you cannot use TSM commands to move the logs into bigger directories, as you cannot start
TSM. What you need to do is create temporary logs elsewhere, then prune the archive log using native DB2
commands. However, remember that the archive log will hold enough information to wind back through the last 2
full backups, so you need to run 2 full backups to clear it down.
1. Create a temporary directory large enough to hold your active logs. The dsmserv.opt file may contain the log sizes in
the ACTIVELOGSIZE parameter, and if not, it will point to the physical log location.
2. Open a DB2 command line and run the commands below to switch the logs to a new location
Set db2instance=SERVER1
db2start
db2 update db cfg for tsmdb1 using newlogpath path\to\new\logs
db2stop
db2start
3. 'Activate' the database to copy the log files with the following command, this command does not affect the original logs.
This may take a while, and success will be indicated when you see a command prompt again.
4. Now you need to back the database up to clear the logs out, and you need to do this on disk, so identify or create a
directory with enough space to take a database backup then run the following DB2 commands.
db2stop
db2start
db2 backup db tsmdb1 to path\to\database\backup\directory
The archive logs will start pruning once you see the Backup Successful message, but this could take a while to
appear if your database is large. M ake a note of the backup timestamp, which will look something like The
timestamp for this backup image is: 20120412130821
5. Find some more space, and run another full DB2 database backup with the command.
db2 backup db tsmdb1 to path\to\another\database\backup\directory
When this second backup completes, the archive log directory and original active log directory are empty of log files.
M ake a note of the backup timestamp again, let us call this one 20120412150425
6. Now you need to delete the first backup using these commands note how you use the timestamp from step 4.
www.lascon.co.uk/tsm-database-and-log.php#db6base
3/12
8/16/13
7. Point DB2 back to the original, empty active log in the original location, you will get this from the
ACTIVELOGDIRECTORY parameter in dsmserv.opt
db2 UPDATE DATABASE CONFIG FOR TSMDB1 USING NEWLOGPATH path\to\activelogdir
8. Connect to the database again, and that will automatically start moving the active logs from the temporary location to
original active log location, and again, this can take a while if the logs are big.
db2 force application all
db2stop
db2start
db2 connect to tsmdb1
9. Now you need to start the TSM server up and run a good backup. You need to start the server in the foreground to do
this, so open a normal Windows command line and navigate to the server directory and run dsmserv. If you have more
than one TSM server on this machine, you may need to use the -k option to get the right server. This will bring you up
a TSM server command line. Disable your client sessions then take 2 full database backups. You need to know your
backup device classes to be able to do this.
Disable sessions
Backup db type=full dev=your_db_devclass
10. Delete the second DB2 database backup as follows, using the database timestamp that you recorded in step 5.
db2 PRUNE HISTORY 20120412150425 WITH FORCE OPTION AND DELETE
11. Now you can halt your server in the foreground, and start it normally. Remember to enable sessions.
It is possible to query what is happening while a database recovery is in progress with the db2pd utility, a DB2
diagnostic tool that is provided with the TSM server installation code. You simply run this as a command from the
shell prompt, like this:
tsm:~ # db2pd
db2pd> You are running db2pd in interactive mode.
db2pd> If you want command line mode, rerun db2pd with valid options.
db2pd> Type -h or -help for help.
db2pd> Type q to quit.
Now runstats will not start automatically when you restart TSM server. However you need runstats to keep your
database optimised, so once you are happy that your TSM server is up and running, submit the following
commands to the DB2 instance for your TSM server and Runstats will resume normal processing.
db2 connect to tsmdb1
db2 update db cfg for TSMDB1 using AUTO_RUNSTATS ON
www.lascon.co.uk/tsm-database-and-log.php#db6base
4/12
8/16/13
back to top
You might have to run the restart command a few times before the issue is resolved. If this does not fix the
problem you probably need to contact IBM Support, although you can use the db2dart command to run a
database analysis. This generates a report file that would be useful for IBM support.
db2 force application all
db2stop
db2dart db2inst1 /db
back to top
and to change the value use the command below, selecting a value that is suitable for your server.
chdev l sys0 a maxuproc=2048
back to top
www.lascon.co.uk/tsm-database-and-log.php#db6base
5/12
8/16/13
To calculate the deduplication overhead, use the formula below to get the number of chunks
chunk_count = total_backedup_data_in_GB * 10,000 * 2
The doubling factor at the end of the formula is to cater for 'base deduplication chunks' that is, chunks that must
remain even after a file is expired and deleted from TSM. The extra database overhead is then
chunk_count * (490 + 190 * extra_backup_copies)
Running this formula on an existing server with a 135GB database predicted an increase of 105GB with
deduplication, which is not a trivial amount.
back to top
www.lascon.co.uk/tsm-database-and-log.php#db6base
6/12
8/16/13
Recovery log processing was enhanced in TSM 4.2. If the DB Backup Trigger is set correctly, and the LOGMODE
is in ROLLFORWARD, then a database backup will start when the log reached 100% full. If the Recovery log hits
100%, then TSM will stop all processes except the database backup. When the backup completes, TSM issues
the message
ANR2104I Activity log process restarted - recovered from an insufficient space condition in the Log or Database.
This should help us avoid some difficult recovery situations.
back to top
Database Defragmentation
This contentious issue applied to legacy databases only. The legacy TSM Server database has a b-tree format,
and grows sequentially from the beginning to end. When file entries are deleted by expiration processing or file
space/volume delete processing, this leaves spaces or holes in that database. These may be re-used later
when new information is added, but they mean that the TSM database is using more space than it needs to. The
only way you can compress the database so that the 'holes' left by deleted pages are not present is to use the
database unload/reload utility.
The problem is that while the dump takes about an hour, the load utility can take several hours. Does it make a
difference? I have seen performance improve after defragmenting a database, and I've also see an unload/reload
make performance worse. A defrag will reduce the physical size of your database.
The Tivoli team supplied a new command with TSM 5.3 for to you to check to see what an unload/reload would
achieve, called 'ESTIMATE DBREORGSTATS' This will estimate the amount of space that would be recovered by
an unload reload.
For older releases of TSM use the QUERY DB to see if you need to defrag your TSM DB.
Here a 49GB database can be reduced by 9.4GB = 19%, but it is only 76% used, so 5% could be reclaimed by
defragging. Some people claim that TSM tries to allocate pages in a way that leaves you with as good as
possible performance, and defragging the database will degrade performance. Its also possible that after a
defrag, the database will quickly become defragmented again, as it inserts data into the tree. The following
formula can be used to see how much space could be reclaimed by an unload/reload.
SELECT CAST((100 - (CAST(MAX_REDUCTION_MB AS FLOAT) * 256 ) /
(CAST(USABLE_PAGES AS FLOAT) - CAST(USED_PAGES AS FLOAT) ) * 100) AS
DECIMAL(4,2)) AS PERCENT_FRAG FROM DB
A high PERCENT_FRAG value can indicate problems. If you think your database needs a defrag, then if possible,
take a copy and try that first. That will give you an indication of how much time is needed for the load.
back to top
THEN in TSM
If you use incremental database backups, then remember that after an EXTEND DB the next DB backup must be
a full backup.
back to top
www.lascon.co.uk/tsm-database-and-log.php#db6base
7/12
8/16/13
Which will format a 5 meg.log volume called tsmlogvol7. Size options are 'k' 'm' 'or 'g' and data type options are
'db' 'log' or 'data'
back to top
will audit the archive storage, and runs for 1-2 hours depending on your database size
/dsmserv auditdb fix=yes diskstorage detail=yes
will audit the disk storage pools, and takes about 30 mins, depending on the size of the database, and how full
the disk pools are. Best done when all the data is migrated out to tape.
/dsmserv auditdb fix=yes inventory detail=yes
www.lascon.co.uk/tsm-database-and-log.php#db6base
8/12
8/16/13
Each step is called based on the architecture; the DSMSERV utility runs several concurrently, 5-10 at a time,
returning output as each step completes and picking up the next step in order. Steps 1-9 will finish almost
immediately. Steps 10-16 will run next, and will take a slightly longer time, these follow definitions in order of
creation. When Step 17 begins, it will trigger Step 33, and depending on how many entries there are in the
database, the output from 33 will appear mixed with the output from Steps 18-32. Step 33 is reviewing all client
data in the database, this is the longest running step in the audit process.
Typical output from Step 33 (from a large database) will look like this:
ANR4134I AUDITDB: Processed 8260728 entries in database tables and 0 blocks in
bit vectors. Elapsed time is 1:05:00.
ANR4134I AUDITDB: Processed 9035641 entries in database tables and 0 blocks in
bit vectors. Elapsed time is 1:10:00.
ANR4134I AUDITDB: Processed 9812999 entries in database tables and 0 blocks in
bit vectors. Elapsed time is 1:15:00.
ANR4134I AUDITDB: Processed 10663992 entries in database tables and 0 blocks in
bit vectors. Elapsed time is 1:20:00.
ANR4134I AUDITDB: Processed 11677212 entries in database tables and 0 blocks in
bit vectors. Elapsed time is 1:25:00.
ANR4134I AUDITDB: Processed 12014759 entries in database tables and 0 blocks in
bit vectors. Elapsed time is 1:30:00.
Note this output refers to 'entries'. Entries are not a standard reference in TSM, this is a parsed view of data files,
part of the occupancy. To estimate how many entries will be scrolled through the audit, run this formula on a
command line within TSM:
select sum(num_files)*3 from occupancy
The '3' refers to the three pieces to a file: the entry, a header for the entry, and an active/inactive flag. Remember
that this is only an estimate, the reason for running the audit is possible corruption, there may be pieces
missing or mis-filed.
Entries are read anywhere from 500K to 1 million every five minutes, so based on the output from this formula,
this is how to estimate the time for the audit to complete.
Audits can be run on pieces of the database instead of the whole - a specific storage pool or the administrative
portion - this can be a considerable time-saver, but if it is unknown what part of the database is corrupt, this may
not be a worthwhile option.
To run an audit, the TSM server instance must be down. If there are multiple TSM instances on a server, the
DSMSERV executable must be in the primary server directory, but if the audit is running on a secondary
instance, for example, parameters must be passed to operating system so the utility will know where it is
looking for the database:
AIX# export DSMSERV_DIR=/usr/tivoli/tsm/server/bin
AIX# export DSMSERV_CONFIG=/usr/tivoli/tsm/server/bin/<subdir>/dsmserv.opt
To run an audit just on the administrative portion of the database (the fastest, 10-15 minutes), start the utility this
way:
AIX# at now
dsmserv auditdb fix=yes admin detail=yes > /tmp/tsmauditadmin.log
[ctl-D]
The process will run in the background, and a log will be kept; this log can be run with the tail -f command by
multiple users to track the progress.
To run the audit on the archived data (1-2 hours, depending on size of archives), enter this:
dsmserv auditdb fix=yes archstorage detail=yes >/tmp/tsmauditarchive.log
To run the audit on the diskpool (very fast if all data is migrated), enter this:
dsmserv auditdb fix=yes diskstorage detail=yes > /tmp/tsmauditdisk.log
To run on the client data only, not including the archives (still the longest running), enter this:
dsmserv auditdb fix=yes inventory detail=yes > /tmp/tsmauditdata.log
Again, running on the inventory, while it can be run separately, it is almost a moot point.
If any data is found to be damaged, location messages as well as the fix (usually a deletion) will output to the
log as follows:
ANR1777I afaudit.c(967: Object 0.62882489 is \WINDOWS\INF\DSUP.PNF for node
<NODENAME> (257), filespace \\<nodename>\c$ (1).
ANR1777I afaudit.c(967: Object 0.62882490 is \WINDOWS\INF\DSUPT.PNF for node
<NODENAME> (257), filespace \\<nodename>\c$ (1).
ANR1777I afaudit.c(967: Object 0.62882491 is \WINDOWS\INF\DVD.PNF for node
<NODENAME> (257), filespace \\<nodename>\c$ (1).
ANR4303E AUDITDB: Inventory references for object(0.62882489) deleted.
ANR4303E AUDITDB: Inventory references for object(0.62882490) deleted.
ANR4303E AUDITDB: Inventory references for object(0.62882491) deleted.
www.lascon.co.uk/tsm-database-and-log.php#db6base
9/12
8/16/13
Be sure sufficient outage time is scheduled. Once an audit begins, it is not good practice to halt the process,
because the current location of the audit is not truly known - a data file could be open, and halting may actually
cause further corruption.
back to top
FILE PREPARATION
Obviously, you need a good backup of a TSM database, and you need to know the device class that was used for
the backup. For illustration, we will assume the latest database backup is on a tape called T012456L and used a
devclass called T_LTO3.
You also need a list of the database and log file names and sizes. If you use DRM then the best place to get this
from is the latest prepare. If you don't use DRM, you can get this info when your TSM server is running with the
commands
query dbvol f=d
query logvol f=d
That's all very well if you have a planned outage, but what if your database crashes and you don't have a prepare?
You can still get the info with dsmserv commands, use
dsmserv display dbvolumes
dsmserv display logvolumes
Create two text files, one called DB.VOLS that contains the Database file names, paths and sizes and one called
LOG.VOLS for the log files. The files should look like this, but use your own file names, paths and sizes. The file
sizes are in MB, so this is a 20GB database.
DB.VOLS
"H:\TSMDB\EXT01\DB01.DSM"
"H:\TSMDB\EXT02\DB02.DSM"
"H:\TSMDB\EXT03\DB03.DSM"
"H:\TSMDB\EXT04\DB04.DSM"
5000
5000
5000
5000
LOG.VOLS
"H:\TSMRLOG\RLOG01.DSM" 4096
"H:\TSMRLOG\RLOG02.DSM" 4096
10/12
8/16/13
Navigate to the c:\program files\tivoli\tsm\server\ directory and run the following command. Note that the logs are
described first, then the database, and that you need to say how many of each type of file you are formatting, so 2
log volumes and 4 database volumes.
DSMSERV FORMAT 2 FILE:LOG.VOLS 4 FILE:DB.VOLS
When you run a DSMSERV FORMAT on a Windows server, it resets the registry entry for the TSM server, and this
must be put back before you attempt the restore. Use REGEDIT and navigate to the correct registry entry for your
server. If you just have one TSM server on this Windows box, it will be Server1, otherwise Server2-4 depending on
which server you are working with. The Server1 key is HKEY-LOCALMACHINE\SOFTWARE\IBM\ADSM\CurrentVersion\Server\Server1 and you need to change the path entry from
c:\progam files\tivoli\TSM\Server to Server1.
CLEAN UP
OK, now you have a copy of your TSM database as it was when it was backed up. Your problem now is that you
may have data on your disk storage pools that is not recorded in your database, or your database will think data
exists on disk that has been moved off. The database has also lost all record of any tape activity that has
happened since the backup, so you need to get these two in step again.
Before you start TSM up, go into the dsmserv.opt file and add the lines
NOMIGRRECL
DISABLESCHED YES
EXPINT 0
These three commands will prevent migration, client schedule and expire inventory from running. Now start your
server in the foreground and run command DISABLE SESSIONS to stop clients from contacting the server.
Audit your disk storage pools using AUDIT Volume volume_name FIX=YES, and that will hopefully fix any
problems, but you may need to delete and redefine your disk volumes, discarding faulty data, to get migration to
run clean.
Audit your tape library, and that will let TSM know the current location of all tapes.
check your latest saved volhist file for any tapes that have been deleted or updated since the backup ran. You will
need to audit these tapes too. Once you complete the audits, back out the changes you made to the dsmserv.opt
file, halt the TSM server, then start it normally and enable sessions again.
back to top
TSM mirroring
Software mirroring just applies to the legacy database. If TSM is managing the mirror and it detects
corrupt data during a write, it will not write the corrupt data to the second copy. TSM can then use the good
copy to fix the corrupt mirror. TSM also mirrors at transaction level, and hardware at IO level. Hardware will
always mirror every IO, but TSM will only mirror complete transactions. This also protects the mirror from
corruption.
back to top
11/12
8/16/13
TSM will only perform one concurrent IO operation to every storage pool volume, so it is better to have a number
of smaller volumes than a single large volume in your disk storage pools. Also, it is easier to relocate a small
volume to a different disk pool if space requirements change. However, every volume will use one TSM
processing thread, and the TSM server will crash if too many threads are allocated.
The normal process is to initially write backup data to disk, then move it off to tape. It is possible to copy the data
to tape but not delete it from the disk pool, if the disk cache setting is set to 'yes'. The TSM server would then use
the disk data for restores and delete it as space is needed for fresh backups. This would speed up recovery, but
slow down backups as the TSM server has to do extra work clearing out data, and would also make the TSM
database bigger as TSM needs to store two locations for recently backed up data. It is your choice, faster restores
or slower backups and a bigger database.
back to top
HOM E
BACKUP AND DR
HARDWARE
M AINFRAM E
Cookies Policy: T his site does not use cookies to gather personal data.
See the Privacy section for details
www.lascon.co.uk/tsm-database-and-log.php#db6base
OPEN SYSTEM S
DATABASES
STRATEGY
12/12