IBM Tivoli Software IBM Tivoli Software

[ | | | ]


IBM Tivoli Storage Manager for Space Management for UNIX: User's Guide


Migrating Your Files

The HSM client automatically migrates files to Tivoli Storage Manager storage when space is needed on your local file system, and automatically recalls files when you access them. Stub files, containing necessary information to locate and recall your migrated files, replace the original files on your local file system.

At any time, you can migrate and recall specific files. For example, if you run threshold migration and some files do not migrate to storage because their last access dates make them ineligible for migration, use selective migration. Files migrate immediately to Tivoli Storage Manager storage. Use selective recall to return them to your local file system.

Note:
Any application that touches a file may implicitly cause that file's last access date to change to the time that the application touches it. This is a function of the file system, not the application. Because of this, when the client backs up or archives a file, it may trigger an update to the file's last access date. This can cause problems for other applications such as , whose processing relies on accurate last access dates.

You can use the Tivoli Storage Manager preservelastaccessdate option during a backup or archive operation to specify whether to reset the last access date of any specified files to their original value following the backup or archive operation. By default, the IBM Tivoli Storage Manager client will not reset the last access date of any backed up or archived files to their original value following the backup or archive operation.

A file is eligible for automatic or selective migration when it meets the following criteria:

You can migrate any file in a set of hardlinked files that you did not exclude from space management and that you assigned a management class permitting automatic or selective migration.

For AIX GPFS and Linux86 GPFS file systems only: Newly-created files that you migrate either automatically or selectively must be older than two minutes (two minutes is default) before you can migrate them. Migrating newly-created files of less then five minutes might display incorrect results (resident size) when you use the dsmdf and dsmdu commands because the GPFS is not synchronized on all nodes when you migrate files. The dsmdf command will display correct results after GPFS synchronization, and after the next reconcilation of the file system.

This chapter describes automatic migration, premigration, and selective migration.


Migrating Your Files Automatically

To ensure that free space is available on your local file systems, the HSM client monitors space usage and automatically migrates files whenever it is necessary. Files are prioritized for automatic migration based on the number of days since they were last accessed, their size, and the age and size factors that you set for your file system. The dsmscoutd daemon searches these files in cycles and upon request from automatic migration.

The HSM client provides two types of automatic migration: threshold migration and demand migration. Threshold migration maintains a specific level of free space on your local file system. The space monitor daemon checks space usage on your local file systems at intervals that you specify. When space usage reaches the high threshold that you set for a file system, migration automatically sends eligible files to storage. When space usage reaches the low threshold that you set for a file system, migration stops. For example, if you set the high threshold for your file system to 80 percent and the low threshold to 70 percent, files begin migrating to storage when there is less than 20 percent of available space on your local file system. Files stop migrating when there is more than 30 percent of available space on your local file system.

Demand migration responds to an out-of-space condition on your local file system. The space monitor daemon checks for an out-of-space condition every ten seconds. Threshold migration starts automatically if your file system runs out of space. As files migrate, space becomes available on your file system, and the process that caused the out-of-space condition continues without waiting for threshold migration to complete. You do not receive an out-of-space error message. For example, if you attempt to copy a very large file into your file system, and there is not enough available space for the file, eligible files begin migrating automatically from your local file system to storage. As space becomes available, the process continues to copy the file to your file system.

Premigrate Your Files

For faster migration, the HSM client prepares files for automatic migration using a process called premigration. Files are copied to storage beyond the low threshold that you set while the original files remain intact on your local file system. When free space is again needed on your local file system, premigrated files become migrated files. The HSM client verifies that files did not change since they became premigrated. When your premigrated files migrate to storage, stub files replace them on your local file system.

The HSM client premigrates files each time it completes automatic migration if:

The premigration percentage represents the amount of free space on your file system containing premigrated files that are the next eligible candidates for migration. The default for the premigration percentage is the difference between the percentage that you set for the high threshold and the percentage that you set for the low threshold for your file system. You can change the premigration percentage at any time.

Start Threshold Migration Manually

The space monitor daemon monitors space usage on all file systems for which space management is active. Threshold migration starts automatically to reduce space usage to the low threshold that you set if space usage equals or exceeds the high threshold that you set.

To reduce space usage to the low threshold on your file system before it reaches the high threshold, start threshold migration manually. If space usage exceeds the low threshold that you set for your file system when you start threshold migration manually, eligible files migrate until space usage drops to the low threshold.

To start threshold migration manually for one or more file systems, follow these steps:

  1. Click Space Manager in the Hierarchical Storage Management window.
  2. Click the file systems that you want to select for threshold migration. A window displays for each file system.
  3. Select the window that you want to view.
  4. Click Selected->Start Threshold Migration. The Threshold Migration Status window displays.
  5. Click Stop in the Threshold Migration Status window to stop the migration process. Any files that migrated successfully remain in a migrated state.
  6. Click Close to close the Threshold Migration Status window.

If additional files are eligible for migration and you did not exceed the premigration percentage that you set for your file system, additional files premigrate after the threshold migration process completes. The status of the premigration process displays in the Threshold Migration Status window.

Start Migration With the dsmautomig Command

To start threshold migration from the command line, use the dsmautomig command. For example, to start threshold migration for the /home file system, enter:

   dsmautomig /home

To display information about your migrated files, use the -detail parameter with the dsmautomig command.

See dsmautomig for more information about this command.

Build a New Migration Candidates List

A migration candidates list is a prioritized list of files in your file system that are eligible for automatic migration at the time the list is built. The HSM client uses the list to determine which files to migrate and the order in which to migrate them during threshold and demand migration. Immediately prior to migration, each file is checked again to determine whether it is still eligible for migration.

The dsmscoutd daemon continuously scans your managed file systems for candidate requests from the automatic migration processes, and for elapsed scan-interval hours of all your HSM-managed file systems. A slave scout daemon is started to scan for new candidates. After a candidate scan is completed for your file system, a new migration candidates list is created and the slave scout daemon ends. Another slave scout daemon for the same file system is started after the number of hours you specified on the candidatesinterval option in your dsm.sys file.

The maximum number of slave scout daemons that can scan in parallel on a system is five. This is the default. You can change this number on the maxcandprocs option in your dsm.sys file. If a request to build new candidate pools is high, the number of parallel scout daemons might exceed the maxcandprocs value by three.

When the dsmscoutd daemon locates the maximum number of eligible candidates, it stops scanning your file system and stores the list. Use the candidatesinterval option in your dsm.sys file to specify how many hours must elapse before the dsmscoutd daemon runs again. Use the minmigfilesize option in your dsm.sys file to specify the minimum file size, in bytes, for a file to be eligible as a candidate for automigration. The dsmscoutd daemon improves the quality of an existing candidates list during each run.

The dsmscoutd daemon manages two candidates pools (APool and BPool) for each filesystem. These pools are located in /etc/adsm/SpaceMan/candidatesPools. The dsmautomig process uses the APool, while the dsmscoutd daemon periodically refreshes the BPool. Each pool has a maximum size of 10000 KB each. If the dsmautomig process migrates all candidates from the APool, the BPool then converts to the APool and the dsmscoutd daemon immediately starts to build a new BPool. Specify the maximum number of migration candidates that the dsmscoutd daemon should search for with the maxcandidates option on the dsmmigfs command. Candidates that require more space are not included. The dsmscoutd daemon stores fewer candidates if there is insufficient storage in the parent filesystem of /etc/adsm/SpaceMan/candidatesPools.

Attention: If your system root filesystem does not have sufficient capacity to store this information, either increase capacity, or create a dedicated filesystem with sufficient size mounted to /etc/adsm/SpaceMan/candidatesPools. Kill all running dsmscoutd processes. Create and mount the dedicated file system. Restart the scout daemon manually, using the dsmscoutd command.

For a file to be eligible for automatic migration, it must:

The HSM client prioritizes files for migration based on the number of days since they were last accessed, their size, and the age and size factors that you set for your file systems. The age factor determines how much consideration is given to the number of days since a file was last accessed, and the size factor determines how much consideration is given to the size of a file.


Migrating Selected Files

Attention: On large file systems, selective migration can take a while.

Use selective migration either from menus or from the command line to move specific files from your local file systems to storage. For example, if you know that you will not be using a particular group of files for an extended time, you can migrate them to storage to free additional space on your local file system. According to the space management options and settings that you select, the HSM client migrates files that are eligible for selective migration.

When you migrate a file selectively, the access time (atime) for the file does not change.

The number of days since you last accessed a file has no effect on whether your file is eligible for selective migration. This is different from automatic migration. An eligible file must meet the following management class requirements:

Migrate Selected Files

To migrate selected files from your local file system to storage:

  1. Click Selective Migration in the Hierarchical Storage Management window. The Selective Migration window displays.
  2. Click Select File Systems in the Selective Migration window to select a file system.
  3. Click List Selection in the Selective Migration window to view a list of the selected files.
  4. Select the file systems that contain the files you want to migrate in the Select File Systems for Selective Migration window.
  5. Click the Select button. A directory tree of the file systems that you selected displays in the Selective Migration window.
  6. Select the files that you want to migrate using one of the following methods:
  7. Click Migrate. The Selective Migration Status window displays. To stop migrating files, click Stop at the bottom of the window. If you click Stop, any files already migrated remain migrated.

    If you set the tapeprompt option to yes in your dsm.opt file, and the destination for a migrated file is a storage pool that consists of removable media, such as tape, you are prompted to wait for the medium to mount, or to skip the file. See "Assign Management Classes to your Files" for information about management class attributes.

  8. Click OK to close the Information Dialog box after you receive confirmation that all files migrated successfully.
  9. Click Return to return to the Selective Migration window.
  10. Click Cancel to close the Selective Migration window.

Start Selective Migration With the dsmmigrate Command

To migrate one or more files to storage from the command line, use the dsmmigrate command. For example, to migrate a file named proj1rpt from the /home/proja directory, enter:

   dsmmigrate /home/proja/proj1rpt

To display information about your migrated files, use the -detail parameter with the dsmmigrate command.

To migrate files in any subdirectory below the specified directory that matches the file specification, use the -recursive parameter. For example, to migrate all files in a directory named /migfs2/test/dir1 and in all of its subdirectories, enter:

   dsmmigrate -R /migfs2/test/dir1

See dsmmigrate for more information about this command.


[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]