Categories Software

Continued from Types of Data Warehouses

Although a data warehouse sounds like a single entity, it is really a multitiered, multiapplication conglomerate that comprises several components. Each component may be handled by one or more pieces of hardware or software. No vendor has a complete data warehouse package.

Functionally, a data warehouse extracts data from operational systems and loads it into a holding area where it is "scrubbed" (that is, made to conform with warehouse standards), merged, time-stamped or dated in some way, and loaded into databases for use by data access tools.

Since data goes through a number of transformations and is ultimately placed in data structures different from the ones it came from, those changes are mapped in catalogs or dictionaries. Such catalogs are managed with metadata tools. Data that defines or describes data in the warehouse is called metadata. There are typically two kinds of metadata. Things that users need to know, such as table and column names and definitions, are called frontend metadata. Everything else, such as how a particular data element maps to its original database, is backend metadata.

Design and management tools are also important components of a data warehouse. Even though it contains nonvolatile data, a data warehouse is far from static; you can't just build one and forget about it. A data warehouse is a high-maintenance system that expands as the company's demands on it increase. Even if the design of the warehouse itself changes little, the designs of the production systems that feed it are likely to change. These changes must be captured in the data warehouse metadata. Unlike an OLTP system, which gives a snapshot of the present, a data warehouse is meant to provide a complete picture of the data over time. This means that whenever any data structure in a production system changes, you have to capture both the change and when it occurred in the warehouse; otherwise queries against historical data may yield peculiar results.

Next: Practical Considerations

Published as Enterprise Computing in the 3/9/99 issue of PC Magazine.

Related Links
• ZDNet Enterprise