• IMARC Comprehensive Review
• IMARC Report
• IMARC Summary
• IMARC Data Score
• IMARC Data Review Process

IMARC^™ — Independent Metrics for the Assessment of Risk from Catastrophes

Recent catastrophe events, such as Hurricane Katrina, revealed shortcomings in the catastrophe models and significant deficiencies in the exposure data being supplied to the models. It has become clear that many companies do not understand the uncertainty inherent in the models and how to use the models in light of this uncertainty. There have been no real standards or established best practices for using these highly complex tools, yet companies are relying on the models to make very important financial decisions.

Rating agencies, regulators, and even the policyholders are taking a much more active interest in the models and how they are being used. There is growing interest in finding out not just which models companies are using and what the models say, but how companies are using the models and ensuring the integrity of the data going into the models. Over the next several years, there will be even more scrutiny of the entire modeling process and risk assessment process.

Karen Clark & Company is uniquely qualified to help companies develop better and more holistic risk assessment and management processes. A comprehensive risk management process incorporates independent benchmarks and metrics to assess the quality of catastrophe model input and output.

IMARC® Comprehensive Review

The IMARC Comprehensive Review covers the important components of a catastrophe risk management process that conforms to best practices. The five most important components are:

• Preparing the exposure data
• Benchmarking the exposure data
• Performing the model analyses
• Benchmarking the model-generated loss estimates
• Applying the model results

Through the review process Karen Clark & Company consultants help companies gauge the integrity of the information they are relying on for important financial decision-making.

IMARC® Report

The IMARC Report is a key deliverable of the IMARC Comprehensive Review. The IMARC Report provides:

• Documentation of the internal risk management process
• An assessment of each component in the process including data quality
• Recommendations for process improvements

The IMARC Report helps companies understand how their exposure data and catastrophe risk management processes stack up relative to peer companies.

IMARC® Summary

The IMARC Summary is an abbreviated report suitable for external stakeholders. The IMARC Summary allows outside entities to better differentiate companies based on their knowledge and use of the catastrophe models and model results.

IMARC® Data Score

The IMARC® Data Score provides a consistent measure of exposure data quality that is comparable across companies. While the data attributes and weights that go into the scoring algorithm vary by company according to covered perils, geographical area, property lines and types of business, the score itself reflects the uncertainty characteristics of the data in a consistent fashion across companies. The scores can range from 300 to 800 in increments of 25.

	Exceptional companies have established data processes geared to the collection and verification of the data attributes that have a significant impact on catastrophe losses. There are many checks and balances in their systems and no process gaps. All or almost all of the important data attributes for the nature of business underwritten are complete and accurate. These data attributes are transformed and formatted correctly for the catastrophe models. The insured properties comply with underwriting guidelines as verified by onsite review and field surveys. The exposure data should contribute no more than 20 percent to the uncertainty in the catastrophe model estimates. Likely percent of US companies meeting these criteria: ~2
	Companies earning an IMARC® Data Score in the Excellent range have established processes geared to the collection and verification of the data attributes that have a significant impact on catastrophe losses. Most of the important data attributes for the nature of the business underwritten are relatively complete and accurate. These data attributes are transformed and formatted correctly for the catastrophe models. The insured properties comply with underwriting guidelines as verified by onsite review and field surveys. The exposure data should contribute no more than 35 percent to the uncertainty in the catastrophe model estimates. Likely percent of US companies meeting these criteria: ~13
	Companies earning an IMARC® Data Score in the Good range collect the most essential information on the insured properties, typically at least seven to eight attributes that are the key drivers of catastrophe losses. Internal processes include verification of these data attributes along with other exposure data checks and balances. These data attributes are reasonably complete and accurate for the majority of insured properties. The exposure data should contribute no more than 50 percent to the uncertainty in the catastrophe model estimates. Likely percent of US companies meeting these criteria: ~35
	Companies earning an IMARC® Data Score in the Fair range collect the data attributes that are the most essential for estimating catastrophe losses but there are significant deficiencies in one or more these attributes. Internal processes need improvement in terms of data collection, verification, transformation and/or checks and balances. The exposure data could contribute up to 100 percent to the uncertainty in the catastrophe model estimates. Likely percent of US companies meeting these criteria: ~35
	Companies earning an IMARC® Data Score in the Poor range have significant process and data issues that could result in catastrophe model estimates being inaccurate by over 100 percent. Likely percent of US companies meeting these criteria: ~15

IMARC® Data Review Process

The IMARC® Data Review is independent of any particular catastrophe model. The review process is applicable to the exposure data elements used to assess the catastrophe loss potential for all regions, perils and vendor models. It is a standard process that results in an exposure data score providing external stakeholders with a consistent and comparable measure of exposure data quality across companies.

The first component of the IMARC® Data Review is on-site evaluation of company exposure data processes. There are significant differences between insurance companies in terms of how much information they collect on their insured properties, the processes used to collect this information, and the completeness and accuracy of this information. No company has totally complete and accurate data, and no two companies are the same with respect to internal processes surrounding the exposure data. These differences translate directly into differences in the quality of the catastrophe model-generated loss estimates.

In the typical insurance company, exposure data on individual policies travels through several different computer systems and data processing steps. Certain data elements are dropped, some are transformed and others are added at various transition points from the time the data is first collected until it finally ends up in the catastrophe models. This complex, multi-step process provides many opportunities for data to be lost or corrupted and for other data errors to enter the process. How frequently and thoroughly exposure data is checked and verified as it flows from one transition point to another is another difference among insurance companies.

To add to the complexity, a typical insurance company has multiple lines and types of business for which internal data processes can and do differ. The IMARC® Data Review examines the life cycle of the information for each type of policy as it moves from point to point in the company's internal processes. Transition points are reviewed for checks and balances. Automated and manual data verification steps are tested, and documentation is reviewed.

Preparing the exposure data files for the catastrophe models usually entails extracting data from multiple computer systems, combining multiple data files and then mapping company internal codes to the codes required by the catastrophe models. Each insurance company has its own processes for performing these functions. These processes are error-prone. In addition, the mapping of construction, occupancy and other codes is a subjective process, frequently left to the interpretation of non-experts, and highly likely to differ between companies. The IMARC® Data Review can uncover processes that can lead to artificially high or low model-generated loss estimates because of erroneous or inconsistent mappings.

Along with the company's internal data processes, the second component of the IMARC® Data Review includes extensive evaluation of the exposure data collected by the insurance company. All of the exposure data elements provided to the catastrophe models are tested for resolution, completeness and accuracy. These tests utilize independent data sources, methodologies, and actual engineering surveys of the insured properties.

Karen Clark & Company utilizes dozens of independent data sources and models to benchmark the quality of the data. External databases include census data, tax assessor data, and construction cost data. Since no one data source or combination of data sources is complete and totally accurate, Karen Clark & Company also performs engineering surveys of samples of insured properties and statistical sampling of the data collected from the process review. Independent estimates of replacement values of buildings, contents and time element exposures, each of which are key data elements for the catastrophe models, are based on construction and engineering expertise and up to date knowledge of local conditions.

The third component of the review process is real-time verification of the company's processes and data. Karen Clark & Company consultants evaluate company and third-party data collection systems and processes in real-time, to independently verify if data quality controls and procedures are being implemented. The diagram above illustrates the IMARC® Data Review process.

IMARC® Data Scoring Algorithm

Karen Clark & Company employs a detailed data scoring algorithm that accounts for

• the data attributes relevant for the particular peril, geographical area and type of business,
• the resolution and completeness of valid information for each data attribute, and
• the accuracy of each data attribute.

The data attributes are weighted according to their importance in the catastrophe loss estimating process. Over 100 process and data elements are used in the scoring algorithm. The scoring algorithm produces the IMARC® Data Score, a quantitative measure of quality and uncertainty characteristics of the exposure data used to generate the catastrophe loss estimates.

Each data attribute is scored for completeness and accuracy. For many attributes important in determining loss, there is no complete "truth" data set, which necessarily creates uncertainty around the true accuracy of the data attributes. Karen Clark & Company quantifies this uncertainty using statistical samples of data from the benchmark data sets, the company internal processes, and the engineering surveys. A weight is applied to each attribute according to how important that attribute is in determining the loss from a particular peril. The weights vary by geographical area. For example, year of construction is an important attribute for Florida hurricane loss estimates since the building codes changed significantly in certain years. In other geographical areas, this building attribute is less important. Different vendor catastrophe models incorporate different assumptions as to the importance of the property attributes. Despite all of the claims data available from recent events, there is not enough data on many data attributes to quantify their precise impacts on catastrophe losses. This means there is uncertainty around the weights and the degree of uncertainty differs between the data attributes. This is accounted for in the IMARC® Data Score.