Proving Your Data Is Ready: A Framework for Prototyping Performance Measurement

In our previous article, Blinded by Pageantry: the Perils of Neglecting Performance Data Readiness, we highlighted the advantages of scrutinizing input data fitness and doing so early in a performance transformation initiative. We also presented the Meradia Performance Data Readiness Assessment (PDRA)—a robust, ready-to-use methodology, condensing decades of experience into an efficient and effective tool for identifying and remediating performance data gaps, avoiding surprises that would otherwise cause major setbacks to timelines and budgets.

Yet even after a thorough PDRA scrubbing, certain types of issues are likely to remain undetected. The PDRA examines a complete set of data elements individually, yet simple inspection alone is unlikely to spot inconsistencies in the complex, temporal and internal relationships among the various input data sources and the systems that generate them. Unit breaks, cash breaks, inconsistent sign conventions, misspecified market values, stale pricing, missing positions, missing transactions, duplicate security identifiers, unspecified reflexive flow identifiers: these and other potholes lurk in the interstices between position, transaction, market, analytic, and security reference data.

Furthermore, issues like these tend to occur with low frequency, yet have a high (negative) impact on performance accuracy. They’re devilishly hard to find in a limited data sample—it takes a broader and more robust test: multiple accounts, covering a complete range of asset types, transaction types, and investment strategies, over consecutive historical months. Such an analysis needs to be automated and thorough—testing every position and transaction—to surface such problems.

Meradia’s Performance Prototype takes a step beyond the PDRA, embodying our repeated experience that the most intractable performance data problems won’t be discovered until you calculate performance. The prototype is a fully functional and configurable calculation and storage engine—ready to deploy at the client’s site. It incorporates the same performance data model specified in the PDRA—to boot it up requires only the staging, loading, and mapping of client historical test data.

The prototype performs not only a robust set but a myriad of automated data diagnostics—by account, by position, by day—over the entire supplied test set. Where are the return and contribution outliers, and what flows are associated with them? Are there transactions for a given day when no position was supplied, or vice versa? Are prices missing, stale, or dummy? Do positions have security IDs not found in the security reference? Do yesterday’s shares, plus today’s transactions, equal today’s shares? Are back-valuations and back-dated transactions being consistently supplied?

As a bonus, the Prototype’s practical benefits often extend well beyond initial data analysis:

• Prototype results can be shared with performance consumers and stakeholders, to develop and confirm up-front consensus around the calculation requirements and methodology. Are we calculating the right thing? Are we calculating the thing right?
• Implementation streams can be supplied with prototype output—e., a set of expected results—thereby facilitating efficient, detailed, accurate testing, and user acceptance over a new calculation process.
• In phased implementation programs, the prototype can live beyond Day-One, and provide a sandbox for modeling and obtaining feedback on requirements and functionality in subsequent phases.

HOW MERADIA CAN HELP

The Meradia Performance Prototype is the ultimate solution for finding infrequent, high-impact data issues efficiently and early in a performance transformation initiative, and for facilitating their timely remediation. Paired with the PDRA, it represents a rapidly deployable and highly effective tool for ensuring the on-time, on-budget success of a performance transformation project, by minimizing “gotchas” that lurk beneath the input data’s surface. Prototyping for the Win!

Download Thought Leadership ArticleServices: Solution DesignExpertises: Performance, Risk & AnalyticsClients: Asset Managers, Asset OwnerAuthors: Meradia Consultant