Implementation

The story of standardizing reference and market data feeds began in late 2008, when investment management firms were struggling with an increasingly complex web of data sources. Each vendor - Bloomberg, Reuters, IDC, Telekurs, SIX Valordata, and Copp Clark - spoke their own distinct “language,” creating a Tower of Babel scenario in the world of financial data.

Our mission was ambitious: create a universal translator for financial reference and market datasets that would serve as the foundation for the next generation of Securities Reference Manager (SRM) software. This wasn’t just about building interfaces; it was about reimagining how the industry handled creation of security master file.

The Data Universe

The project encompassed crucial financial data categories:

  • Equities: From basic share information to complex corporate ownership structures
  • Fixed Income: Complete bond characteristics, payment schedules, and covenant details
  • Futures/Options: Contract specifications, expiration schedules, and underlying relationships
  • Pricing: Multiple price types, ranging from closing prices to complex derivative valuations
  • Security Analytics: Risk metrics, duration calculations, and yield analytics
  • FX Rates: Spot rates, forwards, and cross-rate calculations
  • Ratings: Agency ratings, outlook information, and rating history
  • Benchmarks: Index composition, weights, and rebalancing information
  • Corporate Actions: Dividends, splits, mergers, and other corporate events

The Innovation

The breakthrough came in the form of the “Adaptive Feed Interface” (AFI) framework, which introduced several revolutionary concepts:

  1. Universal Data Dictionary
    • Created standardized definitions for 800+ data points
    • Developed vendor-specific translation maps
    • Implemented automated data quality rules
    • Established hierarchical relationships between data elements
  2. Smart Mapping Engine
    • Designed intelligent field mapping algorithms
    • Built cross-reference resolution system
    • Created data lineage tracking
    • Implemented override management
  3. Composite Engine
    • Developed priority-based data consolidation
    • Created conflict resolution rules
    • Implemented data quality scoring
    • Built audit trail system

Technical Architecture

The system architecture was built on three core principles:

  1. Modularity
    • Vendor-specific intake modules
    • Standardized transformation layer
    • Configurable output formatters
    • Pluggable validation rules
  2. Scalability
    • Parallel processing capabilities
    • Multi-threaded data transformation
    • Efficient memory management
    • Optimized database operations
  3. Flexibility
    • Configurable data mappings
    • Extensible data model
    • Custom validation rules
    • Adaptable output formats

The Transformation

By Q2 2009, the system was processing:

  • 80+ distinct files across all vendors
  • 800+ unique data points
  • Millions of securities
  • Daily updates across all asset classes
  • Real-time corporate action notifications and processing

The real innovation lay in the “Smart Loader” technology that could:

  • Auto-detect changed file layouts
  • Suggest new field mappings
  • Identify day over day data anomalies
  • Generate detailed mapping documentation and logs
  • Create validation rules based on observered patterns

Client Impact

The standardization initiative transformed client’s client onboarding process :

  • Reduced implementation time from months to weeks
  • Decreased bespoke data mapping effort by 70%
  • Improved data quality by 45%
  • Enabled rapid vendor additions, plug and play mechanics
  • Simplified ongoing maintenance and updates

Vendor Collaboration

Working closely with major data providers yielded unexpected benefits:

  • Bloomberg: Enhanced fixed income analytics integration
  • Reuters: Improved real-time pricing feeds
  • IDC: Expanded corporate action coverage
  • Telekurs: Advanced derivative instrument support
  • SIX Valordata: Enhanced European market coverage
  • Copp Clark: Improved global coverage for holidays data

Industry Innovation

The project’s impact extended beyond immediate client benefits:

  • Established new industry standards for data integration
  • Created reusable mapping templates
  • Developed best practices for data quality
  • Introduced innovative data governance concepts
  • Set new benchmarks for vendor integration

This groundbreaking initiative not only revolutionized how the investment management software company handled reference and market data but also set new standards for the entire industry across all vendors, demonstrating how thoughtful standardization can drive innovation and efficiency across the financial sector.