When Data Drives Your Business
  GO
Contact Us 888-828-8201

 
 The Aginity Blog

The quality control subsystem typically executes during the data processing phase of the data factory. It is separated logically because it is managed differently and employs different types of processes. The most significant data quality issues are addressed in the data acquisition phase of the data factory. The quality control phase caters to the serious yet more subtle issues of data integrity and data pattern anomalies.

Posted by: Dan Kuhn - CTO on 9/1/2009 | 0 Comments

In a typical data factory environment, the data processing subsystem looks the most like a traditional data warehouse. This is where data from all sources get ground into a single version of the truth. It generally consists of a staging database where raw data extracts and data acquisition files are stored. The database is then transformed and loaded into an appropriately designed data model.  ETL platforms, as their name (Extract, Transform, Load) suggests, provide a number of tools that automate this process and the better ones also include facilities for lifecycle management, version control and error checking.  But even with these ETL tools, there is a fair amount of manual scripting that needs to be performed and maintained. 

 



  • Syndicate    
     

    Recent Posts

    Archive

    Bloggers

    Category List

    Tag Cloud

       


    MapReduce Clickstream Response Attribution
    Java AP Basket
    MapReduce Keyword Tokenization
    Interactive Reporting Patterns

    This Content Requires Adobe Flash Player | Download Now

    This Content Requires Adobe Flash Player | Download Now

    This Content Requires Adobe Flash Player | Download Now

    This Content Requires Adobe Flash Player | Download Now

    This Content Requires Adobe Flash Player | Download Now


    Privacy Statement  |  Terms Of Use  |  Copyright 2010 by Aginity, Inc. Register   |   Login