Big data lacks standards
Poor data quality and lack of standardisation is holding back the financial sector's ability to effectively analyse and reap value from big data.A study carried out by the Centre for International Finance and Regulation (CIFR) found that one single corporate entity had been described in 140 different ways (accounting for issues such as spelling errors and incomplete descriptions) on just one registry database, making it almost impossible to properly analyse all the data associated with that entity.That lack of standardisation is hampering Australia's ability to develop a better evidence base on which to construct financial policy, according to the CIFR.Dr Kingsley Jones, CIFR research fellow, speaking at its big data seminar in Sydney this week warned that it was important to address the issue of data quality urgently, noting that: "In a fast moving world that's in overdrive you can't have a regulatory capability that falls far behind the pace of change."The CIFR has identified data quality and lack of standards as a potential stumbling block in terms of Australia's ability to mine growing reserves of financial data for insights that can be used to either develop strategy or inform policy. Some sectors are particularly ill prepared, according to Professor David Gallagher, CEO of the CIFR, who cited the funds management sector as a case in point. "And that's very disappointing when the industry is the fourth largest in the world," he added.In its recent submission to the Financial Services Inquiry, the CIFR recommended an audit of current data architectures deployed in organisations such as the ATO, RBA, APRA, ASIC and ACCC as an important first step toward creating an accessible repository of standard (if anonymised) data that could be used as an evidence base to better inform policy and decision making across the sector.Dr Jones said that the time was ripe to make a more serious attempt to settle on data standards and processes, given the enthusiasm in the sector for big data. He warned that, without proper standards and improved information integrity, many data analytics efforts were doomed to fail.