During February and March 2022, IntoZetta contacted its existing software and services clients, and organisations from the broader social housing sector, and asked them to complete our annual housing data survey. The survey was designed to gather opinions and stimulate debate on the current use of data within housing providers, the common issues faced managing that data, and the specific challenges that those create.
We published the results from the first four questions in our survey in the previous (May 2022) edition of Housing Technology; as promised, we are now publishing our findings from the next four questions. The results provide an interesting perspective on data challenges across the sector, highlighting the areas where a common perspective exists, and those where differing opinions have been shared.
The respondents all agreed or strongly agreed that IT transformation would continue on a significant scale within the social housing sector for years to come. This result was unsurprising given the industry wide factors that are fuelling transformation. The Social Housing sector is continuing the journey towards a smaller number of larger housing providers, which on the one hand promises economies of scale, but on the other makes transformation and rationalisation of the system landscape necessary.
More specifically, data transformation is being driven by the need for better insights and more accurate data sources. The Building Safety Bill and other legislation will drive the need for asset data that is complete, accurate, and available – an outcome which will require significant effort and investments for the majority of social housing providers.
This question split the respondents almost down the middle, with almost the same number agreeing (45 per cent) as disagreeing (41 per cent) with the statement. This question relates to the management of data within the respondents’ organisations, so the responses might indicate differences in the value that organisations attach to data, and it may also indicate that different types of data are managed differently within the same organisation.
If the social housing sector wants to manage its data assets in the same way as its built assets, it needs to understand where all data assets exist, what they are used for, how and when they were created, and the investment that is needed to maintain acceptable quality. While some organisations that IntoZetta works with may be already travelling down this road, others are only just beginning to plan their journeys.
Half of the respondents felt that their organisation had a good understanding of where data is created, stored and used, but a similar percentage of respondents weren’t so confident.
The creation and storage of data is accelerating across all sectors, and this leads to challenges for IT departments which are usually trying to avoid the duplication of data, the storage of unnecessary data, and data held beyond their core applications.
Understanding and documenting ‘data lineage’ across an organisation can be a time consuming and difficult task, but tools do exist that can accelerate this process, and allow this often-complex map to be documented in a consumable format to support activities such as planning and change-impact analysis.
Understanding the use of data can be a little more challenging than it first appears. IntoZetta consultants often work with housing clients to create a ‘data fingerprint’ for key processes across the organisation. Understanding the data that fuels each process can be an important step in understanding which data is used most widely, supports the greatest volume of key activities and teams, and therefore has the most value to the organisation. It can also help to identify how long data should be retained, and which data has little usage and/or utility and can therefore be archived or deleted.
Half of respondents felt that their organisation didn’t have a single trusted source of important data.
Having supported many data migration projects in the housing sector, IntoZetta consultants have first-hand experience of the fragmentation and proliferation of data within housing providers’ core business applications, as well as large volumes of local databases, spreadsheets and other documents.
The creation of a single trusted ‘golden asset record’ from fragmented data sources of varying quality is often the most technically-complex part of any data migration project. Identifying accurate data and trusted sources can be a laborious process which needs technical expertise as well as the specialist knowledge that may only exist in the heads of a very small number of employees.
Several of IntoZetta’s housing clients are undertaking transformation programmes where the creation of a single trusted data source is an important outcome.
Resolving historic data management issues is only part of the solution, and a key step in the design and implementation of the new system should be a thorough analysis of the things that were wrong with its predecessor.
Why has data quality degraded across certain data entities? Why has data become fragmented across several sources? And why have so many teams and employees felt the need to create their own solutions outside the core system landscape? These are just some of the questions that should be asked to help avoid data management history repeating itself.
In the September 2022 edition of Housing Technology, IntoZetta will be discussing the responses to questions about the management of data held in documents and shared drives, and the challenges of managing data quality.
If you would like to be included in our next survey, and share your experiences of working in our sector, please contact jackie.green@intozetta.com.
Dan Yarnold is a director of IntoZetta.
