Does your organization have a data architecture function, and if so, how do they add value?
Sort By:
Oldest
Director of Data Architecture in Mediaa year ago
-- Incoming long comment -- Yes, we have a data architecture department.
Questions like, "Platform A is better than Platform B", "Platform A is easy to deploy", and "It's just a platform", these sentences are a double-edged sword in the world of data. More than ever, it is easy to integrate most of the tools out there, pulling terabytes of data from multiple sources, systems, and formats with a simple UI/function to build a service, dashboard, or product. But why is it that many companies aren't as agile with their data stack and their strategy? what is the role of architect in this ?
Although there has been continuous uberization of the stack capabilities to make it less monolithic. ? It is simple: Some of these platforms and solutions are deployed in their own "value" silos, some leak operational expenses upstream/downstream, some are shelf-ware used once and not generating value, and others have been inflammatory elements with their tech debt and, more importantly, draining teams' morale.
There might be several reasons for each of the above scenarios. Perhaps one of them is we need to pay less attention to the buzzwords associated with the "modern stack" and bring back the boring principles of architecture, modelling and strategic measurable planning.
We all remember back in 2012 when Davenport and Patil announced data science was "the sexiest job in the 21st century". Though it may still be the case in 2022, but Data Architects are now the new rising stars in the industry who are bridging the gap between micro-level business architecture modelling and macro-level data enterprise architecture.
Since the architecture needs to address a number of complex aspects of an organization's strategy, the role of the data architect has become strategic. Business architecture artifacts such as Data models not only serve as design artifacts for storage, integration, and data quality, but also influence the technical architecture of a solution, platform, or technology selection. The ability to replace small components -reasonably - instead of dismantling the entire architecture can be crucial for some strategic decisions.
Furthermore, modular architectures must be designed to accommodate future frameworks and architectural methodologies. It can't be isolated solutions deployment because architects address inevitable costs, optimizations, and business agility challenges. Apart from designing the architecture blueprint, they ensure that strategies are implemented and, most importantly, that their success is assessed and measured. Architects are the guardians of the data strategy.
In general, an organization data architecture comprise of two pillars : Business data architecture an technical data architectures
Technical data architecture involves the design and development of scalable data products, solutions, and reporting services. This includes the design of data warehouses, Business Intelligence, Data Integration, Reporting/ML, Big Data Framework, and data flow. It is essential to be current with the data technologies and data stack landscape in order to carry out this practice.
While Business Data Architecture consists of policies, standards, business data models, business glossaries, use cases, business capability modelling, alignment between data and organizational capabilities, process/workflow models, ontology models, data cataloging, data lineage, improving efficiency, & reduce risk. Data architects sometimes turn to Enterprise Architecture frameworks such as TOGAF.
Integrating Data Architecture with the process, application and solution architecture is the responsibility of Data Architects. The question they ask is, what, where, and how is the data created, as well as where it is used. Where should it be created and how should it be stored? How is it moving? how is it transforming? Where is distributed? how is it hopping from one layer to another? It is possible for data architects to spend very long-time designing frameworks without even thinking about any technology or vendor.
Having both skill sets in the team facilitates a smooth organization data transformation.
1) They ensure to design the data flow and where data is stored and how they are accessed.
2) They oversee the data infrastructure such as Data warehouse and data lake.
3) They are responsible to ensure the data quality and integrity.
4) For digital initiatives, the team is responsible for the data preparation for such initiatives
5) The team is responsible for ensuring the Data Governance is in place and the roles is clearly defined for data owners and data stewards.
In addition, the team also plays a vital role in assessing the impact and dependence of data related laws and regulation.