How does your organization prioritize data governance and data management practices in the context of AI implementation? Compared to other data and analytics projects, do you take additional steps to ensure data quality, integrity, and accessibility for AI projects?
Sort By:
Oldest
Director of Dataa year ago
I can't comment on our organisation but a colleague recently showed me something that the Canadian government has created called an Algorithmic Impact Assessment tool - as a free checklist for AI/automated decision making that covers these aspects along with impacts and mitigaions that I think is very thorough and could be used by organisations: Algorithmic Impact Assessment Tool - Canada.ca Chief Data Officer in Travel and Hospitalitya year ago
Any use of data for enterprise reporting, decision making, or decision automation, should all demand the same rigor and responsibility regarding the creation, manipulation, management, and distribution of data. We have well known and proven methods for governing numeric/categorical data. These apply to equally to "business intelligence" applications as they do to "artificial intelligence" applications. The area that will now demand new methods and considerations is Generative AI which consumes and produces language. Monitoring for bias, demonstrating accuracy, and providing explainability for these deep learning language models will require an expanded understanding of AI (specifically LLMs) and new approaches to demonstrate responsible use.
Director of Data Architecture in Mediaa year ago
In the context of GenAI, the solution lies in holistic and proactive framework that its guardrails baked into the full lifecycle of the product.Upon capturing the use cases, the guardrails starts with the architecture and engineering practices:
🔻 Using frameworks like App Development and DevOps, rigorously testing for biases can help. But we need to go further. Monitoring prompts, controlling experiments, and the ability to "hot swap" models is critical.
🔻When selecting the model, carefully evaluate its strengths and limitations for the intended uses, considering factors like cost, speed, accuracy, data privacy, and potential risks/biases.
🔻Design for transparency, ethics and rapid iteration from the start enables agility and risk reduction. Having the right data and information architecture also allows relational context to improve accuracy.
🔻Prompt engineering is also key - develop and test templates carefully, manage token limits, and monitor for misuse.
🔻Continuously evaluate other ethical considerations like bias, governance, fairness, hallucinations, and societal impacts.
🔻Engage with stakeholders throughout the process.
In summary, a proactive, vigilant, and responsible approach to developing, deploying and managing GenAI products can help maximize benefits while minimizing risks. Ongoing monitoring and adaptation will also be crucial as this rapidly evolving technology continues maturing.
Quality Assurance: Implement rigorous data quality checks to maintain high-quality datasets.
Data Catalog: Maintain a centralized data catalog for accessibility and traceability.
Privacy Compliance: Ensure AI models adhere to data privacy regulations like GDPR.
Ethical Guidelines: Develop and adhere to ethical AI guidelines to prevent biases.
Continuous Monitoring: Establish ongoing monitoring and updates for data and AI models.
In comparison to other projects, we invest extra efforts to safeguard data integrity, quality, and accessibility, recognizing AI's sensitivity to data input.