E19: Responsible AI, Boardroom Strategy & Women in AI Leadership with Zahra Shah

From Fintech to Frontier Technology

Zahra's career began at Accenture, where she spent more than a decade working primarily on AI and machine-learning applications within financial services and capital markets. Much of this work focused on regulatory compliance, including areas such as KYC, Basel, MiFID and cross-border sanctions. This gave her an early view of how highly regulated organisations were approaching emerging technology.

Investment banks and capital markets firms were, in many respects, early adopters of AI and frontier technologies. However, adoption was not always straightforward. The technology could be expensive and error-prone, while getting employees properly trained and comfortable with new systems remained a significant challenge.

Since then, Zahra has moved through entrepreneurship, charity leadership, investment and board roles, developing a particular focus on the intersection between AI, privacy, governance and regulation.

Connecting AI Adoption with Business Strategy

Today, Zahra is heavily involved in AI transformation consultancy, particularly through Nexa, where she works with organisations on AI solutions and software transformation. A central principle of her approach is that organisations should not adopt AI simply because everyone else is doing it. She describes this as the problem of “FOMO” — companies wanting to use AI because competitors are doing so or because it is perceived as the next big thing.

Instead, AI strategy should begin with the business itself. The starting point should therefore be a genuine business problem. If that problem can be solved more effectively without AI, then AI may not be the right solution. This approach also helps organisations identify use cases that are genuinely relevant to their own circumstances rather than copying what another company is doing.

Moving Beyond the Administrative Burden

For Zahra, one of the most exciting aspects of AI is its ability to remove repetitive and mundane work. Many employees spend significant amounts of time working through spreadsheets, administrative processes and large quantities of information. While these tasks may be necessary, they can prevent people from focusing on activities where their expertise creates greater value.

AI can change that balance by taking on some of the more repetitive work and allowing employees to concentrate on strategic or creative tasks. The opportunity, she argues, is to use technology to ensure that “people” can focus on the areas where they can genuinely “generate value”.

This does not necessarily mean replacing employees. Instead, it can mean changing what their time is spent on.

Starting with Low-Risk AI Use Cases

For organisations that are uncertain about where to begin, Zahra recommends starting with relatively low-risk internal applications. One example she gives is a legal department with thousands of documents that had accumulated over many years. Finding the relevant information for a particular case could take days because the documents were not stored in an effective central repository.

A knowledge assistant transformed this process. Instead of manually searching through files, employees could use a conversational interface to find relevant cases and documents using keywords and prompts.

The important point was accessibility: employees did not need extensive technical knowledge to use the system. For Zahra, this type of internal deployment can provide organisations with an opportunity to test AI in a controlled environment before exposing it to customers.

Building Confidence Before Going Customer-Facing

Moving an AI system directly into a customer-facing environment can feel significantly more risky. There are questions around privacy, security, proprietary information and whether the system might provide customers with information they are not authorised to access.

Zahra therefore advocates testing AI internally first. Once employees have used and evaluated the system, organisations can gain greater confidence in its performance and understand where additional safeguards are required.

The same knowledge-assistant concept can then potentially be extended to customers, but with appropriate guardrails determining exactly what information they can access. This creates a gradual path towards adoption rather than expecting businesses to make a large technological leap immediately.

Governance as a Foundation for AI Adoption

Responsible AI sits at the centre of Zahra's approach. Before implementing a specific tool, she recommends starting with a strategic workshop that establishes the organisation's AI strategy and considers responsible AI principles from the outset. This includes areas such as transparency, explainability, data privacy, security and accountability.

For larger organisations, Zahra suggests establishing an AI board to oversee these issues. Smaller businesses may not have the resources for a formal board, but they can still designate an individual or small group to take responsibility. The important thing is that someone is accountable.That person can work alongside internal subject-matter experts or external AI specialists to develop the appropriate policies, processes and safeguards.

The Growing Importance of Explainable AI

As organisations increasingly rely on AI to support decisions, understanding how those decisions are made becomes increasingly important. Zahra highlights explainability as a particularly important area, especially when AI systems operate in ways that can appear like a “black box”.

If an organisation cannot explain why an AI system reached a particular conclusion, it becomes much harder to challenge mistakes, identify bias or build confidence among users. This is particularly important for policymakers and regulators, who need to understand the systems they are being asked to oversee.

For smaller companies competing against major technology brands, demonstrating transparency and accountability can also become a way of building trust.

Responsible AI Beyond the Big Technology Brands

Large companies such as IBM benefit from decades of reputation and established relationships with customers. For smaller businesses, trust has to be built differently.

Zahra's answer is to demonstrate that responsible AI is embedded into the organisation's approach from the beginning. That means having clear principles, accountability structures and processes around privacy, security, transparency and explainability. The objective is to show customers that responsible AI is not simply a marketing statement, but something that is actively built into how the business operates.

Addressing Bias Through Greater Diversity

Zahra's work with UKAI's Women in AI working group reflects another important part of responsible AI: ensuring that the people designing these systems represent the people affected by them. She highlights research showing that bias in AI can emerge partly because the teams developing systems are not sufficiently diverse.

The consequences can be particularly serious in areas such as recruitment, financial services and healthcare. For example, AI-powered recruitment systems can potentially disadvantage female candidates if the data or assumptions underlying the model contain existing biases. Similar concerns arise when AI is used in financial decision-making, where biased systems could affect access to loans or credit.

Healthcare presents another major challenge. Zahra points to the historical underrepresentation of women in clinical trials, meaning that some medical treatments have not been sufficiently tested across female populations. These examples demonstrate why diversity needs to be considered throughout the development and deployment of AI.

Creating a Women in AI Community

The UKAI Women in AI working group aims to provide a space for women working in, or looking to enter, the sector. Zahra sees it as a place where women can find support around challenges including bias, training, technology selection, policy and strategy. The initiative will involve targeted events, workshops, panel discussions and other opportunities for women from different industries to share experiences and develop practical solutions.

Importantly, Zahra emphasises that this should be a people-centric initiative. Rather than assuming what women in the sector need, the group aims to understand the challenges they are actually experiencing and develop initiatives around those needs.

The Importance of Male Allies

Addressing gender bias in AI cannot, however, be left solely to women. Tim highlights the importance of male allies taking responsibility for helping to address the imbalance, and Zahra agrees.

The argument for greater gender diversity is also not purely ethical. Zahra points to research suggesting that gender-diverse teams can deliver stronger financial outcomes, providing a business case alongside the wider social argument. For AI systems that are increasingly influencing people's lives, ensuring that different perspectives are represented is ultimately a question of better decision-making.

Building AI with Accountability at Its Core

Across Zahra's work, a consistent theme emerges: AI adoption should be deliberate rather than driven by hype. Businesses need to understand the problem they are trying to solve, identify the appropriate technology and build governance around it from the outset. That means thinking about privacy, security, bias, explainability and accountability alongside the technology itself.

For smaller businesses in particular, this does not necessarily require the resources of a large enterprise. What matters is establishing clear responsibility and creating processes that allow organisations to adopt AI in a controlled and transparent way.

At the same time, the industry needs to ensure that the people shaping AI reflect the diversity of those who will ultimately depend on it. For Zahra, responsible adoption and greater inclusion are therefore closely connected: building trust in AI requires not only better technology, but better governance and a wider range of voices shaping its future.