Considerations for Deploying Generative AI at scale

Coming back from Microsoft Ignite and reflecting on some of the announcements and conversations with partners, clients and Microsoft, as well as my own experience over the last year or so working with clients (and IBMers), it has become increasingly clear that Generative AI tools are both here to stay and that many organisations have concerns over how to roll out the experience across their respective organisations.

Microsoft has realised this and is attempting to add features to products like Purview to add the necessary controls (see AI Hub) to provide technical control to the source data that Copilot and Azure AI services have access to.

However, while important, data security is only one element that needs to be considered. Below are my thoughts on what organisations should consider before deploying Copilot (or other Generative AI tools) across them.

Data Security

Organisations have (in most cases) the tools to provide a baseline level of security across their M365-focused estate. Purview P1 is available for M365 E3 users and provides a key set of functions that would allow the monitoring and configuration of data, activity and permissions across the estate, when combined with some of the features available in Entra, you have the basis for an effective data security solution.

The challenge has always been around prioritisation, aligning budgets and resources for organisations to deploy these tools, and I suspect that up until now, other priorities have won over.

However, with the advent of Generative AI, this element of the organisation’s security posture needs to be revisited and, where required, strengthened to cope with the changes that Generative AI tools bring to the enterprise.

ROI

ROI can be a challenging metric to quantify, and I see many providers claiming that X% of efficiency savings have been realised by deploying Gen AI tools. However, there needs to be more transparency around how these figures have been validated/achieved.

As a result, it is essential that a clear and measurable set of KPI’s is defined and monitored to clearly understand whether the required tools are providing the benefits that the original business case was projecting.

At Ignite, Microsoft did announce Copilot Analytics (https://techcommunity.microsoft.com/blog/microsoftvivablog/introducing-copilot-analytics-to-measure-ai-impact-on-your-business/4301717), which is a starting point for clients to define and measure performance against these KPIs.

Talent & Adoption

Going hand in hand with the ROI point above, getting your users actually to use the solutions you deploy is paramount. Like many SaaS platforms, i.e. M365, I don’t believe that Gen AI solutions fall into the technical change bucket. Yes, there are things that you need to do from a tech perspective to enable and, more importantly, secure and govern the tools, but these (in theory) should be relatively easy to manage, and IT teams, in general, should have processes to manage changes in the organisations.

However, user adoption for these types of services is potentially even more critical to the success of a program like this. As a result, a strong and diverse adoption strategy should be considered, which includes user training, but in parallel, should focus on promoting the benefits, success stories, etc, which tends to encourage users to use the solution more than training on its own.

Ethics and Responsible Use

Based on discussions with colleagues and clients and the consensus in the media, this is one of the key concerns that both users and the C-suite have about the adoption of AI tools within (and outside) the organisation.

Guidelines should be created for both how the tools should be used and how the underlying model is created and governed; focus should be given to bias, transparency and accountability.

In Conclusion

There are several other areas where consideration should be given, including whether your cloud or on-premises infrastructure can support the expected workload and even the costs associated with the service, some of which will be known if the service is licensed i.e. M365 Copilot, but if the service is based on a consumption model e.g. Security Copilot, then the ROI and governance aspects become even more critical to control costs and ROI etc.

By implementing some (ideally all) of the above, then at a minimum there is a good foundation to aid in the deployment of Generative AI across the organisation.