11.1.2024 • Business digitalization and knowledge management Metaverse & AI Trainings

Power Virtual Agents and Microsoft Copilot Studio: A Match Made in AI Heaven

What are Large Language Models and how is Microsoft using them?

Large Language Models (LLMs) are a type of artificial intelligence (AI) that can generate human-like text. They are trained on massive amounts of data and can be used for a variety of tasks, such as language translation, content creation, and chatbots. Generative AI is a subset of LLMs that can create new content based on a given prompt. Microsoft is using LLMs and generative AI to develop a range of products, including conversational AI, content creation tools, and data grounding solutions.

Following Microsoft’s investment into the generative AI space with a $1billion investment and a multi-year agreement with OpenAI to host, develop and run their models on Azure, comes products such as Copilots and Azure OpenAI Service. These new technologies and models are being leveraged in almost all of Microsoft’s new consumer and business products.

This article attempts to cover some of the key changes to Microsoft’s Power Virtual Agents with the introduction of these new technologies.

New Copilot Studio

The first and most noticeable change being the integration of PVA as a whole to the new Copilot Studio. The New Copilot Studio includes all the previous capabilities of PVA with the addition of some new features and capabilities including plugin actions and extending M365 Copilot.

PVA chatbots are now given a general term – copilots. There now exists a section to “Build your own Copilot” which essentially means building a custom Copilot (PVA chatbot). Using the new studio we can extend copilots (M365 copilot, custom copilots) through plugin actions, power automate actions, Power Platform connectors etc.

Creating Topics in Copilot studio

As copilot (PVA bot) authors, we usually create multiple topics, each relating to a particular issue or action. Each topic is initiated by a trigger and there are various triggers available within the topic building canvas in addition to the author providing trigger phrases which will trigger the topic. The two types of topics include System and Custom topics.

System topics are out-of-the-box, can’t be created or deleted by the author. They can be edited or disabled.
Custom topics are any topics the author creates.

Generative Answers, Conversation Boosting & ‘Create with Copilot’ Topic creation using natural language

Manual topic authoring can be time and resource intensive, which often requires specialized knowledge in conversational user experiences and deep technical or domain knowledge. Traditionally, authors would create each topic with all the conversational nodes, but now authors can use natural language to describe the topic and Copilot studio can automatically generate the topic by parsing the request and constructing a series of connected nodes and actions. This allows Copilot authoring that could traditionally take hours to be complete, to be done in the matter of a few minutes.

Conversational Boosting is a feature which leverages the language models to connect to a datastore such as a public facing website to generate an answer “on-the-fly” from the datastore when the user query does not match a topic. This is part of the system topic called the fallback topic. We can specify and customize this topic to include particular content and actions but this may sometimes not be as useful to the end user.

This features uses Azure OpenAI’s natural language processing capabilities to ingest information and generate a response in plain language within the context of the user query.

Before the introduction of this feature, we used the fallback topic to query an OPENAI model using Power Automate flows and the REST API to generate responses. This feature solves a large portion of the complexity of using flows and actions to generate responses using the API by simplifying the process to a single step of having the user input a URL to achieve the same functionality.

The behavior described above is called “Generative Answers for fallback.”, but generative answers aren’t limited to fallback scenarios. Within our copilot, a wider range of information sources, internal and external can be utilized to generate answers. These sources could be web sites, Custom Data sources, external or internal web sources and knowledge sources like Sharepoint or OneDrive for Business. Depending on the source, authentication may or may not be required. For sources like Sharepoint, Azure OpenAI on your data, etc. where authentication is required, the Bot User’s Microsoft Entra ID authentication is automatically utilized.

We use the Create generative answers node within a topic to boost our copilots conversation and information capabilities by using different sources.

The sources defined here override the sources specified at the copilot level, which function as a fallback.

This feature can be also used with empty bot to have a production ready bot instantly by simply connecting it to a website.

In an earlier project to build an FAQ bot from a website we had used the classic PVA feature to generate FAQs from a website, which generated static individual topics that contained each FAQ based on the content from the website. This static content included the trigger phrase and answer and was a one-time loading process.

With the new conversational boosting feature, this allows us to build an empty bot and connect it to a live website which will automatically generate answers. This uses NLP to understand the user query and match any content from the website to the question and formulate a plain text response. Instead of it being static Q&As, now it can generate the response on-the-fly and any changes to site will reflect in the content generated.

Generative Actions & Plugins

Capabilities of copilots can now be extended either explicitly from within a topic or automatically using generative actions through plugin actions. Plug actions can be explicitly called within a topic through the Call an action menu, and are only acted upon when the topic it is embedded in is triggered.

Plugin actions are based on core action types, which are – prebuilt or custom connector actions, Power Automate cloud flows and Bot Framework Skills. Each of these core actions has additional information to describe its purpose, inputs etc. that allows copilot to use GPT to generate questions to gather information to fill the inputs required. Plugin actions can generate contextual responses while using the results of the actions and also allows authors to explicitly frame a response.

Generative actions replaces the NLP trigger phrase based approach of topic triggering, and uses GPT to select the contextually appropriate topic or plugin action for the user query. This triggering is automatic and is based on the context that the copilot determines the action is appropriate for by parsing the user query. Generative actions needs to be toggled on with the “Dynamic chaining with generative actions” option in the Generative AI section of the Copilot Studio.

Migrating bots (classic to new unified canvas)

For users that have previously created bots that are deployed, classic bots cannot be used directly with the new studio.

For all the new features to be available, we need to clone Classis bots into the new unified canvas. In addition to this some changes to the administration such as reconfiguring authentication, channels, security settings etc. are required to be made.

With new features, new security considerations are necessary. A new security measure introduced for creating copilots is that admins can now disable publishing bots with generative features through the Power Platform admin portal.

Conclusion

A big question that arose with the introduction of Copilot and LLM technology was the effect it would have on PVA. It is now clear that it does not mean depreciation of PVA, rather an extension of it with these new technologies. This further enhances the capabilities it offers by leveraging generative AI to reduce manual effort and bring forth a new era of custom copilot development.

New training: Copilot Studio Basics – Build Your Own Chatbot

The one-day training provides you with a unique opportunity to learn how to harness the potential of Copilot Studio! Throughout the training day, we will cover the basics of building a chatbot with Copilot Studio, using the bot, and expanding its capabilities.

Join the training and take your chatbot to the next level with Microsoft Copilot Studio!

Interested to learn more?

If you are interested in learning more about Copilot Studio or want to schedule a free consultation with our experts, please contact us today. We are looking forward to hearing from you!