Microsoft announced significant investments in tools during BUILD 2023 to democratise app development for the AI era on Windows.
Regardless of the architecture you’re developing for, be it x86/x64 or Arm64, Microsoft wants to ensure that developers can easily integrate AI-powered experiences into their Windows apps, whether they run on the cloud or at the edge.
At the Build conference last year, Microsoft introduced Hybrid Loop, a development pattern that enables hybrid AI scenarios across Azure and client devices. Now, Microsoft is announcing that this vision has become a reality with the help of ONNX Runtime and Olive.
ONNX Runtime serves as the gateway to Windows AI, providing third-party developers with the same tools Microsoft uses internally to run AI models on Windows, CPU, GPU, NPU, or in hybrid environments with Azure.
With ONNX Runtime, developers can use the same API to run models on the device or in the cloud, enabling hybrid inferencing scenarios. This means that your app can utilise local resources whenever possible and seamlessly switch to the cloud when necessary.
Azure EP preview allows developers to connect their apps to models deployed in AzureML or the Azure OpenAI service. By specifying the cloud endpoint and defining criteria for when to use the cloud, developers gain more control over costs and user experience.
Optimising models for different hardware targets is made easier with Olive, an extensible toolchain that combines cutting-edge techniques for model compression, optimisation, and compilation.
ONNX Runtime can be used across various platforms, including Windows, iOS, Android, and Linux, allowing developers to extend their Windows AI investment to all their app platforms.
Both ONNX Runtime and Olive contribute to streamlining the deployment of AI models into apps. With reduced engineering effort and improved performance, developers can create impressive AI experiences on Windows and other platforms.
The best AI experiences for Windows will also be showcased in a dedicated section in the Microsoft Store to boost their exposure.
There are over 200 million AI-capable discrete GPUs available for Windows 11. Partnering with NVIDIA, Microsoft enables developers to leverage these GPUs to run cutting-edge transformer models that are pre-optimised for Windows.
Microsoft partners such as Qualcomm are delivering innovations like the Snapdragon 8cx Gen3 Compute Platform, enabling Windows devices like the Surface Pro 9 5G and the Windows Dev Kit 2023 to utilise NPUs (Neural Processing Units). With Olive and ONNX Runtime, developers can target Qualcomm AI Engine Direct SDK to run AI models on the 8cx Gen 3 compute platform’s NPU.
Microsoft says it’s excited about the upcoming release of more Windows devices with NPUs from its silicon partners.
AMD has already made early access to the Ryzen AI software available, allowing developers to run AI models on AMD Ryzen 7040 Series processors with Ryzen AI. Intel has also previewed its Meteor Lake product, featuring an integrated AI engine that will scale across the Windows ecosystem later this year. Intel will provide developer tools, including ONNX Runtime support, enabled through OpenVino-EP and DirectML-EP.
Several partners – including WhatsApp, Luminar Neo, and Camo – are leveraging NPUs to achieve significant performance gains and unlock new end-user experiences with their AI models.
Windows developers are encouraged to bring their apps to the Windows platform and leverage ONNX Runtime to deliver AI experiences similar to Camo. Microsoft is excited to witness the creative possibilities that the Windows developer community will explore with these new AI toolchains and NPU hardware advancements.
Elsewhere, Microsoft is integrating GitHub Copilot X into Windows Terminal.
Utilising natural language AI, GitHub Copilot provides suggestions for commands, offers explanations for errors, and performs actions directly within Windows Terminal. Additionally, Microsoft is exploring the integration of GitHub Copilot AI into other developer tools such as WinDBG to simplify your tasks.
A waitlist is currently in place for the new GitHub Copilot features which you can join here.
Microsoft’s commitment to empowering Windows 11 developers as AI developers is evident through the introduction of ONNX Runtime, Olive, and partnerships with leading silicon manufacturers.
All of the tools and collaborations highlighted at BUILD 2023 will enable developers to seamlessly integrate AI capabilities into their apps, creating richer and more intelligent experiences for users across a variety of devices and platforms.
(Image Credit: Microsoft)
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The event is co-located with Digital Transformation Week.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
