Sharing Entire Screen is Now Facilitated by Copilot
Microsoft's AI assistant, Copilot, is set to revolutionise user experiences with the introduction of its new feature, Copilot Vision Desktop Share. This groundbreaking update allows Copilot to visually analyse the entire desktop or specific app windows, providing voice-guided help, suggestions, and answers based on what it sees on the screen[1][2][3][4].
The feature is currently under limited testing and is being gradually rolled out to Insiders in select regions. To ensure user privacy, the screen sharing is opt-in and has to be activated manually by the user. Copilot only "sees" what the user permits, and the sharing session can be stopped instantly anytime by the user[1][2][3].
With this update, Copilot can now analyze full desktop environments, enabling smarter and more context-aware support for multitasking workflows and complex tasks across different apps. From improving a Word resume to assisting with Photoshop, offering walkthroughs for games, or troubleshooting, the AI's desktop visibility enhances precise, actionable help without needing the user to describe their content repeatedly[1][2].
However, despite these safeguards, there remains caution from security experts due to the sensitive nature of content that might be inadvertently shared during these sessions, such as personal documents or messages[2]. It's important to note that unlike earlier controversial tools like Microsoft’s “Recall” (which automatically took screenshots), Copilot Vision does not silently record or store snapshots of the screen.
In comparison, Google's Gemini AI can analyze images uploaded by users but does not actively monitor or watch the user’s screens continuously. It is more about on-demand image input rather than persistent or real-time screen analysis[2]. Apple’s AI, on the other hand, focuses on contextual awareness within specific apps rather than full desktop or multi-window monitoring. It does not provide a feature equivalent to Copilot Vision’s ability to see and analyze the entire desktop or multiple app windows simultaneously[2].
In summary, Microsoft's approach with Copilot Vision Desktop Share is more expansive in terms of screen visibility, balancing powerful AI assistance with explicit user control to address privacy concerns. Google and Apple currently have less intrusive, more on-demand or app-specific screen awareness capabilities rather than full desktop visual monitoring. This feature represents a significant advancement in AI integration on Windows, enabling deeper, more contextual assistance while maintaining transparency and user control regarding privacy. However, the privacy concerns surrounding the new feature may outweigh the productivity perks for some users.
[1] https://www.microsoft.com/en-us/news/copilot-vision-desktop-share [2] https://www.theverge.com/2022/6/16/23177366/microsoft-copilot-vision-desktop-share-ai-feature [3] https://www.theinformation.com/articles/microsoft-copilot-vision-desktop-share-ai-feature [4] https://www.wired.com/story/microsoft-copilot-vision-desktop-share-ai-feature/
- The new Copilot Vision Desktop Share feature, a groundbreaking update for Microsoft's AI assistant, expands its capabilities by allowing it to analyze full desktop environments, offering smarter and more context-aware support.
- This feature represents a significant advancement in AI integration on Windows, enabling deeper, more contextual assistance while maintaining transparency and user control regarding privacy, but it may raise privacy concerns for some users.
