Kalakshar US

Beyond debuting new features for Copilot+ AI PCs and Windows 11’s 2024 update, Microsoft is also giving its Copilot AI a makeover on the web, mobile and desktop. That includes a slightly friendlier interface wherever you access it, along with new capabilities like Copilot Voice, which allows you to talk conversationally with the AI assistant. Ultimately, Microsoft is aiming for Copilot to be seen as more than just a party trick for generative AI search and image creation — it’s trying to make it a core part of your daily workflow.

That starts with a cleaner and simpler UI that makes Copilot look different than a boring old search engine. That new look hits Copilot on the web, iOS, Android and Windows today. You’ll also be able to access Copilot from within Whatsapp, which could be useful if you want to avoid Meta’s AI assistant.

Microsoft also plans to release more experimental AI features via Copilot Labs, similar to the way Google has given users access to early features with its own Labs. There’s Copilot Vision, which lets the AI see everything you’re viewing in Edge, and Think Deeper, a method for letting Copilot work harder on more complex dilemmas. Both features also have the potential to create new privacy and security issues, but Microsoft appears to have learned a bit from its messy rollout for Recall on Copilot+ PCs. Notably, these Copilot Labs offerings will only be available to people paying for Copilot Pro subscriptions.

Microsoft says Copilot Vision sessions are “opt-in and ephemeral,” and nothing is stored for AI training. Any data related to your session is deleted once you’re done. The feature also won’t work on all websites, only a “limited list of popular websites.” And Microsoft notes that Copilot only interprets images and text that you’re seeing on a website; it’s not doing any major content processing. The company also plans to take feedback from early users, refine safety measures and “keep privacy and responsibility at the center of everything we do.”

There’s no doubt that Microsoft has to re-establish trust with its users after the botched rollout for Recall, a feature that was meant to help you find anything you’ve done on your computer, but had some gaping security holes early on. “Ultimately, users want to have confidence with anything happening with sensitive data caches,” Microsoft’s Windows head, Pavan Davuluri, said when we asked him about the lessons the company learned from the Recall fiasco. He added that he believes the company has “gone above and beyond” to create that confidence by establishing new security standards for Recall.

Copilot Discover

There are less controversial features coming, as well: Copilot Daily uses Copilot Voice to delivery a summary of news, weather and potentially reminders about tasks every day. Based on the demos we’ve seen, it’s a lot like an early morning podcast built just for you. Copilot Discover, meanwhile, will give you tips about AI features to use (pictured in the screenshot above), depending on Microsoft services you’ve used in the past.

Personally, I’m still a bit ambivalent about Microsoft’s near-instant transformation into an AI company. That’s particularly true since we’re still dealing with inaccuracies and other issues from OpenAI’s ChatGPT, which Microsoft relies on to power Copilot. These Copilot features seem a bit more useful than being a souped-up search engine, but the company still needs to prove it can build AI features with bulletproof security and privacy.

As with practically any AI product these days, Microsoft is also staggering the release of these Copilot features. Copilot Voice and Daily launch today, but Voice will only be available in English in the US, Canada, Australia, the UK and New Zealand. Copilot Daily, meanwhile, only works in the US and UK for now. Copilot Vision will head to Labs over the coming days for a “limited number” of Copilot Pro subscribers, while Think Deeper is available for Pro users in the US, Canada, Australia and New Zealand.

This article originally appeared on Engadget at https://www.engadget.com/ai/microsofts-copilot-ai-gets-a-voice-and-the-ability-to-see-websites-you-browse-140030003.html?src=rss 

Now that we’re a few years into Microsoft’s obsession with AI and its Copilot assistant, it’s clearer than ever that Windows 11’s role is to show off the company’s artificial intelligence prowess. At least, that’s the message I took away from today’s announcement that the Windows 11 2024 (version 24H2) update is now beginning to roll out. While the OS itself is getting a few new features, like long overdue File Explorer tweaks, Microsoft is far more eager to hype up new Copilot+ AI PC capabilities.

There’s “Click to Do,” which triggers Copilot to assist you with whatever is on your screen, like removing an object in the Photos app, or summarizing a long article. It’s seemingly easy to use — just hold the Windows button down and click your target — and Microsoft says it’ll offer contextually relevant tips. As with so many of the company’s AI features, you can think of Click to Do as an on-demand super-powered version of Clippy, its infamous software helper from years past. But since Click to Do appears at your whim, and there are actually some potentially useful AI features, it should hopefully be far less annoying than that damn paperclip.

Click to Do in Windows 11.

You’ll also be able to upscale pictures in the Photos app by up to eight times their resolution. A slider will let you adjust precisely how much you’d like to increase the quality. This isn’t particularly new—Adobe has its own AI-powered Super Resolution capability, as does the popular Mac photo editing app Pixelmator. But at least it’s helpful to have it built directly into your OS. Generative AI-based fill and erase options are also coming to Paint, allowing you to remove objects or easily create new ones.

Microsoft has hinted at some AI-powered search improvements in the past, and it looks like those are finally arriving with the Windows 11 2024 update. Now you can find files using your own words, without worrying about esoteric search syntax. If you want to find pictures of your dog by the beach, you can just type that.

And of course, there’s Recall, the company’s debut Copilot+ feature for retrieving anything that happens on your computer. It was immediately criticized for being surprisingly insecure — researchers discovered that hackers could access the Recall screenshot database without administrator privilege. Microsoft immediately delayed Recall to revamp its security model: It’s now making the feature completely opt-in, instead of flipping it on by default. Windows Hello biometric authentication is required to use Recall, and it’s also encrypting the screenshot database and other interactions.

Recall timeline in Windows 11.

Last week, the company detailed more of its security methods, including using VBS Enclaves to further isolate Recall from hackers. David Weston, Microsoft’s VP of OS and enterprise security, noted in a blog post that “you are always in control” of the Recall experience. That reassurance may not be enough for users turned off by Microsoft’s initial security flubs, though. If anything, Recall’s rollout was a clear example of how the company was cutting corners to move quickly and be seen as an AI industry leader.

“Copilot will be there for you, in your corner, by your side, and always strongly aligned with your interests.,” Mustafa Suleyman, the company’s head of AI, wrote in an overly enthusiastic blog post. “It understands the context of your life while safeguarding your privacy, data and security, remembering the details that are most helpful in any situation.”

Security will undoubtedly be a major concern for Copilot users moving forward, and according to Microsoft’s Windows head, Pavan Davuluri, the company has learned from its troubled Recall launch.

“Ultimately, users want to have confidence with anything happening with sensitive data caches,” he said in a briefing with press, adding that he believes the company has “gone above and beyond” to create that confidence. He also acknowledged that AI features enable “new attacks and defenses.” For example, Windows has traditionally allowed administrator accounts to have total access to everything on a machine, but now users may want to see “additional rings of protection” with AI features.

The Surface Pro Copilot+ AI PC.

Davuluri stressed that community feedback helped Microsoft make Recall more secure for users, and the company will continue listening as it carefully rolls out more AI-infused capabilities. That’s also why all of the above Copilot+ features won’t be rolling out to everyone immediately — they’ll be available to Windows Insiders first, and then arrive via a phased rollout to “select devices and markets” in November. The Windows 11 2024 update will also have a staggered rollout starting today. If you’re eager to snag it, make sure you’ve enabled “Get the latest updates as soon as they’re available” in Windows Update.

And what about features from this Windows 11 update that don’t require Copilot+ PCs? They appear at the tail-end of Davuluri’s blog post today, almost as an afterthought: There’s Wi-Fi 7 support, HDR backgrounds, Energy Saver improvements to prolong battery life and better hearing aid support using Bluetooth LE. They sound like the Windows upgrades we used to see before Microsoft became completely AI-pilled, but they’re certainly not as exciting as something like Click to Do.

Of course, that’s all intentional. If you want to join the Copilot+ AI PC party, you’ll probably have to get a new computer. And that’s precisely what Microsoft and PC makers want.

This article originally appeared on Engadget at https://www.engadget.com/computing/microsoft-delivers-new-copilot-ai-pc-features-with-windows-11s-2024-update-140048807.html?src=rss