3175x175(CURRENT).thumb.jpg.b05acc060982b36f5891ba728e6d953c.jpg)
Here's what you need to know about how Windows Recall works on Windows 11.
On Windows 11, Recall is a feature that uses AI to record everything you've done on the computer and then makes that information searchable to help you find anything you have done in the past, from a message to a file or folder, a website, and virtually anything.
The magic of Windows Recall is that you can search using keywords or keyphrases and natural language for anything you have done or seen on the device, as the feature can actually understand the content and context.
For instance, you can ask Recall to find an image of a black leather jacket you saw on a website or a picture you took in the past. The feature's precision is remarkable. It can bring up the exact moment the leather jacket has appeared, not because of keywords in the filename or metadata information, but because it understands the object on the image.
The idea behind this feature is fantastic since you no longer have to remember specific locations, file names, or keywords in the document. However, did you stop to think about how Windows Recall actually works?
In this guide, I will dive deep into the inner workings of Recall for Windows 11.
How does Recall work on Windows 11?
The Windows Recall feature takes snapshots of every activity on the screen. When content is on the screen, the feature takes snapshots every five seconds from the previous capture. The feature doesn't record audio or save continuous video. The snapshots are then stored locally and analyzed with AI to understand their contents, including images and text.
It should also be noted that Recall is presently optimized for certain languages, such as English, Simplified Chinese, French, German, Japanese, and Spanish. That means you get the full suite of Recall abilities. Microsoft plans to expand this language support in the future. Currently, Recall can retrieve snapshots from your PC's timeline through text-to-text searches in over 160 languages. which is a more basic Recall feature.
When you open the "Recall" app and use the search box (or the timeline experience), the feature will understand the search intent and return the most relevant results.
(Image credit: Microsoft)
When you select a snapshot, you will enable the "screenray," the feature that runs on top of the screenshot and allows you to interact with the different elements. (I find this similar to the "Text actions" features on the Snipping Tool.)
Inside the snapshot, you will find different actions. For example, you may find the option to open the source application for the content. You can copy text from a message or anything on the screen, delete the snapshot, and access the context menu to access other actions.
Storage
Since the snapshots are stored locally on the computer, this feature requires some available space that the system reserves automatically. By default, Recall sets aside 25GB of storage on devices with 256GB of storage.
(Image credit: Feature)
However, the minimum storage reservation will depend on the system storage. For instance, on a device with 512GB SSD, the default allocation will be 75GB. On a 1TB SSD, the allocation will be 150GB. You can also change this default configuration to increase or decrease the storage from the "Recall & snapshots" settings page.
AI models
Recall uses the required NPU (Neural Processing Unit) to analyze the snapshots using several multimodal smaller language models, including Screen Region Detector, Optical Character Recognizer, Natural Language Parser, Image Encoder, and Image Encoder, which are all integrated and running concurrently on Windows 11 to detect text, images, videos, audio, and more.
(Image credit: Microsoft)
This is possible thanks to the new "Windows Copilot Runtime," which integrates more than 40 AI models to power this and many other features. It also provides the infrastructure to update and maintain the models' quality continuously.
Once the snapshot is captured, the feature combines it with system data to understand the content and context, similar to how a person would understand a piece of content. Everything is saved in the new "Windows Semantic Index" database, which differs from the Search index database.
Security
Since everything happens on-device (also referred to as "Edge Computing"), no data is uploaded to the cloud for processing. However, the feature sometimes connects to the internet to download and install updates.
As part of the default experience using the Recall feature, it doesn't save information from certain activities, including using Chromium-based browsers like Microsoft Edge in incognito mode or content with Digital Rights Management (DRM). Still, everything else will be scanned and stored on the computer.
If you don't want the feature to save information for specific websites or apps, you must manually configure a filter from the "Recall & snapshots" settings page. However, this will only work on Microsoft Edge if you want to filter out a website.
(Image credit: Future)
One limitation of this filter is that if you open the Windows Recall app and click the "Now" option, the feature will take a snapshot of anything on the screen regardless of whether it's a browser instance in incognito mode or a blocked app or website. In this scenario, you will be shown the snapshot from a temporary file, which won't be saved on the device.
Also, the AI timeline for Windows 11 does not perform any content moderation, so information like passwords and bank account numbers will appear with a simple search as they are saved automatically in the snapshots.
You can quickly mitigate this security concern by excluding the websites and apps that may show sensitive information.
Alternatively, you can also delete some or all the snapshots from your computer to prevent the feature from storing this type of data.
Although the "Windows Semantic Index" database stays locally on your computer, it will only be private if you take the proper precautions since the Recall doesn't include strong security protection once someone is signed into the account. For example, it's not password protected unless you configure a password on your Windows 11 account, and encryption for the data is only available if you configure BitLocker or Device Encryption on the device.
Windows Recall will be one of the new features released with the Windows 11 2024 Update (version 24H2). However, it will initially only be available for Copilot Plus PCs running Qualcomm Snapdragon X series processors as the feature requires an NPU that runs at 40+ TOPS, a minimum of 16GB of RAM, and 256GB of SSD.
It's also important to note that the feature will initially be limited but will keep improving over time.
- UberGeek
-
1
Recommended Comments
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.