Q. Is this really as harmful as you think?
A. Go to your parents house, your grandparents house etc and look at their Windows PC, look at the installed software in the past year, and try to use the device. Run some antivirus scans. There’s no way this implementation doesn’t end in tears — there’s a reason there’s a trillion dollar security industry, and that most problems revolve around malware and endpoints.
They OCR the entire screen and store it in plaintext?! There is no way… I know it’s Microsoft we’re talking about, but are they really this stupid?
It’s encrypted; the author is pointing out that it has to be decrypted to be used, and then the data can be obtained.
Security and privacy concerns aside, I saw someone commenting on the use case, asking who would ever want something like this.
One problem I hadn’t appreciated for a long time was that some people apparently have real problems with dealing with the Windows UI in terms of file access. They don’t know where their data is being saved. This, in my opinion, is in significant part a Microsoft UI problem induced by various virtual interfaces being slapped on top of the filesystem (“Desktop”, “My Documents”, application save directories, etc) to try to patch over the issue that the filesystem layout was kinda organically-designed in a kind of cryptic way back in the day.
But if you can remember a snippet of text in what you were working on, you can find that thing again even if you have no idea where you stored it. Like, it’s content-keyed file access.
That’s not very useful to a techie. They know how to navigate their system’s filesystem, and even if they lose track of a particular thing, they know how to use the system’s filesystem search tools to search for filenames or content. They can search for recently-modified files. They know how to generally get ahold of stuff.
But for the people who can’t do that, reducing their interface to a single search box might make file access more approachable.
Now, let me reiterate that I think that a whole lot of this is Microsoft repeatedly patching over UI problems they created in the past rather than fixing them. And they’ve done this before over the decades with stuff other than document access. It’s hard to navigate the filesystem to find an installed program a la the MS-DOS era, so they stick stuff in a Start Menu to make it more accessible. That gets too crowded, installers start slapping shortcuts on the desktop. That gets too crowded, installers start adding system tray icons. That gets too crowded, the Start Menu becomes searchable. Each interface just becomes progressively less-usable and the solution each time is to stick a new interface in on top of the old one, which in turn contributes to the complexity of the system as a whole.
But that doesn’t mean that they aren’t trying to address a real problem.
I think that they’d do better with something like having a rapidly-accessible log of recently-accessed files (like, maybe have the filesystem maintain a time-based doubly-linked list of those) and be able to rapidly search the content of documents based on mod time so that recent stuff gets hit quickly, then trying to make their existing search tools more accessible. That doesn’t replicate data across the system and produce some of the problems here. It also permits for fully-searching content, rather than just the stuff that was on a screen when the Recall system grabbed a screenshot and OCRed it. Maybe they’ve done something like that in recent years; I’m many years out-of-date on Windows.
I’d also add that I think that personal computer systems in general would benefit from giving users better control over where their data is replicated to. It’s kind of confusing…you’ve got swap (well, encrypted swap probably helps somewhat with this). Browser history. Any clipboard manager’s retention. Credentials stores. Application-saved copies of in-progress files. Various caches. If you use some kind of cloud-based storage, you’re pushing data out to other computers. Backups. Just a lot of state that can be replicated all over the place and is hard to go back and track down and remove. That’s even before stuff like issues with doing secure deletion on existing filesystems (which we had a conversation about the other day, everything from log-structured filesystems to wear-leveling on SSDs inducing data replication). If you want something definitely gone, be able to manage your data’s lifetime, something that I think that a lot of people – even non-techies – would like, you really have to have a lot of technical knowledge of the system’s internals as things stand today. This Recall thing is egregious, replicates data all over, but it’s far from the first feature that makes it harder for people to understand and control the lifetime of data on their computer.
I don’t think that the software world has done a great job of letting people control that data lifetime. And I think that it’s something that a user should reasonably be able to expect out of their computer.
There was an article going around a while ago that was arguing most users these days, including the youth we often stereotype as “digital natives” who “get computers”, don’t understand file systems. They might not even know they exist as a concept.
Which makes sense if you’ve only ever really used modern UIs. You don’t have to know anything about files and folders. I bet a lot of people don’t even know they exist in any meaningful way.
Most users are shockingly ignorant, and a lot of them are not really paying enough attention or interested enough to learn much.
I remember reading an article a few years back about physics undergraduates who didnt know how to use a computers file system. They could learn, but these are smart likely at least fairly tech inclined kids and they didnt know how to navigate folders on a computer at 18.
My daughter certainly doesn’t have a good understanding of file systems even though I’ve been trying to teach it to her.
I don’t think any of the UX problems you’re describing have been solved on any platform. If anything Windows is one of the better examples here, because I’ll be fucked if I can ever find a file on Android and don’t get me started with Linux.
Yeah this is why Apple has been slowly peeling away traditional file / folder features from front and center. The user doesn’t care where or how they get their files, they just want them at any given time. Spotlight being the most successful at obfuscating where anything is yet allowing access to everything. Microsoft has started to pick up on that and attempt to solve the same problems.
The bizarre thing is, they have solved it. PowerToys Run is the Spotlight omnibar of everything and they bizarrely haven’t chosen to bake it into Windows proper. I can’t use Windows without it now. Search files and folders everywhere faster than the start menu search, search running processes, execute commands, do maths, calculate hashes, open web pages. It’s fantastic.
Do you really need screen snapshot to do fine grained search though? It sounds like you’re describing Spotlight in some way https://en.m.wikipedia.org/wiki/Spotlight_(Apple)
I completely agree with this. I work as a User Experience researcher and I have been noticing this for some time. I’m not a traditional UX person, but work more at the intersection of UX and Programming. I think the core problem when it comes to discussion about any software product is the people talking about it, kind of assuming everyone else functions the same.
What you mentioned here as a techie, in simple terms is a person who uses or has to use the computer and file system everyday. They spend a huge amount of time with a computer and slowly they organise stuff. And most of the time they want more control over their stuff, and some of them end up in Linux based systems, and some find alternative ways.
There are two other kinds of people. One is a person who uses the computer everyday but is completely limited to their enterprise software. Even though they spend countless hours on the computer, they really don’t end up using the OS most of the time. A huge part of the service industry belongs to this group. Most of the time they have a dedicated IT department who will take care of any issue.
The third category is people who rarely use computers. Means they use it once or twice in a few days. Almost all the people with non-white collar jobs belong to this category. This category mainly uses phones to get daily stuff done.
If you look at the customer base of Microsoft, it’s never been the first. Microsoft tried really hard with .NET in the Balmer era, and even created a strong base at that time, but I am of the opinion that a huge shift happened with wide adoption of the Internet. In some forum I recently saw someone saying, TypeScript gave Microsoft some recognition and kept them relevant. They made some good contributions also.
So as I mentioned the customer base was always the second and third category. People in these categories focus only on getting stuff done. Bare minimum maintenance and get results by doing as little as possible. Most of them don’t really care about organising their files or even finding them. Many people just redownload stuff from email, message apps, or drives, whenever they need a file. Microsoft tried to address this by indexed search inside the OS, but it didn’t work out well because of the resource requirements and many bugs. For them a feature like Recall or Spotlight of Apple is really useful.
The way Apple and even Android are going forward is in this direction. Restricting the user to the surface of the product and making things easy to find and use through aggregating applications. The Gallery app is a good example. Microsoft knew this a long back. ‘Pictures’, ‘Documents’ and all other folders were just an example. They never ‘enforced’ it. In earlier days people used to have separate drives for their documents because, Windows did get corrupted easily and when reinstalling only the ‘C:’ drive needs to be formatted. Only after Microsoft started selling pre-installed Windows through OEMs, they were able to change this trend.
Windows is also pushing in this same direction. Limiting users to the surface, because the two categories I mentioned don’t really ‘maintain’ their system. Just like in the case of a car, some people like to maintain their own car, and many others let paid services to take care of it. But when it comes to ‘personal’ computers, with ‘personal’ files, a ‘paid’ service is not an option. So this lands on the shoulders of the OS companies as an opportunity. Whoever gives a better solution people will adopt it more.
Microsoft is going to land in many contradictions soon, because of their early widespread adoption of AI. Their net zero global emission target is a straightforward example of this.
I don’t think that the software world has done a great job of letting people control that data lifetime. And I think that it’s something that a user should reasonably be able to expect out of their computer.
That’s true.
I once thought about this, that maybe it’s a good idea to use a tagged and maybe log-style filesystem, where 1) every directory name in file path becomes a tag for it, other than the user-added tags (which can be searched separately), 2) there are temporary and permanent files, where temporary ones are deleted once their lifetime passes or that plus once the space is required, while permanent ones are stored indefinitely, 3) with hardlink functionality transparently available to the user, from the GUI, 4) the GUI itself should drop the bullshit and return to DOS times in the sense of control - with this thing I describe it may well be that the casual user won’t feel as lost as they do now.
Maybe (again, transparent and user-accessible) filesystem overlays for every application are a good idea too here, like with Docker, chroots, MacOS DMG images, etc.
In addition to that indexing file contents may make sense too, like you said.
Frankly there are so many good things one can do which haven’t been done, before just OCR`ing everything on the screen and storing it.
About “why MS chose this” - because they consistently choose the dumbest and ugliest way to deal with any problem. The heaviest artillery available, to look relevant.
Offtopic - the searchable start menu problem is what scares me off Gnome every time I try it. You just get that tablet-like one-level place with a search field and icons. A frigging lot of fscking icons for every dot-desktop file Gnome found. Then I panic and get back to FVWM.
