Thinking Out Loud: What Should an AI PC Do?

What Should an AI PC Do?

Previous versions of PC-based AI did live up to their promise. Cortana was too primitive. Like Apple’s Siri, it performed only rudimentary tasks. Microsoft deprecated Cortana as a Windows 11 feature. It now exists only as a download from the Microsoft store. I will return to Siri later, as it is likely the Apple interface to AI.

AI will likely transform the way personal computers work. They will likely start with advanced information retrieval powered by voice-to-text (more likely direct voice, but the interface will mimic text in the background until the chatbot modalities evolve).

Information Finding You

In my work with Microsoft, I suggested that one aspect of the future of computing would include “finding information for you.” This idea suggests that local devices build a model of the owner, as well as their information assets, including files and communications. When attending a meeting, for instance, the system will compile a view of information required by that person at that moment. Rather than asking, “Where is that PowerPoint?” the latest version will be at the top of the list of content required to support the meeting.

Learning Architectures and Adaptive Workspaces

But this idea wasn’t new. At Forrester/Giga, I crafted planning assumptions that talked about two AI-influenced ideas. The first, I called Learning Architectures. It consisted of a block diagram with sections for “query generation” and “categorization engines.” These services sat upon metadata that enhanced context stored in what I called a “universal repository.” The universal repository was an abstraction of all types of content, from e-mail and structured data to the enterprise directory, collaboration platforms, videos, images, text, and sound. You get it. Everything. 24 years later, software makers still don’t sell a universal repository. Not IBM. Not Oracle. Not Dell. Not OpenText.

However, generative AI cheats by abstracting away the repository in its working memory, its vectors and weights. While it does not provide direct access to source documents via foundation engines, it starts to offer some of the features I described in my writing about Learning Architectures. To be clear, Learning, in this case, was about computers learning, not about supporting human learning, although the entire rationale for a Learning Architecture was to support people effectively finding, applying, repurposing, and collaborating on the content the system surfaced.

A year later, I started writing about Adaptive Workspaces. Adaptive Workspaces built atop the Learning Architecture with another layer of intelligence that looks at the end user computing experience as a set of indexable objects, from data to applications. It pointed to a fundamental flaw in the computing experience that still exists today: the lack of synergy between the computer’s owner, the data, and the work being done.

As with information finding you, Adaptive Workspaces suggest that the entirety of the computing experience be informed by the owner and the work. My computers, be they Microsoft Windows or Apple macOS-based, conform to my will because I have manually created environments that suit my needs. I have moved items to favorites or the taskbar because I wanted them there, not because the OS sensed a pattern in my work that would benefit from easier access to tools. That is a simple pattern recognition chore.

Slightly more complex would be the repeated process of compacting a PowerPoint-generated PDF with Adobe Acrobat and changing its name by amending an “email” to the smaller file. And while the Mac will remember my last configuration and start me off where I left off, it is not capable of reconfiguring itself on demand, to say, a start-up configuration. For me: Load TheBrain. Load Mail. Load the browser with the last open instances and sites. Load the Calendar. If the next calendar entry includes a conference call, load the appropriate meeting platform.

Will Privacy Kill the AI PC?

As I wrote in Will Privacy Kill Evidence-Based Performance? I point out that many of the ideas related to gathering data about an individual fall into the very muddy realm of personally identifiable information, or PII. That article suggested that the rich data associated with how people work is feared by the likes of SalesForce, Microsoft, Google and Cisco because they don’t want to be seen gathering intimate details about people’s work and using that for marketing, or worse, mishandling it and exposing untoward details about people’s habits, or offering data to a supervisor that might be used not to coach, but to punish.

Microsoft’s Viva lost many key capabilities and continues to lose more, some based on privacy and others on new AI models that Microsoft will likely use to replace it.

Without learning from the end users who interact with them, the AI in PCs will not move far beyond its existing capabilities. Today, AI results are just as good when cut and pasted from a web browser as they are when integrated into the likes of Edge or Windows. AI will remain just a tool until PCs start to have conversations with their owners and intuit intention. The implementation of bidirectional learning, when PCs learn as much from their owners as their owners learn from them, will spur the real transformation of personal computing.

Unless software developers can find the right balance between data protection and learning, the AI PC will likely stall in its infancy, failing to achieve much beyond its current capabilities and certainly not reaching any of its loftier goals.

What Should an AI PC Do?

These ideas, at this point, represent no particular order, nor of difficulty or desire. The list is unordered; however, at some point, the ability to implement and the potential return on the item will suggest an order, which may be different for each person.

As much as the AI PC will personalize the personal computing experience, vendors will need to create broad capabilities because they will not be able to anticipate what people will want to do. Applying use metrics too early will likely narrow insights and lead to premature conclusions.

• Automate actions/tasks based on recurring patterns.
• Facilitate conversations with content.
• Query tools for their capabilities. Use files as input to queries.
• Transform ideas into usable content. Microsoft’s PowerPoint copilot, for instance, will turn a narrative into a set of slides. This should eventually work more like Meta’s Llama 3, where the content owner types and drags over images and reference material, and the presentation grows from the interactivity, not from a single act of transformation. This could also include an assembly process. For instance, the owner could provide images, text and other assets to the AI to create a blog with the right visual and tonal style and post it. The AI would be a proxy author on the blogging platform with read-write permissions.
• Build the account owner’s profile, including voice (literary and perhaps even actual). Although it may be unethical for a synthetic computer voice to impersonate someone, there should be no ethical dilemma associated with the person who owns the voice using a synthesized version of their voice to respond to others, such as rewriting an e-mail as a voicemail and sending that rather than the e-mail because the recipient has shown a propensity for answering voicemails more frequently than they respond to emails. As Lenovo recently demonstrated, profiles will also include visual style capture for artists and designers so systems will be able to create content in the style of their owners. Big questions remain about how profiles will integrate and how they will be managed.
• Act on an explicit or implicit context (creating, reading, meeting, etc.) by providing easy access to the most relevant content to support that context.
• Manage communications. Automatically writing responses for review, perhaps sending those with low input thresholds. Unsubscribing. Enhanced spam filtering. AI may prove to be the tool that finally eliminates e-mail as a primary communication channel. AI should be able to federate sources such as Slack channels, Teams channels, ticketing systems and other enterprise systems. Integrating the variety of channels associated with personal lives may take longer.
• Suggest what the owner should be working on to meet deadlines. Using a past experience, like a “leave by because of traffic” in a calendar entry, the OS could say, “Get started on X because it will likely take you 90 minutes to get it done.”
• Tie into directory information and associated intelligent metadata to suggest possible collaborators on projects. Metadata about people within enterprises will include competency profiles as models, not just a list of skills. The AI builds these models by monitoring work and learning content. Recommendations will be based as much on work compatibility and competence.
• Build mindmaps from brainstorming conversations.
• Configure the PC by voice or text input. This should include all features. As an observation, this may take some time, as Microsoft has taken forever to completely replace the Control Panel with Settings.
• Automatically set preferences based on behavior. This will also include such mundane tasks as connecting to a local printer without doing anything other than printing.
• On-device translation.
• Proactively manage calendars, including negotiating with other people’s calendars.
• Integrate with automation. Manage equipment in a factory or an office. At home, integrate with home automation to optimize comfort and security.
• Integrate with health and wellness systems to monitor lifestyle and suggest modifications to behavior to improve health.
• Enhance security with more “propensity of evidence” security models (multiple forms of biometric and behavior authentication) as well as predictive threat analysis.
• Adapt to alternative modalities to accommodate access for those for whom typing and voice may be insufficient as an interface.
• Performance optimization will be driven by monitoring system components and managing heat, power, I/O and other characteristics. This falls into the “autonomic” AI classification, meaning it will rely on machine learning rather than generative AI, except perhaps in having a conversation with a PC about its performance.

Some of these ideas may appear intrusive. Most of them require no more access to data than is already granted to a PC. Today, however, the PC does not include features to interpret this data. The AI PC should be able to incorporate data from content and interactions and build models, perhaps slowly at first, but eventually, all the items on the list should be possible.

As a scenario planner, I suggest that AI PC designers keep the social acceptance, regulatory and legal frameworks, environmental impacts, and economics of the designs in mind. Just figuring out how to deliver a feature does not mean it will be desired or allowed.

One example of scenario thinking to consider comes from the overreliance on generative AI as the “AI” to drive the AI PC. The integration of summaries, categorization, writing, and image creation may overwhelm research on more adaptive and progressive models capable of doing even more, as designers place too much burden on the existing models to deliver in ways they were not designed to deliver. Research also needs to continue on autonomic “AI,” features like camera image enhancement, noise reduction on microphones, and various security features that use pattern recognition for malware or local awareness, such as locking the PC when no user is present.

The higher-order work, as discussed above, will require agents and access to data that may not be easily granted.

What Hardware Will AI PCs Need?

Some AI PCs will need nothing more than a button to invoke a Copilot. That will be the gimmick version.

More sophisticated AI PCs will run AI workloads locally, starting with compact models. Unlike the models that can be run on relatively modest devices using LLMStudio, AI PCs will integrate the compact models into actions and applications. Most likely, as Lenovo showed at CES 2024, conversational configuration may be an early feature, along with modeling style for writing and image creation.

All of the next-generation PCs will include neural processors, along with the CPUs and GPUs, as Apple already does across its product line. Intel (“Meteor Lake”/Core Ultra) and AMD (Ryzen 7000 Series) have started shipping chips that deliver on SOC neural processors or NPUs. These chips, along with Apple’s M-series, represent the early stages of chip-based support for AI.

New forms of RAM, such as magnetoresistive RAM or m-RAM (see A Paradigm Shift in RAM Is About to Make Computing Unstoppable, Popular Mechanics). A new approach from researchers at Hebrew University in Jerusalem uses very fast oscillating light waves to control magnets. This new RAM reflects a practical use of quantum theory principles. m-RAM may be coming along at just the right time. The question is, can it scale to meet the demands of the computing market?

If AI catches on, it will likely start to erode the low-end of the Intel and AMD product lines, with all PCs requiring high-end CPUs and more memory, increasing entry-level costs. Power consumption is also likely to rise in conflict with burgeoning environmental goals for energy consumption on devices. AI may be able to offset its own increase in power use during peak workloads by better managing power when devices aren’t being used, though learning models that run in the background may take up these excess cycles.

AI and the Cloud

If done well, AI PCs will likely create a resurgent demand for local storage. Most cloud storage systems, such as Box, Dropbox, Apple iCloud, or Microsoft OneDrive, routinely make their content available on demand. On-demand content cannot be indexed beyond perhaps the reference to the file name.

More sophisticated indexing is possible, but it would require the cloud index to become available to the local OS, which may introduce both privacy and performance issues.

The more powerful solution involves synchronizing all of the cloud data locally so that it is available, for instance, to feed a RAG query about something for which the owner could be considered a subject matter expert or something the owner has acquired, such as research reports, academic papers, or other types of files. Ideally, all content on local storage, unless explicitly excluded, should be available as part of a query using the AI engine. A prompt like, “Provide an overview of ProjectX from the files stored in the folder named “ProjectX,” could more elegantly be a right-click on the folder and include a command, “Ask AI about the contents of this folder.”

If that folder just reflects pointers to cloud files, it would require downloading all of those files every time the space is freed up upon the conclusion of the query. This would require a lot of I/O and bandwidth. Alternatively, the query could be passed to an AI in the cloud associated with that storage system. This AI would likely prove suboptimal to the local AI tuned to an individual.

AI on Apple’s Mac

It is unlikely that Apple will integrate AI into its keyboard, given its previous keyboard foibles, most notably the touch bar. Siri already offers very basic AI capabilities. Making Siri smarter will likely be Apple’s move. That will make voice input to generative AI the primary interface, and one, if Apple is successful,

Apple is likely holding back on generative AI because of its inaccuracies and biases. I speculate that Tim Cook might perceive Satya Nadella as verging on reckless with his relentless bet on OpenAI, rapid roll-outs of generative AI as a free feature in Windows and Edge, and the premium pricing associated with subscription-based Copilots that extend the features of the free versions. Apple isn’t known for being reckless.

Cook is also likely looking at cost. One reason Copilots cost so much is that they require significant operational costs, primarily compute infrastructure and energy. The Edge and Windows Copilots are loss leaders in search of subscriptions to more finely honed and value-added Copilots.

I don’t see Apple going after AI to automate Pages, Numbers or Keynote. Instead, they will concentrate on core operating system functionality, information retrieval, smart home and health. The order and effectiveness of each of those areas will depend on LLM trustworthiness and the ability to put hooks into the various aspects of the Apple ecosystem. Apple may also choose to develop learning models with domain specificity to augment more general foundational models. Apple will likely avoid shipping prototype software, which might risk its reputation.

I do not see Apple charging for their AI services at this point.

Feedback and Learning

Core to my beliefs related to learning systems is that they should learn. AI PCs will need to learn. First, they will need to learn how their owners work. They will then need to learn as their owners learn, working against the tendency to make productivity improvements static once instilled. Adaptive interfaces need to be just that, adaptive. Finally, AI PCs will need to learn how to transfer learning between systems. I want my PC to teach my iPad. I want this year’s PC to teach my 2027 PC. New devices should no longer start as blank slates but rather bring across more than the data stored in an iCloud or OneDrive; they should be personalized soon after they first power up.

Unfortunately, again, fundamental behaviors at Microsoft, such as the inability to store core settings for Microsoft 365 in the cloud suggest they still need to learn how to learn from their customers. Someday, I would like to sign into my Office 365 account on a fresh PC and not have to tell Word I don’t want entire words selected. I have changed that setting every time I initiate a new instance of Microsoft Word, and I have done so for decades. With the cloud, that should just be part of my profile. With AI, it should learn that preference. Learning behaviors from learning models, some of which may be different in future models, will prove a big ask.

When Will We See AI PCs?

The initial batch of AI PCs will be available mid-2024. They will likely be the big story at CES 2025. These early versions will be available in volume by late-2024, though they will include only a few of the features listed in this post. The first batch of AI PCs will offer a Copilot key, AI processors, some generative AI integration for content creation and modification, and a number of autonomic AI features for security and performance.

Many items on the list will be years away because generative AI and the other AI models currently available are not designed for this kind of work—but they could and should be. Some may become available but will be restricted because of the cost of purchasing subscriptions to the feature. Subscriptions will test the viability and perceived value of AI. Low uptake may force Microsoft, Google, Cisco and others to reconsider their AI business models.

Intelligent agents empowered by more open interfaces for data and apps within an OS should be at the top of the features list. Unfortunately, for AI to reach the next level, all the years spent locking everything down on the desktop so that it can’t be compromised by viruses and malware may need to be reconsidered.

Of course, the AI PC assumes that people will still be working with PCs and they are not cut out of the value chain. I don’t think that will happen because AI needs people and the content they horde on their devices to be effective in the last mile of its journey to value. Expertise and innovation live in individual histories, often lost to organizations once people leave or the organizations cease to exist. AI PCs will introduce a new form of digital legacy.

No LLM has access to that information, and it won’t unless I, and many of you, grant them access. For that to happen, there needs to be something in it for those sharing their life’s work with a computer that could, frankly, care less. The organizations that buy computers, however, care a lot despite their seeming propensity for productivity over everything. Productivity only matters for those making money, and making money requires innovation and relationships, activities that may be facilitated by AI PCs but activities that still require human beings.

All images were created via Microsoft Copilot on Edge except where noted in the image caption.

For more serious insights on AI, click here.