AI workers will ignore your Slack messages too
Just like the rest of us
Modern knowledge work — whether you’re writing code, closing sales, or answering support questions — is effectively the process of orchestrating anywhere from 5-20 different SaaS services on a daily basis. As a developer, you read Slack messages, push code to GitHub, update task statuses in Linear, monitor your application status in DataDog, and so on. As a support engineer, you respond to customer tickets in Zendesk, file Jira tickets for the engineering team to handle, and (of course) read Slack messages.
This trend has only been exaggerated in the last 4 years as we’ve all settled into hybrid or remote work. Interpersonal interactions are intermediated by Zoom, and every organization is more depending on Slack than ever. This change has dramatic implications for the future of AI powered work. Most of what we’ve written about in recent weeks — pricing, building AI work products, and interfacing with humans — has been focused on high-level strategic choices that AI companies are facing.
The upshot of this evolution is AI-powered work products must integrate with the stack of tools that every enterprise uses. The challenge, however, is that integration no longer just means reading data from Salesforce or pushing updates to GitHub — the integrations that AI products provide must have the same level of context and nuance that humans have.
In the SaaS worker world, AI-powered work can’t exist without the ability to connect to all of these services bidirectionally. At a baseline, AI work products will have to do what the previous generation of SaaS products did: have the ability to pull the relevant data out of each of relevant system to analyze it publish results (e.g., task updates, ticket summaries, etc.) to each of these systems for human coworkers to read.
The reality, however, is quite a bit more complicated than just some basic reads and writes. People bring a wealth of implicit knowledge to their roles that will have to be learned by AI systems, and even when that’s the case, there will likely be gaps.
Integration and data management. The biggest challenge that remains to be solved is data scoping and access management. Before, you could rely on a user’s identity to determine what data they have access to — as an AI worker, you now have to account for the context in which the interaction is happening. Just because a senior exec asks a question on a public channel, you shouldn’t use their access privileges because that exposes information to everyone on that channel.
Similarly, there’s also a natural filtration process that humans do when sharing information. For example, a human support engineer will read a ticket summary that says, “Customer X had the following issue, and here’s how we resolved it.” That resolution will include some general principles and some customer environment-specific details. When Customer Y has the same issue, the support engineer will naturally filter out the company-specific details before sharing a suggestion — and they certainly won’t share who Customer X is or why they had the issue in the first place.
Data contextualization. Simply pulling in all the data you have access to will more likely lead to noise and nonsense than it will to useful data processing. Every company’s knowledge base is probably littered with half-written PRDs and detailed initiative plans that never actually kicked off. Treating all of that information as equal will lead an AI system to make all sorts of wild assumptions that simply aren’t true. The same will hold for work that’s planned but hasn’t been executed on. Naturally, humans will cross-reference the data they find with their own understanding and with other information — absent this understanding, AI products will quickly go off the rails. (This is one advantage that broad-functionality AI products have.)
What this doesn’t account for is the cases where humans make judgment calls that don’t align with best practices. For example, a single engineer who doesn’t explicitly have access to customer data might be told about an important requirement that’s currently under wraps. How you pick who that person is and track the fact that the normal process was not followed is an even bigger challenge. Realistically, it might not be one we solve in the near future.
Implicit learning. Humans learn implicitly from experience — sometimes without realizing it — and AI systems will need to learn to pick those lessons up. We’re all familiar with alerting systems that make too much noise and implicitly are ignored because almost all their alerts are false alarms. This is, of course, a dangerous problem, but it’s a great example of things that humans learn implicitly. The same is true of a product best practice that isn’t documented anywhere or context about a customer’s environment that was never logged. Without the ability to cross-reference past experience, AI systems won’t be able to effectively manage prioritization and will overwhelm their human coworkers with data to process. This is actually another form of data contextualization, but it’s important enough that we wanted to call it out separately.
None of these problems are insurmountable, but they’re certainly new — which is going to make for a fun couple of years as we figure out the best way to manage each of these challenges. As a community, our instincts are probably going to be to run off and build a bunch of infrastructure solutions for AI-native data management, but we’d warn against that. We love infrastructure products, but it’s incredibly early: Better LLMs might natively solve the access management or data context problems by virtue of enhanced processing capabilities, or more AI-friendly APIs might make scoping easier to manage. The next couple of years will make it clear just how generalizable these problems are, but in the meantime, those of us building in this space are going to have a fun time figuring it out.