ChatGPT Will Change Work Forever… But First, We Have To Change Ourselves
Overcoming the "Garbage in, Garbage out" problem with generative AI requires major shifts in how we think about work.
Garbage in, Garbage out - DALL-E
I’ll never forget the Spring of 2018 when I bought my first iPod Touch. I was still operating on a flip phone, but this new device gave me a glimpse of the future. I got a taste of the Smart Phone revolution and I was hooked.
Do you want to know the first app I downloaded? It was a game where I had to navigate a ball through a maze using the device’s accelerometer.
This game blew my freaking mind.
I tilted my phone and the screen responded… That led to hours upon hours of gametime and searching for games to scratch the same itch.
When I think of my iPhone today, however, the last feature I think about is the accelerometer. There are some novel use cases that are still relevant today, but the shiny new feature that led to most early adopters pouring fictitious beer or popping on-screen champagne faded to the background as more practical use cases emerged.
The more I’ve studied the history of technological revolution, the more I’ve seen this pattern play out.
We play with the novel before we expand on the useful.
Today, this is the same phenomenon I’m seeing with generative AI tools like ChatGPT and Midjourney. Many have been quick to play with the novel use cases of this technology, myself included. The cover art I’m using for my Substack right now is all generated via DALL-E, the image generator operated by OpenAI. I’ve spent hours on ChatGPT trying to test its limits and better understand what it’s good at and where it struggles.
While there are a ton of fledgling startups that are beginning to emerge in this space, there are a couple of consistent blind spots I’m seeing. A lot of the “first-order thinking” being done around these use cases is missing the “second-order thinking” required to understand mass adoption and acceptance of the technology.
My Foundational Thesis: Generative AI is Only as Useful as the Information it’s Trained On
There are two types of applications of generative AI. The first one, and the one that gets the most attention, is the broad application of the technology. Using ChatGPT (or an equivalent model) to run side-car as an assistant based on its existing, off-the-shelf, training data.
This is the equivalent of the iPhone maze game. This technology looks incredibly cool and will blow people’s minds the first time they see it, but its overall usefulness is limited. There are definitely applications that make sense for it, but they’re the vast minority of use cases.
For most professionals, the real magic will come in the form of narrow application of the same technology. Instead of using these models off-the-shelf, the most effective users of generative AI in the coming decade will be focused on training these models on their own internal data. This allows you to craft an AI-powered co-pilot that’s far more useful than a bot that’s only trained on broad, Internet-wide training data.
This presents a problem, though… How do we get great at training AI on our own internal data? There are a few ways to approach this problem:
We can train it on everything: Throw the entirety of your company’s data at the AI and let it find the answers your team is looking for. That’s the bet that Microsoft is making with their Co-Pilot product. It’s an interesting thought, but unless your OneDrive is squeaky clean, well-organized, and includes zero documents that include false, misleading, or out-of-date statements, it’s likely to cause more hallucinations than a trip to Joshua Tree.
We can train it narrowly on select documentation: This is getting closer to the right solution. If you follow the /r/chatGPT subreddit, you’ll see a new solution every day posted that’s designed to ingest documentation and train your ChatGPT assistant on it, allowing you to get answers to any question about the documents themselves. This application would reduce hallucinations and eliminate any data privacy concerns that would arise in solution 1, but it creates a new requirement… Whoever is in charge of this bot must be a master of internal document organization and there must be robust systems in place to ensure documents remain up-to-date. I don’t believe this is a reasonable assumption for >80% of companies that could find this useful.
We can train it on an internal Wiki with built-in architecture and updates: This is what I believe the future of generative AI in the workplace looks like. It combines the ease-of-use of application one with the comprehensiveness of application two. By crafting a well-architected internal Wiki, you can ensure that the AI is trained exclusively on clean, accurate, and up-to-date data. By making the Wiki-building process user-friendly, you can ensure that companies don’t need a data architecture expert to build this documentation, but rather can do it easily themselves.
In the coming years, I believe we’ll see a Cambrian Explosion of AI-powered solutions similar to what we saw after the introduction of the iPhone. The companies who win this shift will be the ones who pivot away from the novel aspects of generative AI and begin to embrace the useful aspects of it soonest. This means getting world-class at building training data to apply in narrow use cases.
The Top Reasons Companies Won’t Adapt Generative AI are the Exact Reasons You Need to Clean Your Training Data
I’ve spent the past few months talking to a lot of folks about whether their organizations are expecting to adopt generative AI broadly in the coming year. While many bleeding-edge leaders are thrilled to use generative AI to make their teams more effective, I hear the same two objections over and over:
What if it gives wrong information? - The word AI folks use here is “hallucinate.” The chatbot can’t find the right answer, so it confidently gives the best answer it can think of… It’s a bullshit machine spewing bullshit. While that might be okay for the fun and novel use case of writing a rap about your pet cat, it’s incredibly scary for organizations that want their employees to trust the information its providing.
Training your model on a clean and up-to-date internal Wiki is the single best solution to this problem. I also believe the solution of the future will have a verification and validation system in-place that uses confidence intervals and Reinforcement Learning from Human Feedback (RLHF) to both communicate with users if they’re not 100% certain and get better answers for harder questions with human intervention.What happens to our internal data? - This is why I can’t get on board with Microsoft’s approach to an OpenAI integration. I could imagine a universe in which they can solve the hallucination problem, but sending all of your company’s internal data to OpenAI which then can and will use the data to inform its model is a mistake. The vulnerability risks here are huge. Not only is a ton of untrained and unfiltered data making its way to a third party, but that third party is then using that data to train itself to provide answers to other users. The probability that another user could query something that ends up returning your company’s private IP is super high.
By developing a “walled garden” where this training data lives, you can eliminate these risks. You still are using data with a third party that presents some level of risk, but the surface area of that risk is both reduced and known. This will remove a ton of barriers for companies worried about data privacy.
If we’re going to thrive in this generative AI world, we need to get great at training it. We’ve already seen what happens when AI models are trained on unsupervised data and it always ends badly. Better training data = Better AI = Better outcomes for everyone who uses the AI in your company.
The Internal Communications Tech Stack of the Future
I genuinely believe that the workplace will look fundamentally different in 5 years because of the impact of generative AI. I believe there will be seismic shifts in how we work and how we communicate because of these advancements, but I believe that these shifts will happen after we make some essential changes to how we work without AI.
The first step to ushering in this evolution is changing the way we think about internal communications. There’s an “old world” model where internal comms means crafting memos and company-wide emails from the CEO. That model is dying as the need for personalized communication is increasing. While generative AI will change internal comms forever, it’s the last step of the value chain, not the first. There are 3 pieces of the value chain that need to be assessed for optimization first… Your chat organization, your async communication platform of choice, and your internal Wiki.
In the future, every company will be responsible for building and maintaining an Internal Comms Tech Stack that includes…
Chat/Collaboration Tool
Async Communication Tool
Internal Wiki
Generative AI Assistant
In the coming weeks, I’m going to dive deeper into each of these solutions, highlighting specific companies and tools that I’d recommend in each area, and providing a framework for assessing each area of need as we evolve our internal communications strategies to meet the AI-powered era we’re entering.
What questions do you have about leveraging generative AI in your business? Hit reply to this email. I read every response and would love to answer your questions in a future post.
Tim Hickle is a marketing leader who helps high-growth startups and scale-ups get unstuck and hit their goals while embracing AI and the future of work. To learn more about how Tim can help your organization grow, visit TimHickle.com