ChatGPT and Knowledge Strategy
Building low-cost cognitive assistants
ChatGPT is catching the world by storm! It is everywhere. My friends are constantly sending me examples of the fantastic and unexpected things that ChatGPT has done. ChatGPT reached over a million users in just five days.
I am not surprised at how good ChatGPT has become. But I am amazed at the incredible growth of the awareness of these tools in the last two months. And that has encouraged me to help everyone understand how ChatGPT and large language models will impact your overall knowledge strategy and your ability to build cognitive assistants for all your knowledge workers.
Many readers think of me as an enterprise knowledge graph adoption strategy expert. However, I have followed large language models closely for several years since my friend Ravi kondadadi convinced me about their potential in 2019. I started writing about GPT and BERT to help teachers generate STEM lesson plans, glossary terms, and multiple-choice questions. And with every release, GPT got better. It became easier to generate the right content from a simple prompt.
Now OpenAI has added reinforcement learning to their workflows to once again fine-tune their models. The results have been stellar. And the generative AI is riding the hype curve up to levels I didn’t think were possible. But it is essential to pause and reflect on what is going on here and how generative AI will impact the productivity of our knowledge workers. The key is to appreciate that GPT can be quickly fine-tuned cost-effectively to build cognitive agents over general knowledge for specific repetitive tasks that knowledge workers perform.
Libraries and Search
The way I would like to approach this is for us to compare generative AI to internet search. Before we had search engines like Google and Bing, finding knowledge took a long time. We had to drive to the library and ask the reference librarian for help. The librarian would then consider our prompt, reflect on it, ask clarification questions, and then guide us to the right bookshelf in the library.
Search
After search engines came out, we depended less on our local library and the librarians. We could type keywords into Google, which would return a summary ranking of the most relevant and popular documents it found on the web. We then had to scroll through the documents, find the right one and then open the document to find the information.
Although many companies had internal search tools, they were not very good at retrieving the correct documents. The content you checked 10 minutes ago would be buried 100 documents down in the search results. As a result, our employees usually did a Google search first and a Sharepoint search only as a tool of last resort.
ChatGPT
Now with tools like ChatGPT, our world has changed again. We no longer have to open the files and find the right text fragment. ChatGPT will generate a detailed narrative answer for us. ChatGPT synthesizes many different sources of knowledge and forms a coherent body of text complete with bullet lists, sample code, and explanations of how to install the libraries needed to use the code. Furthermore, we can ask it to explain more details in the following prompt if we need more information.
Limitations of Search
Now let’s look at the fundamental problems with public search engines:
- They don’t include your organization and personal documents. They can’t tap into your notes and your personal knowledge graph.
- They can’t synthesize the results together in new content.
Generative AI is training on large collections of documents. But they don’t just build reverse indexes for fast search. They build neural networks that generate embeddings for documents. These embeddings are set up so that knowledge is stored consistently based on concepts, not just keywords.
How Generative AI Can Help
Can tools like OpenAI GPT help here? The answer is most certainly yes. Although ChatGPT does not have a way to be fine-tuned on your company’s documents today, the OpenAI GPT systems can be easily fine-tuned. Creating a set of prompt-response pairs from your internal documents allows us to create customized versions of generative AI that get smarter the more data we have.
Lowering the Cost of Cognitive Assistants
How do you do this? You work with each business unit to understand its key challenges. You harvest their documents and build knowledge graphs. And when they have questions, you let them type them into a natural-language interface. If you can answer the question by generating simple documents, that is step one. If you need to execute a query, you need to extract the parameters from the document, execute the graph query and return the results in the appropriate medium: text, tables, or charts.
Every Tech Forward organization will have hundreds of cognitive assistants that help their knowledge workers a few years from now. Just like our employees expect they can access Google, they will expect tools like ChatGPT. But here is a big difference: their future versions of ChatGPT will be fine-tuned on their internal and personal knowledge graphs.
GPT fine-tuning costs on the Microsoft Azure system are currently priced at around $84/hour. $84/hour seems high when we realize we can use the powerful new Habana Gaudi servers with 8 HBUs for $13/hour. As other organizations build GTP fine-tuners, the cost will come down.
Summary
In the past, knowledge workers were forced to use search engines that didn’t integrate and synthesize their knowledge. Now every knowledge worker in your organization can have their business processes streamlined using cognitive assistants that take in natural language questions and return coherent synthesized knowledge.
