What would you do if you were running a 10K road race, struggling to get up a tough hill, and suddenly the rules of the race changed? What if drivers started picking up runners in cars and then raced each other to the finish line? Would you keep running, knowing full well you’ll place in the back of the pack? Or get in the car, hit the gas and compete for the grand prize?
In business today, AI is that car that’s disrupting the way companies run. Companies can still choose to move ahead the way they always have – developing long-range plans, adhering to processes, pushing employees to work harder than ever to succeed in increasingly competitive environments. But AI is changing the nature of the race. It’s giving companies a new vehicle to move faster and give workers new routes to zoom around problems. Any business that doesn’t take the wheel and instill the power of AI into its workforce will be left behind on that long, steep hill.
Embracing the Future by Becoming a Manager of AI
Here at Cockroach Labs, we learned very quickly that Gen AI can help us do things we never thought possible. We’ve used it across the company for gen AI search, recommendation systems and semantic search.
One of the best examples of how AI can transform a workforce process is taking place in our education department. Our team is using AI to accelerate the development of curricula that helps customers, partners and our own work force become experts in the operation of our database product line.
We recently created a course that featured 21 hands-on exercises and 20 slide decks with detailed student notes. Before starting the project, we estimated that, using our normal development process – factoring in industry standard estimates for how long it takes developers to produce one hour of content – this would take three to five months to complete.
So, what happened? Incorporating Gen AI into our existing processes, we were able to finish the task in five weeks.
In the process, we learned a number of lessons.
- We’re all managers of AI. Each of us has an opportunity to think very differently using AI. Each of us should act as managers, whether we have direct reports or not, because we manage a virtually unlimited supply of intelligence capacity that we can put to work on challenging projects. How much can you automate? How creative can you be? How effectively can you prompt your AI tool, challenge it, and deploy the new model it generates? You can harness it. You can manage it. You can do essentially as much as your own personal capacity will allow you to do.
- Don’t expect AI to do everything. There are tasks it’s simply not suited to perform. But you can task it to do things workers shouldn’t be doing anymore – jobs that are time consuming, but still require a degree of intelligence.
- Don’t blindly accept the results it churns out. Check, check and recheck. Trust in the technology, but always verify – because accuracy turns assumptions into achievements.
The Step-by-Step Process of Deploying AI for Task Management
Here’s a quick summary of some of the ways AI helped us get up the hill, to the finish line, much faster than we expected.
- Different models: Different models have different strengths. So, just like manufacturers use best-of-breed components when building a solution, feel free to swap models when it makes sense to take advantage of those strengths. We used Claude Sonnet 3.5 to author the first exercise draft because it excelled at creating engaging prose and instructions. We used ChatGPT 4o&”o” reasoning models as technical reviewers to refine commands and ensure technical accuracy in the second draft.
- Reproducible outcomes: When doing highly technical tasks, we wanted to be able to enforce clear technical constraints and produce structured outputs that supported reproducible outcomes. To do that, we provided explicit structure requirements and format examples.
- Prompts for highly technical tasks: Be very specific about what you ask AI to do –
otherwise it can do crazy things. Clearly state any assumptions about the inputs or environmental conditions and ask the model to handle unexpected cases.
- Refined prompts: It’s important to encourage AI tools to ask clarifying questions. First prompts won’t be perfect, so expect multiple rounds. Incorporate any improvements or steps that the model suggests back into your base prompt, and iterate with the AI and your teammates.
- Testing, testing, testing: Consistency checks are critical. One way to measure the effectiveness of your prompt is to ensure consistent output. So, we tested often to ensure that we were putting in the same input and that the output remained the same.
Human Expertise at the Helm: The Essential Role of AI Oversight
While AI removes time-consuming tasks from workers’ day, it doesn’t remove them from the workflows altogether. Humans still play critical roles in our curriculum development, and they need to be integrated in AI-driven processes to ensure that the processes succeed.
A good example is in how our education team conducts prompt engineering. Humans are responsible for crafting the initial prompt, including context from relevant sources. Then, after the Gen AI tool executes the prompt, the human reviews the output of the tool. It’s essential that this person is a subject matter expert who can catch errors early in the process. Teammates continue to collaborate with the tool and iterate until the team is satisfied that the prompt is ready to publish.
While this collaborative human/AI has proven to be effective, it does require a human to manage the context and transitions between models.
Without humans in the loop, teams would be at the mercy of AI tools that can be notoriously unreliable. When we first started with our curriculum project, the tools did well summarizing or explaining concepts, given the right contexts. However, they did hallucinate often. Today the models are better at reasoning, but a human still needs to manage the process. Now, humans can focus on review and creativity and not just on process management.
In the future, AI agents will take a bigger role in the process. Instead of humans manually gathering context from sources, crafting prompts with context, moving work between AI models, and reviewing and refining outputs, we’re developing agents that can perform a lot of these tasks – with a bit of help. The agent can autonomously collect and process source materials as context, generate skills taxonomies and course outlines, execute our established workflows, and present only key decision points to human experts.
Conclusion
While brisk runs are great for keeping in shape, cars long ago transformed humans’ ability to get where they need to go. AI is providing the same benefits in the workplace – helping companies improve processes and generate better outcomes. Those who embrace it and harness its compound efficiency gains will leave competitors behind.
