Navigating GenAI: How to Review Prompts and Responses

Legal firms are increasingly turning to generative AI (GenAI) to streamline operations and gain a competitive edge, and understanding how to leverage GenAI effectively is becoming a distinguishing factor of firms that are leading in innovation. The capabilities of open-sourced and specialized large language models (LLMs) have created immense opportunities for quickly identifying knowledge and information, and to truly benefit from the capabilities of LLMs, users must be educated on best practices. Understanding how LLMs work and how to apply them to legal work, while simultaneously accounting for their limitations is key to success.

‍

In this article, we'll explore strategies for reviewing prompts and responses when utilizing GenAI, highlighting why it's crucial to grasp its proper utilization, and offering practical tips for maximizing the benefits of GenAI while mitigating risks.

‍

Maximizing the Utility of GenAI

Mastering the art of interacting with GenAI to extract relevant information from their firm's database saves attorneys time on repetitive and mundane tasks. 

‍

However, it's important to acknowledge that LLMs, while powerful, are not infallible. Instances of "hallucinating" or generating false information have underscored the need for users to exercise caution and diligence. Recent incidents, such as a federal judge's mandate for attorneys to verify the accuracy of AI-generated filings, highlight the importance of human oversight in the use of GenAI tools when drawing information from open-sourced models in particular.

‍

While hallucinations are possible, GenAI tools are still highly efficient and reliable for law professionals when used responsibly. Additionally, tools like DraftWise, which generate responses specific to a firm’s proprietary data, eliminate the risk of hallucination.  

‍

Prompt Engineering: What’s a Prompt or Query 

A prompt, or query, is a request to receive information from the LLM. Research has shown that GenAI technology is highly responsive to how prompts and queries are written. Even small differences that may seem insignificant to users, like word choice or phrasing, have been shown to create different responses from GenAI. To obtain optimal outputs, users should familiarize themselves with best practices.

‍

Writing Effective Prompts: 

Here are some tips & tricks for writing optimal prompts and queries: 

1. Aim for clarity by using straightforward language and avoiding ambiguous phrasing in your prompts.
2. Be specific by providing the AI with all the relevant details and context needed to understand and respond accurately to your query.
3. Don't be afraid to experiment with different prompt formulations to see which ones yield the best results.
4. Before using a response, review the AI's outputs to ensure they are factual, relevant, and aligned with your intended purpose.

‍

Reviewing Responses: Best Practices for Responsible Use

While GenAI continues to evolve, human oversight remains critical to ensuring accuracy and ethical compliance. To minimize the risk of errors and misinformation, consider the following tips:

1. Know where your AI tool is pulling information from – ask your technology vendor what data the AI model in use is trained on and formulating responses from
2. Cross-verify data given by an AI tool – when in doubt, attorneys should verify the data given by an AI tool, just as they would double-check the work of your colleagues.   
3. Understand what the GenAI tech can and cannot do – attorneys should research the capabilities and limitation of any legal tech built by LLM and security experts 

‍

Staying Ahead of the GenAI Learning Curve

LLMs are quickly developing, and while they may not be able to answer everything we want them to, educating attorneys on strategies for utilizing GenAI is the first step to achieving significant benefits. 

‍

Lance Eliot in an article for Forbes, details a Chain-Of-Thought Step-By-Step Technique for working with AI to achieve the best results. He suggests six major ways to apply this approach, some include entering your prompt into generative AI on a step-by-step basis, seeking a step-by-step elaboration to an answer it’s given you, or asking it to engage in an interactive, step-by-step dialogue with you. By doing this, AI can catch itself making any errors. 

‍

Pushkar P. Apte and Costas J. Spanos have suggested using techniques to help AI better understand the physical world and provide higher-quality responses. This can be achieved in four ways: synergizing AI with scientific laws, augmenting data with insights from human experts, employing other devices to explain AI’s decision-making, and using models to predict behavior. The main objective is to anchor GenAI technology in something real without adding more data into the AI system. These strategies have proven to be successful across multiple industries, including retail and manufacturing.

‍

GenAI Provides Lasting Benefits

While generative AI is not without limitations, its benefits outweigh the potential risks when used responsibly. GenAI can save attorneys valuable time by automating tedious tasks that don't require human creativity and expertise. This allows attorneys to dedicate more time to high-value work like researching cases, building stronger client relationships, and understanding client preferences. As a result, law firms can reduce billable hours spent on unnecessary tasks, driving cost savings. Additionally, GenAI enhances legal research by rapidly synthesizing large, complex data sets to provide comprehensive answers in a fraction of the time required in a manual process.

‍

Prioritizing Education: Empower Legal Professionals & Create a Lasting Culture of Innovation

In today's rapidly evolving legal landscape, mastering the intricacies of GenAI offers a distinct competitive edge. Firms that prioritize educating their attorneys on effectively harnessing GenAI, including crafting prompts and conducting thorough reviews, position themselves for success in an increasingly digitized world.

‍

By empowering legal professionals with the knowledge and skills to work seamlessly with GenAI, firms can unlock new levels of efficiency and accuracy. Those that neglect this crucial aspect risk falling behind their more technologically adept counterparts.

‍

In essence, embracing GenAI isn't just about adopting the latest technology—it's about cultivating a culture of innovation and continuous improvement. By investing in training and skill development, firms can pave the way for enhanced productivity, better outcomes, and sustained relevance in an ever-changing industry landscape.

‍