The current revolution in generative AI owes much of its success to the large language models (LLMs) that have been developed and refined in recent years. Based on powerful neural architecture, LLMs are sophisticated AI systems capable of modeling and processing human language. This technology is now reshaping every industry and profession by automating mental tasks and optimizing routine operations.
At DreamProIT, we collaborate with clients from various industries, including the legal domain. Recently, we conducted research for one of our legal clients and gained valuable insights into the practical applications of Large Language Models (LLMs) in the legal industry. In this blog post, I’d like to share our findings with you.
Understanding our client’s domain: Nuances of the legal sector
The legal domain is a vast and complex field with a rich history dating back centuries. For hundreds of years, laws were created and modified within a paper-based framework, where any change to a law relied on making amendments – a natural language-based system. This paper-oriented approach to changes created a tangled web of scattered legal texts, which made the process of legal research a time-consuming and challenging task.
The introduction of digitalization brought transformative changes to the legal domain, bringing in digital linking systems. These new systems enhanced accessibility and streamlined retrieval of legal information. However, they have also posed new challenges, such as ensuring the accuracy and relevance of digital links between various documents and keeping track of changes in them.
How is the work with laws organized now?
Currently, we rely on complex ETL (Extract, Transform, Load) pipelines and sub-systems that are designed to efficiently process large volumes of domain data. However, a significant challenge in developing and maintaining these systems lies in the complexity of source data. The data is often inconsistent, unpredictable, and subject to frequent changes, often requiring us to handle various edge cases, which makes development more complex and time-consuming.
How can Generative AI help us?
As Generative AI, and LLMs specifically, have made such a huge impact on the technological arena, our client has decided to research the possibility of integrating the LLMs into our existing stack of technologies. The reason behind this initiative stemmed from the immense potential of adopting LLMs in the legislative field.
LLMs can extract and interpret natural language linking systems – a challenging task traditionally requiring robust software infrastructure for parsing and comprehension. This capability to interpret contextual nuances and understand intricate language connections can greatly simplify legal-related tasks. It can potentially help legal professionals receive answers to questions based on document data or summarize documents in different scopes (such as the whole bill or its sections) to review complex, lengthy documents with ease.
Key takeaways from our research on leveraging LLMs
To understand how to interact with this technology, we need to differentiate between the two types of LLMs – open-source LLMs and closed-source commercial LLMs. Each type comes with its own set of limitations, and it’s essential to be aware of them for effective implementation.
Open-source locally running LLMs
Open-source LLMs refer to language models whose source code is publicly available, allowing anyone to view, use, modify, and distribute the code. These models are typically developed collaboratively by the community of contributors in an open and transparent manner. Examples of open-source LLMs are often released on popular platforms like Hugging Face and TensorFlow to foster innovation and advance natural language processing as technology.
Here are a few things to consider if you want to run LLMs locally:
-
GPU requirement. Local setups for open-source LLMs demand a GPU to ensure optimal performance.
-
Cost considerations for scaling. Scaling LLMs for multiple users can be costly. It’s important to evaluate the financial implications associated with expanding the user base, considering hardware upgrades and additional resources.
-
Performance comparison. Open-source LLMs might have lower performance compared to closed-source ones. A useful resource for performance benchmarking and comparisons is available at this link, where various models are ranked based on their performance metrics.
-
Data privacy considerations. Local LLMs can work with private and uncensored data without any limitations.
Closed-source LLMs
In contrast to open-source LLMs, the internal algorithms and architecture of closed-source models are proprietary and not disclosed to the public. These models are often developed by private companies or organizations that retain control over their intellectual property. Popular examples of closed-source LLMs include ChatGPT developed by OpenAI and Google Bard.
Here are a few things to consider if you want to use closed-source LLMs like OpenAI’s GPTs:
-
Per-request cost. Closed-source LLMs often operate on a cost-per-request model, meaning users are billed based on the number of requests or API calls made to the model. Therefore, make sure to review the pricing details provided by the model provider, including request thresholds or pricing tiers.
-
Scalability for multiple users. While closed-source LLMs are easy to scale, the scaling typically results in increased costs, which is also important to consider if your user base is growing.
-
Robust performance. Closed-source LLMs often demonstrate top-tier performance across various benchmarks.
-
Data privacy concerns. Even though many companies expose their private data to such LLMs, handling sensitive information by external models may pose risks. Before implementing closed-source LLMs, it is better to thoroughly evaluate the provider’s privacy measures.
-
Censorship and limitations. These models may have restrictions on generating responses related to sensitive or private data, impacting the comprehensiveness of the model’s output.
The nature of LLMs as a tool
During our research of the LLMs, we have gained valuable insights that helped us better estimate risks and adjust project requirements. Here, we would like to share the key takeaways we learned about LLMs:
-
LLMs as a tool are basically a lossy compression of large amounts of data.
-
LLMs are stateless and contain only data they were trained on.
-
LLMs generate text based on a statistical approach meaning that there is no “brain”, “reasoning”, or “background considerations”.
-
The LLMs’ context window has a limit for the amount of incoming tokens. Our experiments showed that filling the context window almost completely may produce lower-quality results.
-
LLMs struggle with data exactness. In our experiments, they struggle even with the simple task of quoting back to your large text (filling up LLMs context window completely).
-
LLMs can provide you with made-up incorrect completions (so-called hallucinations).
-
LLMs completions must not be accepted as is, without any verification or testing infrastructure in place.
Closing thoughts
LLMs are very powerful tools for certain kinds of tasks. However, it is crucial to understand that this technology is not a magical silver bullet that can solve every problem. Understanding the limitations and shortcomings of LLMs is crucial for correct estimation and requirements solicitation. To overcome the limitations, developers must be smart, creative, and sometimes even cunning. In future blog posts, I will discuss how to make LLMs work with your own private data, so stay tuned for more insights and strategies.