Harvey on r/legaltech

By far the thing that worked best was demoing Harvey on specific matters / work product the lawyers were involved in. So for litigators I would go to PACER and find the last brief they wrote and then have Harvey draft counter arguments. This worked super well with litigators because they like lasered onto the screen reading every word. Was a bit risky because if something hallucinated they would say it was wrong (this was super early days in like 2022 so we didn't have as many ways to reduce hallucinations) but when it worked it they were super engaged. For M&A you just went to EDGAR and did the same thing - worked well but not as well as the briefs with litigators

We need to earn that valuation everyday.  Short term, I think if you asked investors they’re looking at a combination of (1) math and (2) momentum.  On the math side of things, there are around 10M global legal professionals, and Harvey serves just single digit percentage points of them.   On the momentum side, we’ve seen an 81% increase in DAU / MAU (percentage of daily users over monthly) since launching in 2023.  I think even more exciting, folks that use multiple product surfaces have usage stats similar to Slack or email.  Shows that we have a long way to go with building out a full platform of use cases for lawyers.

Long term, it seems clear that technology penetration in the legal market will grow significantly with LLMs.  Or mabe another way to phrase it: what percentage of legal workflows are tech augmented today, versus tomorrow?  Today, the legal tech market is around 30B in comparison to the total 1T legal market.  The simplest answer here is that the tech penetration into the legal market is going to change massively and if we build a great product we hopefully capture some of that very large upside.

Probably two other related points here.  First, I don’t see this as job automation, but rather task automation.  I do think a lot of legal tasks will get consumed by technology, but that doesn’t mean the entire job of a lawyer gets consumed, it’ll evolve.

Second, I think this also speaks to a lot of questions about whether there is still room for other legal AI startups: the answer to me is very clearly yes.  I don’t think a single player is going to capture all of the pretty enormous amount of value that will be created in the next ten years in this space.

Lexis is an insanely trusted data source, RELX has been around for a very very long time.  It’s impossible to quantify the percentage breakdown because it varies so much by use case, but I think it’s safe to say customers who use both see a tremendous amount of value from the integration.  Re content, we are working with the Lexis data to build workflows on top of it.  We announced motion for summary judgement (MSJ) and motion to dismiss (MTD).  I think if we can nail these two it’ll be a really cool step change in data+drafting.  Hopefully we’ll have some more in the near future!

I think most lawyers will work the way partners work - you spend the majority of your time with clients and giving high level guidance to associates / agents. Partners primarily work through email and I think user experiences will be similar to that. Harvey can help unify everything and provide that experience at scale and securely that works for a law firm.

I think the only way to onboard future lawyers (and anyone) is to use the models a lot. Banning AI feels like banning the internet. There is obviously a balance of how you handle testing / cheating / etc that I trust universities to find the right solution.

We have talked to and worked directly with Courts, most notably the Singapore court system, on data and other access to justice initiatives. While Courts worldwide are definitely in much different places on their AI journey we've seen a pretty universal consensus that making institutional data more accessible to society is a huge positive and AI is a good forcing function to publish that data and a great way to non-lawyers to better understand it.

Beyond courts, there are some other great institutions doing this work like the Free Law Project and Harvard's CAP that we've talked to and worked with on data accessibility and quality

1. Linkedin messaged everyone.  tbh one really responded until A&OS press then everyone did.
2. First intro to first customer came from an investor / sarah guo.  She met an ex A&OS partner at a stanford class and he asked about us.
3. Volume, it literally was just reach out to as many people as possible and then do tailored demos.  The best was find something a litigator had filed in federal court and then harvey argue against the motion / brief ect live.  People REALLY paid attention if it was their own work.
4. We’re not profitable today and are not optimizing for that right now

Take risk and build up a tolerance for failure. This is by far the most difficult thing for junior attorneys imo. We are basically "graded" on not taking risk - getting the best grades, memorizing everything as perfectly as possible, having perfect test scores, etc. The reality is I probably fail at something 20 times a day and the hardest part of the transition was just getting use to making mistakes and picking yourself up after them. The good side to this is its the best way to learn imo

probably the three main drivers were (1) building out very vertical specific workflows / features, even down to the pass through defenses in pharmaceutical antitrust cases level, (2) focusing immensely on security, (3) integrating with current systems to reduce friction for lawyers

We are definitely more capability constrained than cost constrained. I think legal is a unique domain where the value per token is very high and so we aren’t as focused on optimizing cost per queries the way a consumer product might be. For example, when building tools for discovery or diligence our goal is to get high performance by using many large model calls rather than prematurely optimizing for cost.

Our longer term thesis is the cost per token for a fixed amount of intelligence will continue to decrease (currently halving every 6-12 months) so for most application layer companies the goal should be to build the best product. We have started hitting the scale where we do need to balance these things so there are some cost considerations but definitely more focused on quality than cost.

Winston and I met through mutual friends when we were both living in San diego. We became roommates and best friends and never thought we would start a company together. I was brainstorming assistant startup ideas with some of my friends from Google brain and Winston said "why don't you build a legal assistsant?" and showed us his legal workflows. That was a light bulb moment for us. We cold emailed sam and jason (GC at OpenAI at the time) with the pitch and raised from them before landing any customers

Totally, we're integrating with anyone that will partner with us - so far iManage, NetDocs, Lexis, WK and a lot of others. We also have plans to integrate with more legal AI startups, especially up and coming ones that need more distro / users. We can't build everything, best way to integrate and partner.

We are going to announce something around this next year!

I think the best way to look at this is to define why a profession / role exists.  On the commercial side, lawyers exist because people disagree.  Like that is the simplest reason why we have them - so the question is do we think AI is going to make it so people disagree less?  I highly doubt that.

Customers can integrate with document management systems such as iManage, Sharepoint, Google Drive, etc. to make large repositories of documents available in Harvey while maintaining the same strict access controls as the source system. So if you don't have access to something in the source system, you won't have access to it in Harvey. The access control boundaries will be strictly enforced in Harvey and Harvey does not train on customer data. So for example if your colleague uploads data about a particular case to Harvey, it will not influence the responses that Harvey will give you in the future for your queries.

The core issue is that making an individual lawyer more productive doesn’t map cleanly onto making an entire client matter cheaper or more efficient. Legal pricing isn’t really about the sum of hours worked. It’s about delivering a complete outcome for the client. And the economic value of different hours varies enormously. For example, an hour spent by an M&A partner structuring a deal is far more valuable than an hour an associate spends reviewing an NDA - yet the price difference between those hours is only around 3x. Billable hours don’t reflect the true value distribution within a matter.

Another challenge is the diversity of practice areas and client relationships. A large private equity firm structures its fund formation pricing very differently than how a Fortune 500 company structures its litigation spend. These relationships are long-standing, complex, and often highly customized. There’s no one-size-fits-all pricing approach that can simply “pass savings on.”

We are seeing firms experiment with new pricing models—especially for high-volume or repeatable workflows like diligence and discovery. These are increasingly moving to fixed-fee or hybrid arrangements. Importantly, this shift began even before generative AI. But AI is accelerating the conversation.

In the long run, both firms and clients stand to benefit. Harvey is already enabling discussions where firms and clients restructure matters in ways that are more profitable for the law firm while also providing clearer value and predictability for the client. That’s ultimately our goal: to help both sides rethink how legal work is delivered and priced so everyone wins from increased productivity.

No one has solved legal research / or come closer to it tbh, I think folks thought this would be easy with RAG and the two largest cos are legal research providers obviously but there’s been very little improvements here

sup

They should put WAY more resources into building their internal AI teams.  Law firms have not invested heavily in tech (either internally or from the vendor side) and they should 100 percent do both.  No way to understand your AI strategy and where vendors fit in if you haven’t invested here

Short answer is yes, we have an existing access to justice program in Singapore. As you know, there are many local and regional considerations that impact our ability to scale this work. However, Gabe and I are super committed to this as a North Star for how we scale Harvey.

Harvey provides a couple ways to do this:

1. Workflow builder (no-code visual builder of workflows) - this is what we see firms most commonly using to build custom and practice area specific workflows.
2. APIs / MCPs - we provide the ability for law firms to connect to Harvey in both directions (add internal systems to Harvey and pull content from Harvey) in order to build more complex integrations.

Our goal is to build an extensible platform for lawyers and innovation teams to create custom and proprietary workflows that leverage their and their firms expertise to help their clients. I’ve talked with lawyers that have trained their own small models and built really cool systems and we want to support this because we think there will be increasingly technical lawyers and we want to help them put their expertise into Harvey.

Re languages - the language shouldn’t be a big blocker because you can expose these via APIs / MCP so haven’t seen this to be a huge blocker beyond the traditional integration work required. This is something we plan to invest heavily in next year.

Re MCP - this is still early days in terms of maturity but we are seeing more and more firms start to think about it. We hope to help firms solve a bunch of the security and data privacy problems around this when deploying at scale on very sensitive data but we are seeing very interesting unlocks here for firms.

Our bet is that context windows will continue to improve but today the effective context windows are much smaller than the token context window (i.e. the models ability to reason significantly degrades even before you max out the effective context window). For this reason we see huge performance gains by encoding domain knowledge into our AI systems (e.g. show an agent how a human lawyer would process a dataroom).

We’re not currently working with the US Government. When we do make those decisions, they’ll follow the same approach we use for all hard calls: talk with all relevant stakeholders, evaluate intended and unintended impacts, and ask how the tech can benefit the most people while minimizing harm.

This is exactly the kind of societal challenge AI forces: deciding where AI should or shouldn’t be used. We're not dodging that responsibility.

Everything after PMF is hiring and building and retaining the best team, and I mean everything.  I spend almost all of my time outside of product and customers on hiring.  You have to literally just obsess over getting the best people, retaining them, and making sure you promote internally as much as possible.

- Tokens per DAU / MAU → Users are using tens of millions of tokens a month.  We actually went from 1T tokens processed in January to doing 12T+ this month (May).  Most of growth is from the change to cloud agent infra.

- % Scheduled usage → We just started rolling out long horizon agents with scheduled jobs etc so it’s still a very low percentage of usage.  We definitely think the trend will eventually be lawyers are managing multiple of these at scale, kind of how you send different tasks to different lawyers on a team.

It’s a much bigger business to build the AI infrastructure for every law firm and in-house team

Network effects - enterprises and law firms use harvey to find business and collaborate,.

Custom models - we’ve helped law firms and their clients co-train models on their unique relationships

Platform - we’ve spent over a decade building features for law firms and enterprises that solve their problems and cover all the problems the horizontal players won’t

Ecosystem - we’ve trained a massive GTM organization that specializes in helping law firms and in-house team transform their business as well as an implementation and integration ecosystem (this is how Bret Taylor describes one of Salesforces big moats)

Thank you for being a power user of Harvey, we sincerely appreciate it. And yes, connecting and rewarding power users is very much a focus for us for next year, so stay tuned. Our goal is three-fold: 1 - increase the number of power users in each firm by enabling them to learn from what you’ve created because it fits your organization’s core needs. 2 - highlight your work across the community and thank you for being a Harvey power user 3 - to make it easier for you to learn and teach others how to use Harvey. It’s on product, marketing, and customer success roadmaps to prioritize this next year, so stay tune.

We see a couple areas of differentiation between Harvey and the horizontal model providers:

1. Better legal AI - we can make our AI better at legal because we are just focused on that compared to the horizontal providers who have to make it better at everything.
2. Making teams more effective - our goal isn’t to just make individual lawyers more productive but teams of lawyers more effective on an entire client matter. Much of this product is not simply a model intelligence problem but all the practice area specific features and process management required to coordinate complex legal work.
3. Legal tool integrations - deep integrations with the legal tech ecosystems and law firms internal systems.
4. Governance - applying ethical walls and all the other governance law firms and their clients require to perform highly sensitive legal work at scale.
5. Making law firms more profitable - working with law firms to think about how our platform can improve their business. Making an individual lawyer 20% more effective doesn’t necessarily mean a law firm is 20% more profitable.
6. Client collaboration - we are building infrastructure that allows law firms and their clients to securely share data and collaborate.

It’s critically important. About 20% of our team at Harvey are lawyers, and that’s intentional - our goal is to build with and for the industry we serve. From a product development point of view, lawyers and engineers sit side by side and work on everything from how we build a workflow to how we evaluate the success of a new model in Harvey. It’s also critically important that we include feedback from our users, so we spend a lot of time with internal legal teams and engineers collaborating but also our product managers spend a considerable amount of time soliciting feedback from customers in the development phase.

- In general, if you are doing simple contracting like a one page NDA and it’s not connected to any past agreements etc) the base models can be fine.  As you increase in complexity - inside a firm handling complex, multi-party, multi-jurisdictional matters, the models stop being enough.

- For contracting specifically, we just announced contract intelligence - the main difference here for contracting is we’re trying to create a platform that helps you automate contracting based on what you’ve done in the past (trends, risk tolerance, etc) and do this across an entire org instead of just for an individual.

In the beginning the main reason people thought legal was a bad industry for AI was because accuracy is so important.  I think folks didn’t realize that law firms already have the perfect quality control processes already in place → an associate does a first pass and then there are already a lot of review layers baked in.

it's going to become a HUGE problem and you're going to need different UI / UX for reviewing different workflows - I actually think this is going to be one of the largest differentiators long term for vertical cos, simply hwo do you make it super easy for folks to review outputs in their domain

We have a great partnership and deep integration with LexisNexis and Wolters Kluwer. We are also building our own global caselaw dataset to fill any product gaps we can’t cover with partnerships.

In some places it is a pure coverage problem, there are a lot of countries where there is no major data provider (or even consistently digitized data) but lots of lawyers doing great work. More commonly, the data is there but it is not enriched in ways that make it maximally useful for AI. Think things like knowledge graphs and relevant metadata + some novel ones specifically built for agents that we used in projects like Harvey Moot.

The latter is harder to solve with partnerships because it often represents an entirely new class of data problem and requires first-party modification of the data. We've learned that, for AI, content alone isn't king and finding new ways to optimize that content for agents is also what makes our collaborations with the major publishers so effective.

We integrate with iManage and NetDocs - this lets us provide enterprise search over a firm's documents without need to copy the data to our server.

We also increasingly have customers storing more data in our vault product which can be used to index a firms documents and provide isolation and other security guarantees around that data.

No plans to be a law firm but we are starting to support smaller firms, non profits, and government agencies

I think our biggest differentiator from them right now is our scale and infrastructure. We’ve solved a lot of challenges around multi-model / cloud, ZDR, retention, workspace configuration, security, governance and many other problems required to roll out to massive firms and enterprises that I think other players haven’t solved. We also are significantly ahead on client collaboration (law firms and enterprises working together in shared spaces) and model training.

We already provide functionality in the product that allows law firms to apply custom client matter logic as well as ethical walls and permissions. We are also working on a number of partnerships with the DMS providers and ethical wall providers to directly integrate with these systems to allow law firms to impose their ethical walls on Harvey at scale.

We believe this will be one of the big differentiators of Harvey compared to the horizontal products. Exactly to your point a huge reason that law firms need custom legal tech is because of exactly this reason and working with firms to develop this has been an incredibly interesting technical challenge especially when you combine it with all the new challenges arising with GenAI.

First, thank you for being a customer. Second, we shipped a lot of improvements on Word mid-November but we're nowhere near finished. Our internal legal team has been collaborating with our EPD org to ensure we get this right given how foundational it is to firms and enterprises alike. I'll make sure to pass along this feedback to our team :)

Happy to be there!! I visited in September for the first time and can't wait to go back.

There are amazing point solutions out there - especially in the IP / patent area

Some are just starting to, enterprises are way more open to it overall though

Gabe is responding to the first question as we speak, but wanted to answer your last question directly. It's unclear to us that that individual actually worked at Harvey at all, but our DAU/MAU and renewal rates would suggest that person has inaccurate and/or outdated information.

Fine-tuning is one of many techniques we use to reduce hallucinations. Other techniques include rag, inline citations, agentic methods and other applied research methods. I think one common misconception in legal is that an AI system needs to be perfect and have 0 hallucinations to be useful. Our experience has been that these systems are incredibly valuable and the rate of hallucinations will continue to decrease.

We do benchmarking with third party vendors and with many of our clients. We are working on ways to scale evaluation and make it more transparent.

We are now over 3 years old and have gone through multiple renewal cycles and have very strong engagement. Some of our early customers regularly see 80-90% monthly active users and strong weekly and daily active users :)

Yes! We’ve seen lawyers and law firms build really complex and impressive workflows. On the law firm side:

1. Merger controls - A&O built a workflow that for a cross border M&A can check all the thresholds and filings required by country. It is able to analyze and flag any relevant considerations for that stage of the M&A. This is being used actively in many of their deals.
2. Comment memo responses - in PE fund formation one task is responding to all the LP comments which are aggregated into a large word document. We’ve seen firms build workflows for their PE clients that can take previous fund vintages (e.g. a previous fund with the same LPs) and leverage that knowledge to take a first pass at answering the comments.

Some cool in-house ones:

1. Horizon scanning - large in-house teams and financial firms are really interested in building ways to stay on top of the changing legislation. We are enabling workflows that allow them to pull relevant filings and flag relevant changes over time.
2. Investor relations - workflows to generate documents for investors or public filings is maybe a more obvious but very valuable one.
3. Our GC wrote a full post on how our in-house team uses Harvey on a daily basis if you want more specifics to try out.

So I actually think a lot of what makes an incredible lawyer today is still what will make one tomorrow. We are decently large consumer of legal services ourselves (although we try to streamline as much as we can with Harvey :)) and I've found that the best partners are the ones that are incredible at understanding what the actual business needs are and framing an agreement based on that. Same goes for litigation, it's who can come up with the best arguments / story not who is the best at going through emails in discovery.

So overall, I would advice young lawyers to find as much client experience as possible. That's actually the main thing I pitch to firm leaders - they should focus more on giving juniors client experience, and be okay with them making some mistakes - that's how they become the best partners in the future.

I think software businesses are going to drastically change in terms of what products and organizations look like. But I think there will still be very successful enterprise and vertical software companies. I think people hugely underestimate how difficult it is to build an incredible product and a business around it that customers are willing to bet their entire business on for over a decade.

We get the moat question all the time and describe our vision of becoming a platform for all legal work and how this enables collaboration across firms and their clients and enables unique model training and data collection.

Invest A LOT in post sales.  If you are selling into law firms the worst thing you can do is show up great before the sale and then disappear after.

Our pipeline is made up of a lot of different evaluation signals. Anthropic had a great post about the swiss cheese of evals that is a pretty good general description. Big pieces include:

Benchmarks / auto-evaluations -- these are cobuilt by our research teams (legal research and applied research) and mostly used to judge core model quality for legal Human evaluation -- run by our legal research and human teams, this includes both preference and other forms of expert review

Product evaluation -- owned by individual product teams and targeting things their products uniquely care about (e.g. retrieval accuracy, citation quality, etc.) Online metrics -- run by our analytics team to tell us how our evaluation are tracking online performance (things like thumbs up / thumbs down, etc.)

On the product side it’s been difficult to get the balance right between building for admins and end users.  In the beginning you had to build a lot for admin otherwise your features would never actually make it to the users.  Past year, almost everything was for end users but it surprised, me how much in the beginning you had to build just to get your product to users in the first place.  The usage increase in the past five months has really surprised me - after our switch to cloud agents.

We use both - we started with RAG before coding agents using bash tools started working super well. Now it depends on the use case but increasingly seeing code models with access to file system tools working very well.

Assistant has already been migrated to a coding agent under the hood!

A few key things, (1) there were no shortcuts, we spent an incredible amount of time earning their trust, (2) you have to look for early champions - for us we found that in PwC and A&OS, and (3) you have to be willing to listen and act, not just pitch your own vision.

We're thrilled to have Ryan on board! We hired him to partner with the innovation community and so far the reception has been amazing. Our motivation was as simple as that.

One hundred percent faced this and still do a lot! I would think of your age as an advantage (1) there are a lot of lawyers and legal tech folks in the community who are really interested in helping junior folks succeed as long as they admit they are learning machines and aren't arrogant / act like they know the profession better than the experts, (2) be open to mentorship and look to hire folks from the industry that can help guide you - because you are young folks are more willing to help you out, and (3) you aren't stuck in your ways - there are a lot of folks that have been at it for so long its hard for them to think of doing things differently. Don't let this get you down, trust me it is a continuous battle :)

We recently released our legal agent bench which shows some of these results across practice areas and task types for both open and closed sourced models. We will be releasing much more of this to help firms and lawyers understand where to use which models.

Not as a law firm delivering legal services.  I do think some tasks are going to get so compute heavy that you could see a world where during an M&A for example, a client pays a law firm an extra X amount to process documents faster in diligence.  I think law firms would be able to pass through these costs in some instances.

Common law and civil law systems tend to have differences in the value and hierarchy of legal information. For instance, many civil law systems have a much weaker (or no) concept of stare decisis and therefore prior cases tend to be less relevant to making present arguments than they are in, e.g., the United States. Ensuring models understand these differences in both how they search for information and how they present it is critical to making them effective in each system. But worth noting these reasoning differences come from a lot of places beyond just the common / civil divide

Probably not - individual productivity is not equal to organizational productivity. Also legal seems similar to programming. If legal gets higher quality and cheaper then people will consume more legal services. In the long term if these models get really good and the productivity is more like 1000% then things get more complicated and hard to predict

The biggest difference is they operate as multiple entities with completely different decision-makers. It's very similar the verein firms, but we worked with PwC first so it was our first time encountering it. The biggest opportunity we have there is we have a lot of users that are "legal-adjacent" in tax or deal / m&a for example. It's a great user base to test these use cases out

Gabe answered your question on power users, but I think your point is an important one. Our CS team and marketing teams are working to make sure it's a whole lot easier for innovation teams and internal champions like you to help people get started with Harvey.

The key on both of those was the commitment to co-build and buy in from the partners. We had some of the top partners in the world building and trying the workflows themselves, which created better output for our customers but also built a strong partnership from the start.

We already allow this via APIs and have an MCP announcement coming soon :)

We shared some model training results today and we see benefits from both approaches. Different parts of our products will likely use different solutions. But initial results post-training open source models with custom harnesses specialized for legal look very promising.

Yes! The open source post training results we shared today suggest we can match frontier performance on many legal tasks.

We have had a massive influx of demand from in-house legal teams. Benefits include: one system of record for the entire team to work in and reference, collaboration opportunities with adjacent teams, (ex: one of our in-house customers said her finance, IR, and HR teams all want seats), custom workflows that reflect your company's ways of working, and Shared Spaces, which will enable collaboration between firms and clients if teams want to enable that functionality.