Editor: Can you share with our audience the background for your interest, experience and expertise in managed review?
Hwang: I was in private practice for about 10 years, when I decided to join Black Letter Discovery (BLD) in 2008. Even at that time, Black Letter was at the forefront of providing managed review services in the industry. Black Letter Discovery has been around since 2003. In 2012, BLD was acquired by TrustPoint International, a full-service end-to-end eDiscovery provider.
Wilcox: I’ve been with Black Letter Discovery since 2004. I started off as a first-level document review attorney on a project that was expected to last about a month, and here I am about nine years later as the leader of our managed review operations in Ohio.
Editor: What is managed review, and how is it different from legal staffing?
Wilcox: What really differentiates a true managed review offering from legal staffing is the expertise around leading document review teams, which is very different from the practice of law. In addition, true managed review is about following a consistent, while still flexible, process or workflow using tools developed over years of experience in the litigation discovery trenches.
Legal staffing consists of recruiting and managing the payroll process for contract attorneys who supplement law firm teams (and more recently, corporate legal departments) in conducting large-scale document reviews. This model provides a certain amount of cost savings.
Editor: Why is this differentiation so important?
Wilcox: Ultimately, review is expensive, and you’re relying on the expertise of whoever is leading the review team to deliver defensible results within your budget. A strong managed review process ensures that those leaders have the tools they need to ensure the team’s productivity and the quality of its work product. That allows your outside counsel to focus on developing your litigation strategy and winning your case, while review process experts maximize the quality and productivity of the work done by your review team.
What really differentiates the managed review offering from legal staffing is the ability to partner with corporations and law firms in leading the review and to provide process expertise and leadership that’s directly targeted at maximizing the review team’s efforts. This allows law firms to focus on what they do best, which is to manage the litigation as a whole.
Our project management team is experienced in leading a managed review process that ensures that the review team understands the strategy and objectives of the client and outside counsel. They sit down with the litigators and talk through their strategy so as to understand their goals for the review. Together, they design a managed review workflow and review protocol that will deliver what’s needed. Through this process, the review team becomes an extension of the litigation team.
Thereafter, the team works daily to ensure that the quality of the work done by the review team is where it needs to be, that they’re working productively and delivering a high-quality work product. This delivers much greater savings at the end of the day than legal staffing alone.
Hwang: With managed review, the optimal goal is to partner up with a corporation so that we create a lot of efficiencies in the workflow. Once you’ve conducted one managed review cycle for a specific corporation, you can implement that workflow again, due to the institutional knowledge you’ve gained, and can avoid retraining the reviewers.
Editor: From your experience, do you have any advice for corporate legal departments to help them successfully leverage managed review?
Wilcox: First and foremost, the corporation should have a point person who’s directly involved in the oversight of the review process. While outside counsel typically have the deepest knowledge of the individual case and an understanding of the litigation strategy, having someone from the corporation directly involved in status calls ensures that the goals of the corporation – often cost control, in addition to defensibility and quality – are being met.
It is incumbent upon the corporation to weigh the value of the case, and then decide whether to spend more or less money on the review process. In a lower-value case, a corporation may decide to limit the breadth and scope of the review protocol to reduce the cost of review. On a higher-value case, it may be worthwhile to expand the scope of review. In general, the more issues that we’re looking for in a document, the more time it’s going to take to code an individual document. At the end of the day, the corporation’s point person needs to conduct a risk-benefit analysis. What is this litigation worth to us? What are the risks involved, and how much do we want to pay? Neither the managed review organization nor the law firm can answer those questions.
Hwang: I agree that this is extremely important. Most Fortune 500 companies rely on numerous outside counsel. We might be managing five reviews for a single client with different law firms acting as outside counsel in every case. If all five firms set up different workflows, even good ones, this will create a lot of inconsistency and inefficiency. If the corporation is directly involved, we end up following one consistent process and one workflow, which creates many efficiencies.
Editor: From your perspective, what are the key elements of managed review?
Wilcox: I would say number one is reporting, which we typically do every day throughout the course of a project. We report on the number of hours being worked by everyone on the team, what their function is (first-level review, second-level privilege review, quality control), and the number of documents reviewed in a given day. We then aggregate all those numbers so the individual who is managing the review can see day by day how many documents have been reviewed, what our pace is, and when we’re likely to finish. The reports allow us to monitor progress and ensure that we are on track to finish on time and on budget.
Another element of managed review is decision logging – essentially creating an audit trail for defensibility. If a review team is presented with a new category of documents that aren’t clearly described in the review protocol, we need to decide how to treat them. We reach out to counsel with our analysis, saying, for example, “We think this is likely but not certainly responsive under the protocol because of x, y and z, and we think there’s also a possibility that it’s non-responsive for reasons a, b and c. Ultimately, we think it’s responsive; is this correct?” Each time we raise a question, we document it in the decision log and create an audit trail. It’s important to frame the questions this way because it helps outside counsel to provide us with guidance quickly. When someone isn’t there to answer these questions, individual attorneys must make their own coding decisions to the best of their ability, and inevitably coding inconsistencies arise, so that audit trail is key.
Lastly, Black Letter delivers project management and leadership, which to us means having a leader there every single day who can keep a team motivated and focused, and, importantly, who can spot truly novel questions – those not already answered in the guidance – and elevate those up to counsel.
Hwang: When we implement this workflow for the company or client, we’re trying to achieve a process that’s defensible, repeatable and consistent, which is the standard the courts use.
Editor: From these elements, is there an ideal workflow that can serve as a best practice for the review process?
Wilcox: Every single case is different, which is why we meet with counsel early on in the process to learn what they are trying to accomplish in a case, then customize the workflow to meet those unique requirements. But while there is no single ideal process, there clearly is a set of best practices we’ve learned over the years that we bring to the table at the beginning of every engagement. These serve as a starting point so that counsel need not start from scratch. A typical workflow in the context of a commercial litigation review or a government investigation begins with a first-level review, which generally encompasses a decision on whether each document is responsive to one or more requests for production.
With regard to privileged documents – the identification of which is one of the critical functions of document review – in order to optimize both quality and efficiency, we typically recommend that first-level review attorneys focus on identifying potentially privileged documents, and then divert those into a secondary workflow for review by an attorney specializing in privilege review. This way, the first-level reviewer does not get bogged down, and it ensures that we don’t miss privileged documents on first-level review.
We then conduct a second-level review with attorneys trained and experienced in privilege issues who make final privilege decisions in close collaboration with counsel. This collaboration is critical because the decision as to how broadly or narrowly to assert privilege is ultimately a question of strategy. In a very high-stakes case, counsel may take a very aggressive position on asserting privilege any time there is a reasonable claim to be made. In a low-stakes case, counsel may decide to be more restrained in claiming privilege in order to avoid unnecessary fights and reduce costs. Solid review leadership and process ensures that the review team understands counsel’s views on asserting privilege and makes decisions consistent with those views.
The next phase is the quality control process. We implement a four-tiered quality control process that leverages statistical sampling to ensure that the individual attorneys on the review team are making decisions consistently with the guidance of counsel. We also engage outside counsel in the sampling QC process. This ensures that they are meaningfully overseeing the review team, providing feedback as needed, so that the results of the review match their objectives.
Editor: What are the quantifiable elements that clients should expect from a managed review provider?
Wilcox: In addition to the daily reports and the decision log/audit trail, we also log and track technical issues with the documents, such as corrupt files, files that won’t open, or files that are password protected. These issues need to be resolved prior to production, so by creating a tagging or logging system, we can organize these loose ends and efficiently turn them over to counsel prior to production for them to determine next steps. We are often long gone by that time, so it’s very important to plan for the handoff back to outside counsel.
Finally, critical in our process is the defensibility binder we give the client at the end of the process. The binder contains all of the process documents we’ve created during the course of the review, including the final daily productivity report, the decision log, and the review protocol. Challenges to a review can arise years down the road, and those who were involved in managing the review process may no longer be around. With the defensibility binder, counsel can answer any questions regarding decisions made during the review process. If six months down the road the court orders you to produce a category of documents you’d decided weren’t relevant, it’s extraordinarily helpful to know when that decision was made and how those documents were treated in order to go back and identify those documents for future production in an efficient manner.
Editor: What sets Black Letter apart?
Hwang: In addition to our customizable reports and the defensibility binders, we provide Class A secure facilities for our clients that are optimized for managed review.
Wilcox: I believe what really differentiates Black Letter Discovery is a commitment to both the corporate client and outside counsel. We made a decision early on at Black Letter to serve all constituents and to really harmonize and optimize our process because when you have harmony, quality and efficiency follow. When there is friction, people tend to be less productive; they can even miss things. We believe in getting all parties to the review involved early and working with them in a meaningful, collaborative fashion.
A couple of years ago, Ralph Losey posted a great guest entry on his eDiscovery Team blog about storytelling and excellence in document review. It’s still very relevant today. The upshot is that building a strong case is about developing and telling a great story. Your documents are often core building blocks in that story. At the beginning of the review then, the transfer of knowledge from the litigation team to the review team is extremely important. Tell your story to the review team, as if you are giving your closing argument to a jury. Get them excited about your case. Then open up with them about where the holes are in your story, and ask them to help you fill in the gaps. This helps the review team to understand on a deeper level what the core issues are, what your strategy is, and it motivates them to go above and beyond to help you build your case. People engage and do much better work when they believe that their work matters, and when they understand how their contribution fits into the overall team effort. An engaged review team that understands the story of the case will often spot important categories of documents that were not yet on counsel’s radar. This is the way we like to work, and I believe it differentiates us from everybody else.
Leveraging this human element permeates our culture. From the beginning, we’ve focused on document review as a crucial component of the profession, and we strive to motivate people and get them engaged in the case, resulting in efficiency and work product of the highest quality.
Published February 20, 2014.
