Evaluating how wildlife conservation laws are implemented is critical to determining how best to protect biodiversity. Two agencies, the U.S. Fish and Wildlife Service and National Marine Fisheries Service (FWS and NMFS; Services collectively), are responsible for implementing the U.S. Endangered Species Act (ESA). This creates a ‘natural experiment’ for understanding how implementation and interpretation of the same law varies between agencies with different histories, cultures, priorities and funding levels. We take advantage of this natural experiment to quantify differences in how FWS and NMFS implement a core component of the ESA, section 7 consultations. The ESA requires federal agencies to consult with the Services if an action an agency proposes might affect ESA-listed species or their habitats. We quantified the quality of consultations by comparing > 120 consultations to the requirements laid out in the Services’ consultation handbook. These analyses were complemented with in-person interviews of biologists from the Services to help understand how some observed variation arises. Among these consultations, we found those from NMFS had significantly higher quality scores than those from FWS. A common shortcoming from both agencies, but especially severe for FWS, was the lack of accounting for effects that were previously authorized through consultations. The biologist interviews indicated some discrepancy between how they perceive consultations and the outcomes from our quantitative analysis. Building from these results, we recommend several actions that can improve quality of consultations, such as using a single database to track and integrate previously authorized harm in new analyses, and the careful but more widespread use of programmatic consultations.