Technically Legal - A Legal Technology and Innovation Podcast

In advance of this year's RelFest in Chicago, Aron Ahmadia, Vice President of Applied Science at Relativity, joins the podcast to discuss AI evaluation, AI benchmarking and the transition from traditional Technology Assisted Review (TAR) to generative AI in legal document review and eDiscovery. The conversation covers the daily operations of a global AI team, the concept of "evaluation as a product," and the methodology behind testing large language models (LLMs) for legal tasks. Aron also discusses a recent study that compared the accuracy and efficiency of TAR and human document review against Relativity's generative AI tool, aiR for Review.
Aron talks about:
A day in the life of the VP of Applied Science at Relativity.

Navigating the rapid pace of AI and predicting what remains constant in legal tech.

Aron's background: From early computer networking in a law firm to data science at DARPA and Capital One.

Defining "evaluation as a product" and the importance of maintaining private benchmarks for LLMs.

The trade-offs between human evaluations and automated AI evaluations.

TAR (Predictive Coding) versus Generative AI: How prompt-based reasoning differs from label-based classification.

Episode Credits
Editing and Production: Grant Blackstock
Theme Music: Home Base (Instrumental Version) by TA2MI

Emily Logan Stedman, commercial litigator at Husch Blackwell, discusses moving beyond the billable hour not just to modernize legal billing but to also prioritize lawyer well-being. The conversation explores the practical steps required to transition traditional litigation models toward Alternative Fee Arrangements (AFAs). Emily details her methodical approach to utilizing secure legal AI tools to analyze historical time entries, allowing her to offer phased flat fees, strategic counsel subscriptions, and predictable contract review pricing. The discussion also covers strategies for maintaining mental health in demanding legal environments, taking control of chaotic schedules, and reframing the billable hour as a tool for career management rather than a burden.
Things We Talk About in this Episode
Emily's transition from bankruptcy law to general commercial litigation after moving to Wisconsin.

The motivation behind launching The Grace Period podcast and advocating for lawyer well-being in big law.

Strategies for managing the billable hour, shifting daily mindsets, and setting strict boundaries for deep work.

The genesis of applying Alternative Fee Arrangements (AFAs) to unpredictable, complex litigation matters.

Structuring strategic counsel subscriptions and granular phased flat fees to provide cost predictability for corporate clients.

Utilizing AI to analyze historical billing data and build accurate, profitable flat-fee pricing models that secure firm buy-in.

Leveraging secure legal AI agents like Legora and Prompt Composer to map out discrete litigation phases and draft modern engagement letters.

In this episode, Justin Wales, Chief Legal Officer for Crypto.com makes a second visit to the podcast. This time around, he discusses his new book, The AI Legal Handbook, the emerging regulatory landscape, liability frameworks, and practical applications of artificial intelligence. Justin outlines how attorneys and regulators should approach modern AI models, the mechanics behind AI, and the effect of generative AI on legal education. The discussion covers key regulatory principles, the balance between strict liability and negligence frameworks, copyright issues in machine learning training, and the governance challenges of artificial general intelligence (AGI).
Things We Talk About in this Episode
Transitioning from cryptocurrency regulation to developing a unified framework for artificial intelligence law

The importance of legal professionals understanding underlying model mechanics and token prediction to evaluate hallucination risks

Generative AI's disruption of law school assessments and the pedagogical shift from written assignments to oral examinations

The convergence of cryptocurrency rails, agentic AI systems, and decentralized compute infrastructure

Foundational principles for AI regulation, emphasizing proportional oversight, human-centered accountability, and transparency over outright bans

Managing the "black box problem" and balancing algorithm explainability with technical efficiency

Comparative analysis of sector-specific guidelines versus federal mandates and the adaptability of existing rules of professional conduct

Evaluating legal liability standards: applying negligence versus strict liability to autonomous vehicles, healthcare diagnostic tools, and AI chatbots

Intellectual property considerations, fair use, and style mimicry in generative model training data

Preparing legal, economic, and governance frameworks for artificial general intelligence (AGI)

Where to buy The AI Legal Handbook: JustinWales.com or Amazon.
Episode Credits
Editing and Production: Grant Blackstock
Theme Music: Home Base (Instrumental Version) by TA2MI

In this episode April Miller Boise, with stints as Chief Legal Officer at Intel and managing partner at Thompson Hine, shares insights from her career. The conversation explores the shifting dynamics between corporate legal departments and outside counsel and the necessity of embracing legal technology. April also provides advice for junior lawyers on developing long-term expertise and trusted relationships in an AI-driven environment.
Things we Talk About in this Episode
Career progression: From Wall Street M&A partner and law firm managing partner to Chief Legal Officer.

The fundamental differences between driving P&L at a law firm versus serving on an executive leadership team in-house.

Addressing the widening gap between corporate expectations and traditional law firm service delivery models.

Experience navigating global manufacturing, innovation, and supply chain transformations at Eaton and Intel.

Practical approaches to generative and agentic AI adoption in legal departments and outside counsel.

Key attributes for modern legal talent: Intellectual curiosity, agility, and comfort with calculated risks.

Career advice for young lawyers: Balancing AI efficiency tools with the 10,000-hour principle of skill-building.

Why historical legal tech struggled to scale and how current enterprise AI investments are fundamentally altering the legal landscape.

In this episode, theoretical neuroscientist Vivienne Ming discusses her new book, Robot Proof - When Machines Have All the Answers, Build Better People.  The conversation examines the intersection of artificial intelligence and knowledge work, exploring how professionals can adapt to an increasingly automated economy.
The discussion contrasts the limitations of basic AI task automation with the advantages of human-AI collaboration—the "cyborg" model—for solving complex, ill-posed problems. Ming highlights her research on forecasting market outcomes, the critical role of endogenous motivation, and how the legal industry and other elite professions must rethink entry-level training.
Key Topics Discussed
Theoretical Neuroscience and Early AI: Ming's background and the evolution of machine learning models from early academic research to modern agentic AI.

The Polymarket Experiment: An analysis comparing the forecasting accuracy of standalone AI, unassisted humans, and human-AI collaborators, revealing the superiority of deep human-machine integration.

Automation vs. Augmentation: The pitfalls of the traditional "human-in-the-loop" model and why replacing menial tasks often neglects essential human problem-solving skills.

Well-Posed vs. Ill-Posed Problems: Identifying the specific areas where AI excels (algorithmic, factual answers) and where human intelligence remains superior (navigating uncertainty and undefined parameters).

Labor Disruption and Economic Shifts: Examining historical technological revolutions, the Jevons paradox, and the future demand for specific, highly adaptable human skill sets.

Endogenous Motivation: How internal drivers like curiosity, resilience, and perspective-taking predict professional success more accurately than standard extrinsic incentives.

Practical AI Strategies: Actionable methods for professionals to refine their skills, including using AI as a critical "nemesis" to challenge assumptions and encourage deep, effortful processing.

The Future of Elite Professions: The macro-level challenges facing organizations in developing junior talent—such as associate attorneys—when the entry-level tasks traditionally used for training are automated.

Things We Talk About in this Episode
Socos: Vivienne Ming's philanthropy and research newsletter (socos.org).

Thinking, Fast and Slow: Authored by Daniel Kahneman.

Anthropic Research: A study published in Science detailing the productivity of AI-assisted programmers.

BCG AI Study: Research analyzing AI integration and performance among management consultants.

Raj Chetty: Economic research regarding peer role modeling, education, and socioeconomic mobility.