I help teams trust what their AI tells them.
What brings you here?
Most teams can't say how often their model, LLM or otherwise, agrees with their best people. I set up the measures, a reviewed sample set, and the checks that compare model scores with expert reads, so you can see where it holds and where it drifts.
Built AI that scores customer calls on two dozen quality measures and ties every score to the evidence behind it.
Talk about evaluationRead Issue 3, Before you bet on the scoreBefore you back a company's AI, it helps to know how it's measured. I review their evaluation setup, the golden sets, the agreement checks and the thresholds, and tell you whether the numbers mean what the deck says. I take on advisory and fractional work with founders and venture investors.
Spent two years bringing legal, security and AI teams together to ship an AI platform for federal operations.
Talk about diligenceRead Issue 1, Keep the human signalMore dashboards rarely settle a decision. I start from the question leadership has to answer, build a working prototype against real data, and let leaders test their assumptions before they commit money.
Built prototypes that helped executives weigh building in-house against buying, and check whether a simpler tool would do.
Talk about a decisionRead Issue 2, Clean data starts with peopleI come in as a player-coach, a senior expert who still builds. I've hired and grown a data science organization, and I write the code alongside the team.
Built a data science organization and oversaw more than 100 projects.
Talk about a role
Field notes
- built a data science organization
- shipped AI for federal operations
- built LLM-as-judge evaluation that grades calls against expert reviewers
- M.S. Data Analytics, MBA
A note
Hi, I'm James Fouts. I came up through network engineering, spent years in process improvement, then built a data science organization. These days most of my time goes into AI that scores conversations and checks itself against experienced reviewers. I'm a Distinguished Engineer in data science at Verizon.
The part I care about is the gap between what a model says and what's true. I work on it with teams, founders and investors. That means AI evaluation and calibration, plus the tooling and analytics around them. I'm open to senior roles and to advisory or fractional work.
If that gap is costing you, send a note about the decision in front of you. I'll write back with the full story and an honest read on whether I can help.
easiest way to reach me
contact@jamesfouts.workLinkedIn
Resume on request. And if you're nearby, let's grab a coffee.
Want the practical version first? Read the field guide.
Also on the bench
- InstormingWhere brainstorms keep moving up the chain toward production, with people steering and AI carrying.the product
- Tuloy LabMy studio, where new ideas get built far enough to judge.the studio
- jamesfouts.siteThe personal side, with postcards from the road. I've probably been through your neck of the woods.off the clock
- SQUIRTApplied MLA feeder camera that tells squirrels from birds, trained on labels I made myself. The hose waits in dry mode for now.for fun