Which AI Should You Actually Use? I Built an AI for That.
TL;DR
Most people at a company use a fraction of what tools like Claude, ChatGPT, or our internal MemodoAI can do. Access and tutorials don't fix that. I built Sherpa: an open-source agent that interviews one person about their job and tells them which AI features fit their actual work, with a how-to and a conservative time-saved estimate, then asks them to commit to trying one. It runs as a LangGraph app behind an OpenAI-compatible endpoint, on open weights, with an eval harness.
The problem
I built MemodoAI, a fork of LibreChat, to give the whole company an AI chatbot and agents without per-seat subscriptions. Then I had Claude produce an 18-minute onboarding tutorial. Both helped. Neither fixed the underlying problem, which I'd seen well before MemodoAI: most people use AI tools for a narrow slice of what they can do. An email rewrite, a summary, and that's it.
Access isn't the bottleneck. Instructions aren't either. The missing piece is the link between what a tool can do and the specific work a given person does every day. A tutorial covers the features. It doesn't tell a controller which of those features applies to their month-end close.
What Sherpa does
Sherpa interviews one person about their work: recurring tasks, the systems they use, where they lose time, what they re-key by hand. It returns three to five recommendations. Each names a specific AI capability (in MemodoAI, Claude, or any approved tool), explains how to apply it to that person's workflow, and gives a conservative time-saved estimate with the assumption stated. It ends by asking the person to commit to trying one.
The interview runs without the tool catalog loaded. This is deliberate. If the agent knew the available tools while asking questions, it would steer toward needs it can already serve. Running blind, it maps the work first, then matches against the catalog in a separate phase. That also lets it report gaps: real needs with no good tool behind them, which tell you what to build or buy next.
A personal advisor, at scale
Sherpa offers the kind of guidance that normally takes a consultant, someone who sits with each employee and maps their work to what's available. That doesn't scale, so most companies never do it. Sherpa is software, so the same tailored conversation can run for everyone at once, as many times as they need, including each time a new tool or capability shows up.
Top-down mandates and all-hands trainings don't do it. People adopt AI when someone shows them how it fits their own work, and Sherpa can do that for everyone at once.
How it's built
Sherpa is a LangGraph application with three nodes (diagnose, match, commit) and a typed state object carrying a structured needs profile between them. It's served behind a thin FastAPI OpenAI-compatible endpoint, so it can plug into MemodoAI later without rewiring. Inference runs on open weights via Together AI. LangSmith handles tracing and evals. The recommendations are assembled in code from a structured JSON object rather than written freehand by the model, which keeps the output stable and the quality inspectable.
Status
It's a proof of concept. The scope is narrow on purpose: one finance question bank, a single user, a local commitment log instead of an org-wide measurement layer. The point of the POC is to test whether the recommendations come out specific and genuinely useful before building anything around them. Known limitations and the deferred roadmap are in the repo.
What it addresses
Sherpa covers the parts of AI adoption that organizations usually handle as separate problems:
- Discovery: where AI can help in a specific role.
- Adoption: getting people to use tools that already exist.
- Measurement: which suggestions get adopted across the company, so you can see where AI is actually paying off. This is not productivity surveillance. Sherpa counts suggestions, not people. It does not track who is or isn't using AI, and it does not score anyone's individual output. There is no per-person dashboard. Measuring adoption instead of individuals is also what keeps it within works-council and GDPR limits.
- Self-improvement: every conversation produces signal. What people committed to, what saved time, where estimates ran off, which needs had no tool behind them. The eval harness captures it; fed back in, it tunes the questionnaire, the matching, and the estimates. I do that tuning by hand now. With enough real usage, a supervising agent can run it, re-scan for new tool capabilities, and fold both back in.
Try it
It's open source. The clearest picture of the real state is the SDD/ directory: the specifications and critical reviews behind the build, the same spec-driven process I've written about before.