A small collection of side projects, built around things Iām curious about professionally. Right now that's mostly one question: how far can you actually trust an AI to do real work, not just answer a question, but reason through a policy, admit what it doesn't know, or act on incomplete information.
Can an AI answer questions about your health plan without making anything up? A real RAG pipeline, chunked, embedded, and retrieved from six actual health plan documents, answers 88 real questions and gets graded on two separate things: did it find the right document, and did it get the answer right. It never once guessed when the answer wasn't actually in the source.
Full write-up on GitHub.
Can a model keep three separate written policies straight at once, and get the right answer for the right reason? Three fictional companies, three full access policies, all sitting in the same prompt, graded on whether it picked the right policy, cited the exact right rule, and reached the correct decision. It never once cited a rule from the wrong company's policy, even on requests built specifically to bait that mistake.
Full write-up on GitHub.
Can an AI be trusted to act on incomplete information, not just answer a question but decide what to do next? This one runs a full agentic loop: an LLM handles IT access requests with five real tools, look someone up, check policy, ask a question, escalate, or decide, tested against Claude Sonnet 5 and GPT-4.1 side by side. It got the final outcome right 100% of the time, including knowing exactly when to ask instead of guess.
Full write-up on GitHub.