AI/ML Engineer & LLM Evaluation Specialist, Bay Area

Nine years in FINRA/SEC-regulated financial services before pivoting into machine learning. The finance work built systems thinking under regulatory pressure; the AI work is where that thinking goes now that the constraints are open-ended rather than compliance-mandated.

Oracle Tier at Scale AI / Outlier (Aug 2024 - Apr 2025): reviewed and corrected peer annotations as Quality Analyst across twelve RLHF training projects, covering process-supervised code reasoning, multi-turn instruction following, multimodal visual grounding, tool-use accuracy, and adversarial prompt design targeting hallucination and reward hacking. That training data fed pipelines for OpenAI, Anthropic, Google DeepMind, Meta, and Microsoft. Which specific model each project shaped isn't something Scale AI discloses, so I won't claim more precision there than I actually have.

Data science background: BrainStation Data Science Diploma (2023-2024), Google Data Analytics Professional Certificate, and Microsoft Azure AI Engineer / Azure Data Scientist Associate certifications. First place in a Google-sponsored capstone hackathon for a pedestrian-safety scoring model built on 1.9M+ Chicago crime records.

Everything else on this profile is coursework, a work-in-progress, or something I haven't circled back to yet. These three repos are the ones that actually run, in order of how finished they are.

Audits any public website for LLM discoverability and agent readiness: whether AI crawlers can get in, whether the page exposes llms.txt, JSON-LD, and MCP/A2A discovery surfaces, and how much of the site a model or an autonomous agent could actually read versus just a human with a browser. FastAPI backend with a 301-test suite, a Next.js/TypeScript dashboard, an OpenAPI 3.1 schema, and a portable Agent Skill package. Runs locally, no API keys, no telemetry.

Live at machineread.ai, where the Essentials audit runs free with no account required. This repo is the public source tree for that free tier.

Python FastAPI Next.js TypeScript Pydantic MCP

Real-time Equity and Economic Digest. A self-hosted market news agent that writes scheduled stock market briefs from public RSS feeds, using whatever LLM you already pay for: Anthropic, OpenAI, OpenRouter, Ollama, or any OpenAI-compatible endpoint. Four briefs on a trading day plus a weekend recap on Monday, read through a terminal-style React dashboard. Live demo at georgejieh.dev/reed.

The design point is that the model does not do the research. Feeds are fetched, deduplicated, and time-filtered first, and then a single LLM call with zero tools exposed turns those headlines into a structured digest with sentiment, tickers, and numbered sources. Any story whose URL was not in the pre-fetched set gets dropped, which is what stops the model from inventing a source. The first version used a crawler and a scraper; it was slow and frequently blocked, so I moved the research step off the model entirely and the briefs got more reliable as a result.

Python FastAPI Pydantic React TypeScript APScheduler

An end-to-end ML pipeline predicting which grape varietals suit a region's climate, built on 143,000+ wine records and 60+ years of weather data (1961-2016). Scraped Wine Enthusiast ratings myself with a custom Selenium tool to supplement the Kaggle dataset, engineered features around the actual vine growth cycle (budburst, flowering, veraison, harvest), and landed on 56% accuracy with H2O AutoML across a 19-class imbalanced problem: a real result, not a great one. The repo is messier than the other two and the model's accuracy has room to grow, but it's the one I'd point to for seeing a full pipeline, scraping through EDA through a deployed Flask app, in one place.

Python H2O AutoML scikit-learn Flask Selenium SMOTE

More context on all of these, plus the projects that are further from done, is at georgejieh.dev.

• AI/ML: LLM evaluation and RLHF, rubric-based grading, chain-of-thought and process supervision, adversarial prompt design, multi-agent orchestration, production inference pipelines
• Core stack: Python, SQL, scikit-learn, PyTorch, TensorFlow, H2O AutoML, Flask, FastAPI, Next.js/TypeScript
• Finance: Series 7, Series 66, SIE licensed. Nine years at Merrill Lynch and Bank of America in wealth management and branch operations, also where the finance-domain grounding for AI evaluation work comes from
• Education: UC Berkeley (BA, American Studies), BrainStation (Data Science Diploma), Google Data Analytics Professional Certificate

Open to AI/ML developer and research roles. Also available for freelance AI training, evaluation, and annotation work on weekends.

Portfolio · LinkedIn · Hugging Face · Email