Mehrnoosh A.

Key Skills

Software

No software listed

Top Subject Matter

LLM/agent evaluation and red-teaming for DFIR forensic decisioning

Prompt-injection detection

Evaluation Domain Expertise

Top Data Types

Text

Document

Top Task Types

Red Teaming

Freelancer Overview

Eval-Driven Optimization Loop for a DFIR Agent (find-evil-hackathon). Brings 10+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Internal, Proprietary Tooling, and DVC. Education includes Bachelor of Science, SANS Technology Institute (2023) and Associate of Science, De Anza College (2023). AI-training focus includes data types such as Text and labeling workflows including Evaluation, Rating, and Red Teaming.

Labeling Experience

Na0S (AI Prompt Injection Detector)

TextRed Teaming

Developed an open-source prompt-injection detector that screens LLM inputs and evaluates detection performance against structured test sets. Fused rule-based and ML signals across a taxonomy and measured effectiveness using held-out canary data. Red-teamed the detector by creating an evasion harness to perturb attacks and re-score them on a committed reproducible benchmark. • Achieved 0.925 F1 with zero false positives on a held-out 230-sample canary set (115 hard negatives). • Combined 120 rules with a three-signal ML ensemble (TF-IDF/logistic regression, all-MiniLM-L6-v2, Meta Prompt-Guard-2). • Quantified bypass rates under lightweight obfuscations (leetspeak, spacing, typos) using the evasion harness. • Versioned the data pipeline with DVC and used a fact-extractor backed by a 9,300-test suite to regenerate metrics from source.

2025 - Present

Eval-Driven Optimization Loop for a DFIR Agent (find-evil-hackathon)

Text

Built and improved an eval-driven optimization loop for an LLM forensics/DFIR agent to measure performance and prevent fabricated outputs. Designed scoring and verification constraints so the optimization could be evaluated deterministically, including detecting regressions and removing reward-hacking behaviors. Implemented an offline deterministic test harness and identified issues in the benchmark ground truth to keep evaluation comparisons honest. • Increased agent verdict accuracy from 0-for-3 to 3-for-3 with zero fabricated findings via trust-path constraints and auto-revert on regression. • Designed an un-gameable metric using evidence presence, fabrication counters, and tool-level provenance, shipped as a 52-test deterministic scorer. • Traced benchmark scoring anomalies to a contaminated control arm and a mislabeled gold answer key. • Focused on adversarial testing and evaluation of agent defenses for robustness.

2025 - 2026

Education

Work History