Yes, I have hands-on experience in legal prompt engineering, and evaluating AI outputs within the judiciary
Text
RLHFYes, I have hands-on experience in legal prompt engineering, and evaluating AI outputs within the judiciary sector. I custom-built and personally trained a private, domain-specific AI assistant (a 'Personal Gem') to augment my daily workflow as a Civil Law Judge in a Basic Court in the Republic of Serbia. Through this continuous personal implementation, I performed high-level data labeling and output optimization, which included: Prompt Engineering & Behavioral Training: Designing precise system instructions and frameworks to ensure the AI correctly synthesized current Serbian civil legislation (such as the Law on Civil Procedure - ZPP and the Law on Contracts and Torts - ZOO), while strictly filtering out deprecated laws. AI Output Review & Quality Assurance: Extensively reviewing and fine-tuning AI-generated legal reasoning , draft judgments, and procedural orders to guarantee exact legal alignment, factual accuracy, and the elimination of AI hallucinations. Advanced Legal Context Annotation: Manually structuring training prompts to teach the AI how to analyze the interplay between domestic civil law and international standards, specifically verifying that its outputs adhered to binding precedents from the European Court of Human Rights (ECHR). Jurisprudence Evaluation: Training the model to cross-reference legal issues with up-to-date (2022–present) case law from the Supreme Court, Constitutional Court, and Appellate Courts, ensuring it evaluated how one legal institute affects another within a judgment. By developing and utilizing this personal assistant, I gained extensive experience in reinforcement learning through human feedback (RLHF) from a user-developer perspective, ensuring strict adherence to complex, specialized parameters.