AI Application Engineer (Part-Time)

Client: medxprts.ai Location: Remote Type: Part-time, with potential to transition to Full-time Schedule: U.S. timezone overlap required DescriptionMedxprts.ai is building an AI-powered platform for the legal and healthcare space, using LLMs, agentic workflows, and automation to create production-grade applications. We are seeking an AI Engineer - LLM Fine-Tuning: a hands-on engineer who has personally trained and fine-tuned open-weight models, built training infrastructure, engineered datasets from messy documents, and established rigorous evaluation and preference/feedback training pipelines. This role works closely with engineering teams to ship models and integrated features into production.

Responsibilities• Design, implement, and run LLM fine-tuning experiments (LoRA/QLoRA and full SFT) on open-weight models (e.g., Llama, Mistral, Qwen) and ship trained models into product workflows. • Build and maintain training infrastructure using PyTorch and Hugging Face tooling (Transformers, PEFT, TRL/Axolotl), including multi-GPU training orchestration (DeepSpeed/FSDP) on AWS or GCP. • Engineer datasets from real-world unstructured sources (long PDFs, medical/legal records), performing deduplication, filtering, contamination checks, and train/eval splits. • Create evaluation harnesses tailored to domain needs: held-out test sets, LLM-as-judge with human calibration, regression tests across model versions, and automated monitoring for model drift. • Implement preference/feedback training workflows (DPO/RLHF-style or similar) to learn from expert corrections (doctor-in-the-loop), and integrate feedback loops into model retraining pipelines. • Collaborate with backend/frontend engineers to integrate fine-tuned models into services, optimize inference latency/cost, and support production deployments. • Participate in PR reviews, release processes, incident debugging, and continuous improvement of training and deployment tooling. • Ensure secure, compliant handling of sensitive data (HIPAA-awareness is highly preferred) during dataset preparation and model training.