Applied Scientist (Multimodal / RAG / VLM)

Role Summary

You will drive Facthory’s applied AI research agenda across multimodal understanding, RAG/KAG systems, vector retrieval, video reasoning, spatial intelligence, and VLM-based knowledge extraction. This is a hands-on leadership role focused on turning real industrial data (video, audio, manuals, processes) into deployable AI capabilities inside the Facthory product. You will work across science, engineering, and product to define Facthory’s AI differentiation.

Responsibilities

• Build next-generation multimodal pipelines (video → structured knowledge → retrieval).

• Advance Facthory’s RAG/KAG architecture (chunking strategies, hybrid retrieval, evaluation).

• Prototype and productionize multimodal models for video understanding, OCR/Document AI, audio processing, and frame reasoning.

• Own research roadmap for factory floor AI: ambiguity resolution, domain grounding, evidence-linked answers, safety in LLMs.

• Lead experiments with latest VLMs (OpenAI o5/6, Gemini Flash, LLaVA-Next, Qwen-VL, InternVL, SpatialLM).

• Define and maintain evaluation frameworks for multimodal accuracy, latency, and robustness.

• Collaborate with backend and infra to deploy scalable inference and model routing.

• Mentor scientists and engineers; set scientific standards for the organization.

Required Skills

• Deep expertise in multimodal ML (video, vision, OCR, audio) and retrieval systems.

• Strong applied experience in RAG architectures, vector databases, embeddings, evaluation.

• Hands-on skills in Python (PyTorch, JAX), model optimization, quantization.

• Experience building production AI systems (latency, caching, scaling).

• Knowledge of industrial/manufacturing data and constraints.

Preferred Skills

• Spatial reasoning models, video QA, VLM fine-tuning.

• Experience designing scientific roadmaps.

• Prior work in high-stakes or safety-critical AI.