Associate Director, Evaluation and Quality Standards

At Johnson & Johnson,we believe health is everything.

Our strength in healthcare innovation empowers us to build aworld where complex diseases are prevented, treated, and cured,where treatments are smarter and less invasive, andsolutions are personal.Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity.Learn more at jnj.com .

As guided by Our Credo, Johnson & Johnson is responsible to our employees who work with us throughout the world.

We provide an inclusive work environment where each person is considered as an individual.

At Johnson & Johnson, we respect the diversity and dignity of our employees and recognize their merit.

Job Function:
Data Analytics & Computational Sciences

Job Sub Function:
Data Science

Job Category:
People Leader

All Job Posting Locations:
Cornellà de Llobregat, Barcelona, Spain, Madrid, Spain

Job Description:

Evaluation & Quality Standards to join our Data Science and Digital Health team (DSDH).

This is a newly created leadership role within the Generative AI organization, reporting directly to the Head of Generative AI.

The GenAI team is deploying AI across discovery, development, regulatory, and operations to create AI assistants and agentic systems across the therapeutic areas and functions.

As we scale from proof-of-concept to enterprise deployment, we need a dedicated evaluation function that defines what quality means for generative AI in a regulated pharmaceutical R&D environment, communicates the metrics effectively, and enforces that standard across every platform, vendor, and capability we build or adopt.

This role owns evaluation and governance for all generative AI work across R&D.
Key Responsibilities
Evaluation Framework Design

• Design and maintain evaluation frameworks for GenAI solutions spanning LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.

• Develop therapeutic-area-specific evaluation criteria with business teams, reflecting domain-specific quality requirements per therapeutic area and application type.

• Define benchmarks for scientific validity, regulatory compliance, data quality, and operational reliability.

• Establish gold-standard validation datasets and automated evaluation pipelines.

Vendor and Solution Assessment

• Lead structured vendor assessments producing written Evaluation Reports

• Challenge vendor claims with independent benchmarking; distinguish genuine capability from demonstration performance.

• Assess integration complexity, total cost of ownership, regulatory posture, and vendor lock-in risk.

Governance and Standards

• Serve as technical lead for the Evaluation & Standards Board, a cross-functional governance body for GenAI quality.

• Present evaluation findings and recommendations to the GenAI Portfolio Steering Committee.
...