Machine Learning Engineer, Ai Evaluation Lab

Stanford University is seeking a Machine Learning Engineer to perform advanced technical research for the AI Evaluation Lab. The AI Evaluation Lab evaluates the performance, safety, effectiveness, reliability, and transparency of AI models intended for real-world deployment. It aims to understand performance degradation in imaging AI applications and develop strategies to recognize, measure, and mitigate these issues.
Reporting to the Executive Director of the 3D and Quantitative Imaging (3DQ) Lab and with guidance and dotted line reporting to senior faculty leaders, the individual will build end-to-end data pipelines and infrastructure for ML models used in Stanford Radiology. They will build robust and modular software engineering infrastructures for trained and inference of ML models that can be used for a variety of downstream applications and will use their knowledge to make recommendations and design decisions for languages, tools, and platforms used in software and data projects.
Duties Include

• Support complex scientific and research programs related to area of specialization; analyze data, monitor and oversee experimental process, and design and develop prototypes, specialized equipment, and/or systems.
• Carry out all activities, including troubleshooting and resolving routine problems for scientist or engineers, independently.
• - Other duties may also be assigned
• Establish, communicate, and enforce compliance with health and safety policies and procedures.
• Collaborate with scientists, engineers, or senior administrative officers to oversee complex non-routine analyses, select optimum solutions, and perform corrective modifications to equipment and system designs.
• Participate in the planning, design, and implementation of scientific or engineering initiatives, and work toward project objective.
• Collaborate with senior engineers and scientists to design and develop special purpose equipment and/or systems.
• Oversee a laboratory space or unit, and supervise the work of technicians and other staff associated with the group.
• Perform supervisory duties, including overseeing the work of technicians and other staff associated with the group/project, supervising the regular installation, maintenance, and operation of complex scientific or engineering projects, and training technicians, operators, and others working in particular scientific or engineering function area.
• Develop training manuals and safety guidelines, and train new instrumentation users, researchers, and/or technical staff.
• Serve as a resource in review of research proposals and research capabilities, and make recommendations.

Desired Qualifications

• Work in shared code environments such as GitHub and collaborate with other developers and be responsive to GitHub issues and pull requests.
• Experience with cloud deployment strategies and CI/CD.
• Lead code reviews for projects/systems as an independent reviewer applying design principles, coding standards and best practices
• Experience working in a HIPAA regulated environment.
• Research and prototype state-of-the-art computer vision and deep learning methods to develop solutions that meets product requirements.
• Ability to install, configure, and implement machine learning algorithms in modern training platforms (such as PyTorch, JAX) and inference platforms (such as Hugging Face, gradio, streamlit).
• Experience overseeing, developing or implementing machine learning operations (MLOps) processes. Experience working with healthcare data.
• Experience with publications in deep learning for medical imaging in healthcare journals or ML conferences a plus.
• Design algorithm evaluation frameworks, schedule, and report algorithm performance on a regular basis.
• Experience in building software and data infrastructure for analytics team, including ability to write Python and SQL for processing large datasets.

Education & Experience (required)
Bachelor’s degree in engineering, science, or related field and three years of relevant experience; or a combination of education and relevant experience.
Knowledge, Skills And Abilities (required)

• Ability to independently oversee and manage instrumentation or system installation.
• In-depth experience with software applications, systems, or programs relevant for the job.
• Demonstrated ability to effectively supervise and train a diverse work staff.
• Demonstrated ability to establish, communicate, and enforce compliance with health and safety policies and procedures.
• Ability to collaborate with senior engineering and scientific staff to design and develop special purpose equipment and/or systems.
• Demonstrated experience applying complex scientific and engineering principles and performing special technical services involving both development and performance.
• Experience overseeing a laboratory space or unit and supervising the work of technicians and other staff associated with the group.
• Demonstrated knowledge and skills of advanced scientific or engineering principles and practices.
• Demonstrated ability to review research proposals, evaluate research capabilities, and make recommendations.
• Experience overseeing the plan, design, and implementation of major scientific or engineering initiatives and ensuring project objective are met.

PHYSICAL REQUIREMENTS*:

• Rarely kneel/crawl, climb (ladders, scaffolds, or other), reach/work above shoulders, use a telephone, writing by hand, sort/file paperwork or parts, operate foot and/or hand controls, lift/carry/push/pull objects that weigh >40 pounds.
• Occasionally stand/walk, sit, twist/bend/stoop/squat, grasp forcefully.
• Frequently grasp lightly/fine manipulation, perform desk-based computer tasks, lift/carry/push/pull objects that weigh up to 10 pounds.
• - Consistent with its obligations under the law, the University will provide reasonable accommodation to any employee with a disability who requires accommodation to perform the essential functions of his or her job.

Working Conditions

• May be exposed to high voltage electricity, radiation or electromagnetic fields, lasers, noise > 80dB TWA, Allergens/Biohazards/Chemicals /Asbestos, confined spaces, working at heights ?10 feet, temperature extremes, heavy metals, unusual work hours or routine overtime and/or inclement weather.
• May require travel.

Work Standards

• Promote Culture of Safety: Demonstrates commitment to personal responsibility and value for safety; communicates safety concerns; uses and promotes safe behaviors based on training and lessons learned.
• Subject to and expected to comply with all applicable University policies and procedures, including but not limited to the personnel policies and other policies found in the University's Administrative Guide, http://adminguide.stanford.edu.
• Interpersonal Skills: Demonstrates the ability to work well with Stanford colleagues and clients and with external organizations.

The expected pay range for this position is $109,000 to $129,000 per annum.
Stanford University provides pay ranges representing its good faith estimate of what the university reasonably expects to pay for a position. The pay offered to a selected candidate will be determined based on factors such as (but not limited to) the scope and responsibilities of the position, the qualifications of the selected candidate, departmental budget availability, internal equity, geographic location and external market pay for comparable jobs.