In this role, you'll be instrumental in developing automated performance evaluation systems for our machine learning models that drive millions in transportation cost savings. You'll work at the exciting intersection of data engineering, software development, business intelligence, and machine learning operations. This position offers a unique opportunity to contribute to forecasting infrastructure, collaborate with cross-functional teams, and make a significant impact on Amazon's global transportation network efficiency.

Key job responsibilities
* Design and implement automated model performance evaluation pipelines
* Develop standardized metrics and KPIs for measuring model accuracy
* Create interactive dashboards and visualizations for model performance monitoring
* Collaborate with scientists and software developers to define and implement model quality gates
* Build scalable data processing solutions for large-scale model evaluation
* Partner with data engineers to ensure data quality and reliability
* Contribute to the development of our performance evaluation platforms
* Participate in code reviews and software development lifecycle processes

A day in the life
As a Sr. BIE, your morning might involve collaborating with scientists to investigate forecast model performance deviations, while meeting with business stakeholders to understand their metrics requirements. The afternoon could be spent building QuickSight visualizations for evaluation dashboards and optimizing SQL queries for newly defined business metrics. Throughout the day, you'll partner with data engineers on implementing quality checks, write Python scripts to automate metric processing, and participate in design discussions about new evaluation features. The role offers a dynamic mix of technical work and cross-functional collaboration with science, engineering, and business teams.

About the team
Our team is a group of Scientists, Business Intelligence Engineers, Data Engineers, and Software Developers who build and maintain critical infrastructure for Amazon Transportation's machine learning operations. We work at the intersection of scientific innovation, data engineering, ML systems, and business intelligence, developing solutions that enable rapid model development and reliable production deployments. Our team combines scientific expertise with engineering excellence to create automated performance evaluation systems, maintain data quality frameworks, and build scalable monitoring solutions. We're passionate about using technology to solve complex problems and pride ourselves on fostering a collaborative environment where innovation and work-life harmony thrive.

Basic qualifications

• Experience in analyzing and interpreting data with Redshift, Oracle, NoSQL etc.
• Experience with data visualization using Tableau, Quicksight, or similar tools
• Experience with data modeling, warehousing and building ETL pipelines
• Experience in Statistical Analysis packages such as R, SAS and Matlab
• Experience using SQL to pull data from a database or data warehouse and scripting experience (Python) to process data for modeling
• Experience with SQL
• Experience in the data/BI space

Preferred qualifications

• Experience with AWS solutions such as EC2, DynamoDB, S3, and Redshift
• Experience in data mining, ETL, etc. and using databases in a business environment with large-scale, complex datasets

Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice (https://www.amazon.jobs/en/privacy_page) to know more about how we collect, use and transfer the personal data of our candidates.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.