As part of the Data Strategy and Insights Org (DSI), this role will be an enabler of our journey to be the world’s leading data-driven retailer. As part of this transformation, we are seeking an individual who will be responsible to establish robust data pipelines and services – including both in house developed data enabling services and systems integrations across the DSI team to ensure we our technical deliverables meet and exceed the quality expectations.
We are looking for a highly motivated, resourceful, team-oriented individual to drive the data engineering process. You are exceptionally talented Data Engineer with an outstanding track record of working with very large data sets and building robust ETL pipelines for data acquisition for internal systems and external data sources. You will be modernizing and improving the data acquisition infrastructure from the ground up. You will be working with structured/unstructured Data sets, building large scale Data processing platforms, implementing world class data governance and operational controls, solving complex performance challenges.
The Data Engineer role will report up to the Lead Data Engineer/Senior Manager Data Engineering.
Play a pivotal design and hands on implementation role in improving the Data infrastructure in a project-oriented work environment.
Influence cross functional architecture in sprint planning.
Gather and process raw data at scale from internal and external data sources and expose mechanisms for large scale parallel processing.
Solve complex SQL and Big Data Performance challenges.
Implement controls, policies, processes and best practices in the Data Engineering space.
You have consistently high standards, your passion for quality is inherent in everything that you do.
Evangelize an extremely high standard of code quality, system reliability, and performance.
Help us improve our database deployment and change management process.
Work closely with the team on development best practices and standards.
Who You Are
You have prior experience with data engineering efforts across a variety of data systems.
You have deep understanding of commercial data sources and understand database concepts and terminology.
You have a demonstrated track record of handling multiple complex sourcing projects and delivering results in the data engineering area.
You have strong SQL experience and the ability to work on multiple aspects of a data projects including ETL, tools integrations, data results and APIs.
You are a team player, with the courage to drive change through disruption while maintaining a respect for the team.
Very Strong engineering skills. Should have an analytical approach and have good programming skills.
Minimum of 5+ years’ experience. Experience in retail business will be a plus.
Excellent written and verbal communication skills for varied audiences on engineering subject matter.
Ability to document requirements, data lineage, subject matter in both business and technical terminology.
Demonstrated ability to transform business requirements to code, specific analytical reports and tools.
This role will involve coding, analytical modeling, root cause analysis, investigation, debugging, testing and collaboration with the business partners, product managers other engineering team.
Demonstrates expertise in writing complex, highly optimized queries across large data sets.
Proven work experience in Spark, Python/Scala, Java, SQL, Hive, any RDBMS/NoSQL database.
Strong Hadoop scripting skills to process petabytes of data.
Proven track record in producing code (e.g., COBOL, SQL, Java, Python, Scala), deploying applications on Mainframe and/or Unix/Linux.
Exposure to one or more tools such as apache Airflow, Nifi.
Outlined below are the required minimum qualifications for this position. If none are listed, there are no minimum qualifications.Bachelor’s degree in Computer Science and 2 years’ experience in software engineering or related field OR 4 years’ experience in software engineering or related field.
Outlined below are the optional preferred qualifications for this position. If none are listed, there are no preferred qualifications.Masters: Computer Science