Data Engineer- REMOTE

Requisition Number: 60903

Corning is vital to progress – in the industries we help shape and in the world we share.

We invent life-changing technologies using materials science. Our scientific and manufacturing expertise, boundless curiosity, and commitment to purposeful invention place us at the center of the way the world interacts, works, learns, and lives.

Our sustained investment in research, development, and invention means we’re always ready to solve the toughest challenges alongside our customers.

The global Information Technology (IT) Function is leading efforts to align IT and Business Strategy, leverage IT investments, and optimize end to end business processes and associated information integration technologies. Through these efforts, IT helps to improve the competitive position of Corning's businesses through IT enabled processes.

IT also delivers Information Technology applications, infrastructure, and project services in a cost efficient manner to Corning worldwide.

Purpose of the Position:

The Data Engineer, Advanced Analytics platforms will work with our core platform development team as well as domain experts, application developers, controls engineers and data scientists.

The primary responsibility will be to develop reliable and instrumented data ingestion pipelines that land inbound data from multiple process and operational data stores throughout the company to on-premise and cloud-based data lakes.

These pipelines will require data validation and data profiling automation along with version control and CI/CD to ensure ongoing resiliency and maintainability of the inbound data flows supporting our advanced analytics projects.

Day to Day Responsibilities

• As a Data Engineer for our advanced analytics platforms, your main responsibilities will be:
  • Design, test, deploy and maintain production big-data ingestion pipelines using established frameworks, patterns of practice, agile software development and CI/CD practices
  • Work with cross-organizational data source teams to define data ingestion requirements for structured, unstructured and semi-structured data, pilot their implementation, ensure the data source teams accept the resulting landed data as valid
  • Define and implement automated validation and profiling capabilities needed to ensure reliable data delivery, using agile software development and CI/CD practices
  • Work with data source teams, domain experts and data scientists to define data cleansing and data enrichment requirements for landed data
  • Implement data cleansing and enrichment code using established patterns of practice
  • Work with data source teams, domain experts and data scientists to validate landed, cleansed and enriched data, using agile software development and CI/CD practices, while ensuring that the final datasets are directly usable by them without additional processing effort
  • Participate in code reviews and technical information sharing with your team members and the broader software engineering community at Corning
  • Stay up to date with industry standards and technological advancements that will improve the quality, productivity and performance of your work.
  • Provide support in a DevOps environment to monitor tokens, jobs and overall system performance.

Education and Experience:

• Bachelor's degree in computer science, engineering, mathematics, or a related technical discipline
• Prior experience in big data engineering roles, developing and maintaining ETL and ELT pipelines for data warehousing, on-premise and cloud datalake environments
• Expert level fluency in SQL to write and debug complex queries
• 3 years of demonstrated proficiency in Python or Spark applied to large scale data sets
• Strong understanding of workflow management platforms/ orchestration tools (Airflow or similar)
• 3 years of experience developing batch, micro-batch and streaming ingestion pipelines using high-level Apache Spark APIs (pySpark, SparkSQL, SparkR or Scala)
• Obsession for service observability, instrumentation, monitoring and alerting
• 强,第一手技术熟悉Databricks, S3, parquet and Delta Lake architecture, technologies and tools
• Understanding of the Data Lifecycle Management process to collect, access, use, store, transfer, delete data
• Understanding of relational databases (e.g., MySQL, PostgreSQL), NoSQL databases (e.g., key-value stores like Redis, DynamoDB, RocksDB)
• Expert level proficiency with agile software development & continuous integration + continuous deployment methodologies along with supporting tools such as Git (Gitlab), Jira, Terraform, New Relic
• Strong, firsthand familiarity with notebook environments including JupyterHub
• DevOps experience with AWS platform services, including AWS S3 & EC2, Data Migration Services (DMS), RDS, EMR, RedShift, Lambda, DynamoDB, CloudWatch, CloudTrail
• Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and data engineering strategy

Preferred Qualifications

• 之前完整美联社p development experience (front-end, back-end, microservices)
• Familiarity with the following tools and technology practices: Oracle, Microsoft SQL Server
• Established enterprise ETL and integration tools including Informatica, Talend, dbt
• Established opensource data integration and DAG tools
• Reporting and analysis tools including PowerBI, Tableau, SAS JMP

Corning Puts YOU First!

We are committed to supporting your health, financial, career development, and life goals as you grow professionally and personally to achieve your highest potential. All benefits begin as soon as you start your career at Corning.

• Our monetary peer-to-peer recognition program is tied to our值and celebrates you and your colleagues’ contributions.
• Health and well-being benefits include medical, dental, vision, mental health/substance use, fitness, and disease management programs.
• Financial benefits include a 401(k) savings plan with company matching contributions and a 100% company-paid pension benefit that grows steadily throughout your career.
• Companywide bonus and attractive short- and long-term compensation programs are available based on your role and responsibilities.
• Professional development programs help you grow and achieve your career goals.

We prohibit discrimination on the basis of race, color, gender, age, religion, national origin, sexual orientation, gender identity or expression, disability, veteran status or any other legally protected status.

The range for this position is ($105,000-$160.000).Starting pay for the successful applicant is dependent on a variety of job-related factors, including but not limited to geographic location, market demands, experience, training, and education.The benefits available for this position include medical, dental, vision, 401(k) plan, pension plan, life insurance coverage, disability benefits, and PTO.