654 Data Engineer Python Pyspark jobs in New York,NY

Sr. Data Engineer

Subway Restaurants

Wilton, CT 06897

$119,200-$149,000/yr

~ 1 hr 8 min OnsiteUrgently HiringEducation AssistanceHealth InsuranceRetirement Benefit

Bachelor's degree in Computer or Information Science or related field, or equivalent combination of education and experience
5–8 years creating quality data pipelines and system integrations
At least 3 years of experience in a cloud environment
Hands-on experience building lakehouse solutions on Databricks (Delta Lake, Unity Catalog) or Snowflake (Iceberg Tables, Horizon Catalog)
Proven ability to design Bronze/Silver/Gold layers for curated, analytics-ready data
Experience implementing Lambda or Kappa architectures using Databricks Structured Streaming / DLT or Snowflake Dynamic Tables / Snowpipe Streaming
Experience building and managing semantic models using Databricks AI/BI Genie or Unity Catalog Metrics or Snowflake Semantic Views / Cortex Analyst
Building pipelines with Airflow, Databricks Lakeflow, or Snowflake Openflow; familiarity with dbt is a plus
Strong PySpark or advanced SQL skills; Python for data engineering or automation
Dimensional modeling, Data Vault, or schema design for analytical workloads
Cluster/warehouse sizing, partitioning, clustering keys, Z-ordering, or query optimization
Working knowledge of Unity Catalog (Databricks) or Horizon Catalog (Snowflake) for lineage, access control, or data quality
Git, Databricks Asset Bundles or Snowflake CLI/Schemachange, automated testing, or deployment pipelines
AWS (S3, Glue, Kinesis), Azure, or GCP services supporting modern data platforms
Familiarity with Databricks Mosaic AI or Snowflake Cortex for GenAI or ML use cases
Strong communication skills to partner with Product Owners, Analysts, or cross-functional engineering teams
Authorized to work in the country the position is based

SmartExplore AI is experimental.

Recommended

Apply Directly

Principal Engineer - Python API Development

Fidelity Investments

Weehawken, NJ 07086

~ 12 min OnsiteEducation AssistanceHealth InsurancePaid Time OffRetirement Benefit

Bachelor's or Master's degree in Computer Science, Software Engineering, or a closely related engineering discipline
8+ years (typically 10+) building and operating production platforms and services at scale
Deep software engineering expertise in Python and distributed systems
A track record of building production‑grade services, libraries, and internal platforms
Linux fluency and scripting are required
Cloud platform leadership (AWS) —hands-on with S3, Lambda, Batch, Step Functions, EventBridge, CloudWatch, and SNS/SQS—and experience shaping platform patterns that other teams adopt
Experience enabling managed ML services (e.g., SageMaker) as part of broader platform capabilities; exposure to Azure or GCP is beneficial
DevOps and CI/CD at scale, owning standards for automated build/test/deploy (e.g., Jenkins, Git‑based workflows), containerization (Docker), release governance, and multi‑environment promotion for ML‑enabled workloads
Infrastructure as Code (CloudFormation, Terraform/OpenTofu) and platform reliability engineering (SLOs/error budgets, capacity planning, cost observability, incident response, and post‑mortems) for ML serving and data/feature pipelines
ML enablement in production: model packaging, deployment strategies (batch/online/streaming), inference routing, traffic management, performance tuning, observability, and controls for responsible use—without a research or modeling focus
Cross‑org technical leadership: you mentor junior and senior engineers, are a backbone of code review across repos, and routinely consider impacts on upstream/downstream systems when proposing changes
Set platform strategy and standards for ML packaging, deployment, serving, and observability—driving consistent adoption across squads and business units
Partner with Data Scientists to package, scale, and operationalize models; define the APIs, guardrails, and automation that take work from experimentation to reliable production
Enable secure, scalable access to traditional and generative models by collaborating with platform and application engineers to integrate through enterprise gateways and services
Advance model/data observability—tooling for data and feature drift detection, prediction‑quality monitoring and uncertainty signals, and automated diagnostics/ explainability
Lead cross‑platform incident response and post‑mortems, drive systemic fixes, and evolve standards to prevent recurrence—across applications and the platform
Uplevel engineering velocity by introducing reusable frameworks, paved paths, and CI/CD templates that simplify integration, reduce toil, and improve reliability at scale
Reduce cost and complexity across the ML ecosystem through pragmatic technology choices, clear abstractions, and a long‑term platform roadmap
The base salary range for this position is $107,000-216,000 USD per year.
Base salary is only part of the total compensation package. Depending on the position and eligibility requirements, the offer package may also include bonus or other variable compensation.

SmartExplore AI is experimental.

Recommended

Apply Directly

Principal Engineer - Python API Development

Fidelity Investments

Lyndhurst, NJ 07071

~ 23 min OnsiteEducation AssistanceHealth InsurancePaid Time OffRetirement Benefit

Bachelor's or Master's degree in Computer Science, Software Engineering, or a closely related engineering discipline
8+ years (typically 10+) building and operating production platforms and services at scale
Deep software engineering expertise in Python and distributed systems
A track record of building production‑grade services, libraries, and internal platforms
Linux fluency and scripting are required
Cloud platform leadership (AWS) —hands-on with S3, Lambda, Batch, Step Functions, EventBridge, CloudWatch, and SNS/SQS—and experience shaping platform patterns that other teams adopt
Experience enabling managed ML services (e.g., SageMaker) as part of broader platform capabilities; exposure to Azure or GCP is beneficial
DevOps and CI/CD at scale, owning standards for automated build/test/deploy (e.g., Jenkins, Git‑based workflows), containerization (Docker), release governance, and multi‑environment promotion for ML‑enabled workloads
Infrastructure as Code (CloudFormation, Terraform/OpenTofu) and platform reliability engineering (SLOs/error budgets, capacity planning, cost observability, incident response, and post‑mortems) for ML serving and data/feature pipelines
ML enablement in production: model packaging, deployment strategies (batch/online/streaming), inference routing, traffic management, performance tuning, observability, and controls for responsible use—without a research or modeling focus
Cross‑org technical leadership: you mentor junior and senior engineers, are a backbone of code review across repos, and routinely consider impacts on upstream/downstream systems when proposing changes
Set platform strategy and standards for ML packaging, deployment, serving, and observability—driving consistent adoption across squads and business units
Partner with Data Scientists to package, scale, and operationalize models; define the APIs, guardrails, and automation that take work from experimentation to reliable production
Enable secure, scalable access to traditional and generative models by collaborating with platform and application engineers to integrate through enterprise gateways and services
Advance model/data observability—tooling for data and feature drift detection, prediction‑quality monitoring and uncertainty signals, and automated diagnostics/ explainability
Lead cross‑platform incident response and post‑mortems, drive systemic fixes, and evolve standards to prevent recurrence—across applications and the platform
Uplevel engineering velocity by introducing reusable frameworks, paved paths, and CI/CD templates that simplify integration, reduce toil, and improve reliability at scale
Reduce cost and complexity across the ML ecosystem through pragmatic technology choices, clear abstractions, and a long‑term platform roadmap
The base salary range for this position is $107,000-216,000 USD per year.
Base salary is only part of the total compensation package. Depending on the position and eligibility requirements, the offer package may also include bonus or other variable compensation.

SmartExplore AI is experimental.

New, Posted 1 day ago

Recommended

Easy Apply

Human Resources Technology Data Engineer

LHH US

New York, NY 10022

$85-$100/hr

HybridHealth InsurancePaid Time OffRetirement Benefit

Bachelor's degree in Computer Science, Engineering, or a related field.
5-8 years of professional software engineering experience.
Proven track record of hands on development in enterprise scale systems, preferably in HR.
Strong proficiency in at least one core programming language (Python or Java).
Deep experience with relational databases, NoSQL stores, and complex query design.
Experience working with HR, people operations, or workforce analytics data, including sensitive data handling, HRIS integrations (e.g., Dayforce, Culture Amp, etc.), and secure data exchange patterns.
Demonstrated ability to build or support platforms involving employee lifecycle data, compensation structures, organizational hierarchies, and talent management workflows.

SmartExplore AI is experimental.

Recommended

Easy Apply

Human Resources Technology Data Engineer

LHH US

Long Island City, NY 11101

$85-$100/hr

Hybrid~ 10 minHealth InsurancePaid Time OffRetirement Benefit

Bachelor's degree in Computer Science, Engineering, or a related field.
5-8 years of professional software engineering experience.
Proven track record of hands on development in enterprise scale systems, preferably in HR.
Strong proficiency in at least one core programming language (Python or Java).
Deep experience with relational databases, NoSQL stores, and complex query design.
Experience working with HR, people operations, or workforce analytics data, including sensitive data handling, HRIS integrations (e.g., Dayforce, Culture Amp, etc.), and secure data exchange patterns.
Demonstrated ability to build or support platforms involving employee lifecycle data, compensation structures, organizational hierarchies, and talent management workflows.

SmartExplore AI is experimental.

New, Posted 1 day ago

Recommended

Lead Software Engineer - Data Streaming/Integration Platforms

Disney Entertainment and ESPN Product & Technology Careers

New York, NY 10025

$155,700-$208,700/yr

Apply Directly

Manufacturing Engineering Manager - Boonton, NJ

Hubbell Incorporated

Boonton, NJ 07005

~ 49 min Onsite

BS in Electrical, Industrial, Mechanical Engineering or similar
5+ years experience in an Electronics, Electro-Mechanical, or relevant manufacturing environment
Proven track record for handling new product launches through PFMEA assembly line creation, ERP setup work etc.
Proven track record of conceptualizing and implementing Test automation framework on the manufacturing floor to reduce takt times and improve process quality,
Must be fluent with mechanical 3D CAD package.
Must be able to read schematics to troubleshoot board level issues
Experience with Electronics and manufacturing of Printed Circuit Board (PCB) Assemblies.
Must possess excellent verbal and written communications skills
An understanding of lean tools and concepts is essential, i.e. time studies, kaizen events, Value Stream Maps, etc
A continuous improvement mindset is critical
Must be familiar with ISO 9000

SmartExplore AI is experimental.

Apply Directly

Associate or Senior Associate, Fixed Assets / Cost Segregation Engineer (AMCS)

KPMG

New York, NY 10025

OnsiteHealth InsuranceRetirement Benefit

Bachelor's degree from an accredited college/university in engineering, architecture, construction management, or a closely related discipline is required
Minimumone year of recent experience in an engineering firm, corporation, or public accounting firm; background in cost segregation, fixed asset consulting, construction, engineering, or a related technical field is preferred
Understanding of construction processes and building systems, and how project costs translate into asset classifications
Basic knowledge of federal income tax depreciation principles and accounting for capital expenditures, or a demonstrated interest in developing this skillset
Experience reviewing technical drawings, construction documentation, or large project cost datasets
Strong proficiency in Excel (such as advanced formulas, pivot tables, lookups) and experience working with structured data from financial or operational systems; demonstrated experience or strong aptitude with data-wrangling/automation tools (such as Power Query, Alteryx), business intelligence tools (such as Power BI, Tableau), or query/scripting languages (such as SQL, Python, R)
Applicant must be authorized to work in the U.S. without the need for employment-based visa sponsorship now or in the future. KPMG LLP will not sponsor applicants for U.S. work visa status for this opportunity (no sponsorship is available for H-1B, L-1, TN, O-1, E-3, H-1B1, F-1, J-1, OPT, CPT or any other employment-based visa)

SmartExplore AI is experimental.

New, Posted 1 day ago

DiversityJobs