3245 Big Data Systems Engineer jobs in New York,NY

Featured Opportunity

Software Big Data Operations Engineer

Genesis10 - New York, NY

Software Big Data Operations Engineer

New, Posted 16 hours ago

Recommended

Apply Directly

DataBricks Data Engineer

INNOVA

New York, NY 10060

$70-$75/hr

OnsiteHealth InsurancePaid Time OffRetirement Benefit

At least 7 years of experience in data engineering, with a minimum of 3 years hands-on experience building and managing Databricks solutions.
Extensive experience with Databricks platform components: Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows.
Strong proficiency in Python and/or Scala for data engineering; SQL expertise required.
Strong experience with core data engineering practices including data ingestion, transformation (ETL/ELT), data modeling, and pipeline orchestration.
Experience integrating Databricks with cloud platforms (Azure, AWS, or GCP) and data sources such as Azure Data Factory, Event Hubs, Kafka, or equivalent.
Must work well in a team environment while being a resourceful, independent self-starter able to work effectively with minimal direction.

SmartExplore AI is experimental.

Recommended

ResumeLibrary

Senior AWS Redshift Data Engineer (Contract | 1099)

BrainTrust

New York, NY 10001

Recommended

Apply Directly

Product Engineer ( Systems)

Disability Solutions

Totowa, NJ 07512

$85,600-$145,500/yr

Apply Directly

Head of AI Data Science, Intelligence Ventures

Spectrum

New York, NY 10036

Onsite

Deep expertise in transformer-based sequence modeling and its application to behavioral or interaction data at consumer scale — including architecture design, training methodology, fine-tuning, and embedding quality evaluation
Proven track record developing and deploying household- or user-level embedding models applied to real-world use cases in media, marketing, commerce, and/or customer intelligence — not just research environments. Demonstrated understanding of the unique characteristics of behavioral sequence data: sparsity, temporal dynamics, multi-entity structure, and the signal differences between behavioral intent and explicit interaction
Strong command of the full data science lifecycle in production settings — from exploratory data analysis and feature engineering through model training, validation, deployment, monitoring, and iteration — at large dataset scale (billions, even trillions of records)
Hands-on proficiency with Python, PyTorch or TensorFlow, and distributed ML training frameworks; experience running ML workloads on cloud platforms (AWS SageMaker, Snowflake Cortex, Databricks, or equivalent)
Experience designing and operationalizing feature stores and predictive modeling pipelines that serve downstream intelligence products, audiences, or decision systems in production environments
Ability to communicate complex AI/ML concepts clearly to non-technical executive audiences, product stakeholders, and external partners; comfort operating as an external-facing technical spokesperson for the platform's modeling capabilities and intelligence differentiation
Track record of leading and growing high-performing data science teams; experience recruiting and developing senior ML talent in competitive markets
Genuine intellectual curiosity about the application of AI to behavioral science, consumer intelligence, and agentic systems; awareness of the evolving landscape of foundation models, retrieval-augmented generation, and multi-agent AI architectures
Bachelor's Degree in Computer Science, Statistics, Mathematics, or a related quantitative field
Experience leading applied ML or data science teams building consumer-facing or enterprise intelligence products — 7 years
Hands-on experience designing and training transformer or deep learning models on sequential behavioral data at scale — 5 years
In-office position preferably based in New York City
Travel as required for partner engagements, executive meetings, and industry events

SmartExplore AI is experimental.

New, Posted 1 day ago

Recommended

ResumeLibrary