Job Description

DescriptionKforce has a client that is seeking a Remote Senior Data Engineer.Responsibilities:* Design and own end-to-end data architecture: a landing zone with secure cloud object storage for raw partner files and API payloads, validated ingestion pipelines into our transactional Postgres, and a curated analytics layer that decouples reporting and AI workloads from production* Build ingestion pipelines for the data we receive today, including partner data files (CSV/JSON/XML/HL7/X12 as applicable) and REST/SFTP API integrations with schema validation, quarantine of bad records, and full lineage from raw bytes to curated row* Stand up and operate the curated layer (data warehouse/lakehouse-lite) so analytics and ML models can consume data without slowing down the transactional system* Choose, integrate, and operate the smallest set of tools needed, including object storage, an orchestrator (Dagster, Prefect, Airflow, etc.), dbt or similar for transformations, a single validation library (Great Expectations/Pandera/Soda)* Design and enforce data governance for a HIPAA-regulated environment: PHI/PII classification, encryption in transit and at rest, role-based access, audit logging, retention and minimum-necessary policies, and de-identification where appropriate* Partner with backend, ML, product, and clinical stakeholders to define data contracts with our health plan and ACO partners and hold the line on data quality* Build and maintain reliable feature data for ML models, including embeddings (e.g., pgvector) and curated feature tables for risk stratification, engagement, and outcomes work* Instrument the data platform for observability including pipeline SLAs, data freshness, schema drift, quality metrics, and act on what the data tells you* Participate fully in our Agile process: backlog grooming, sprint planning, demos, and retrospectivesRequirements* BS/BA degree or higher in Computer Science, Engineering, or a related technical field* 8+ years of professional data engineering experience, with a track record of shipping production data systems end-to-end* Mastery of PostgreSQL: schema design, indexing, query tuning, partitioning, logical replication, JSONB, extensions (pg_partman, pg_cron, pgvector, etc.), and operating Postgres at scale* Strong experience designing and operating data pipelines, including file-based ingestion (SFTP/object storage drops) and API-based ingestion (REST, webhooks)* Strong experience with dbt (or equivalent SQL-based transformation framework) and modern data modeling patterns (Kimball dimensional, Data Vault, One Big Table - and an opinion about when each is right)* Hands-on experience with one or more cloud platforms (AWS preferred) and their data primitives: object storage (S3), managed Postgres* Experience designing data warehouses and/or data lakes and the judgment to know which one a given problem actually needs* Experience with at least one orchestration framework (Dagster, Prefect, or Airflow) and a clear point of view on which to use when* Experience with data validation and data-quality frameworks (Great Expectations, Pandera, Soda, or equivalent)* Experience with change-data-capture from Postgres (logical replication, or equivalent)* Data governance experience in a HIPAA-regulated environment or, at minimum, demonstrated instincts for protecting PHI and PII (encryption, least privilege, audit, de-identification, BAA-aware vendor selection); HITRUST or SOC 2 experience is a strong plus* Experience supporting ML workloads* Experience using AI coding assistants (e.g., GitHub Copilot, Cursor, Claude) to accelerate development* Strong Python skills for ingestion, validation, and tooling* Comfortable with infrastructure-as-code and CI/CD for data systemsJob TypeDirect HireCompensation140000 - $170000

Job Tags

Similar Jobs