Data Engineer

Job description

Responsibilities We are seeking a highly skilled and innovative Data Engineer to join our team in the greater DMV area, supporting the Army National Guard. Responsibilities: - Lead design, build, and optimization of data pipelines to ingest, normalize, and store logs, events, and telemetry for SOC, DCO, vulnerability management, CDAP, and analytics use cases. - Implement schemas, mappings, ETL/ELT logic, and data models to support correlation, threat hunting, reporting, dashboards, and RMF/CCRI evidence generation. - Onboard and integrate new data sources (OT/DCI telemetry, threat feeds, CDAP outputs, cloud logs), ensuring proper parsing, enrichment, and normalization. - Monitor data quality, latency, completeness, and pipeline reliability; identify gaps and implement remediation or fallback strategies. - Collaborate with detection engineers, analysts, and data consumers to translate analytic requirements into pipeline designs and delivery SLAs. - Develop and maintain data flow documentation, schemas, codebooks, and lineage artifacts to support traceability and auditability. - Implement data retention, partitioning, and storage optimization patterns to balance performance, cost, an...