Data/Full Stack Engineer, Data Storage & Ingestion Consultant

Job description

About us At Eon, we are at the forefront of large-scale neuroscientific data collection. Our mission is to enable the safe and scalable development of brain emulation technology to empower humanity over the next decade, beginning with the creation of a fully emulated digital twin of a mouse. Role: We’re a San Francisco team collecting very large microscopy datasets and we need an expert to design and implement our end-to-end data pipeline, from high-rate ingest to multi-petabyte storage and downstream processing. You’ll own the strategy (on-prem vs. S3 or hybrid), the bill of materials, and the deployment, and you’ll be on the floor wiring, racking, tuning, and validating performance. Our current instruments generate data at ~1+ GB/s sustained (higher during bursts) and the program will accumulate multiple petabyes total over time. You’ll help us choose and implement the right architecture considering reliability and cost controls. Outcomes (what success looks like): Within 2 weeks: Implement an immediate data-handling strategy that reliably ingests our initial data streams. Within 2 weeks: Deliver a documented medium-term data architecture covering storage, networking, ingest, and...