Sr. Big Data Engineer
Chicago, IL
Category: Engineer
Industry: Technology
Reference ID: JN -062026-107417
Date Posted: 07/11/2026
Shortcut: http://careers.eliassen.com/Kkdbdn
Description:Hybrid at least 2 days per week in office in Chicago, IL
Our client seeks a Staff Data Engineer to set technical direction for large-scale data processing in a privacy-preserving attribution platform. The role will architect and operate secure pipelines across cloud environments, collaborate with cross-functional teams, implement governance and access controls, and drive architecture for batch and streaming workflows. The engineer will build observability and operational tooling, troubleshoot distributed systems, mentor senior engineers, and evolve attribution, measurement, forecasting, and analytics capabilities with strong security and compliance alignment.
We can facilitate w2 and corp-to-corp consultants. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.
Rate: $84.00 to $94.00/hr. w2
Responsibilities:
- Set technical direction for large-scale data processing solutions using Scala, Spark, SQL, and modern data platforms across the attribution ecosystem.
- Architect and operate trusted pipelines handling advertiser, customer, and measurement datasets within secure cloud environments and approved data-sharing ecosystems.
- Collaborate with Product, Data Science, Security, Privacy, and Platform Engineering to deliver privacy-preserving attribution, measurement, forecasting, and analytics solutions.
- Define architecture for batch and streaming workflows, including orchestration frameworks and cloud-native services.
- Implement data classification, access controls, and privacy-preserving processing techniques to meet security and compliance requirements.
- Establish architectural direction for clean-room and trusted data-sharing environments with approved aggregated or privacy-protected outputs.
- Build observability, monitoring, and operational tooling to ensure reliability, performance, and compliance.
- Troubleshoot complex platform, performance, and pipeline issues across distributed systems.
- Drive architecture decisions, engineering best practices, and operational excellence across attribution and adjacent data products.
- Mentor and grow senior and lead engineers and provide technical leadership across teams and product areas.
- Design and implement pipelines within trusted environments across AWS, GCP, Azure, and on-premises infrastructure.
- Implement access controls, data classification policies, lineage tracking, and governance controls for sensitive data.
- Define and maintain data handling standards with Security, Privacy, and Compliance stakeholders.
- Enforce privacy-preserving principles to ensure only aggregated, anonymized, tokenized, or otherwise approved outputs leave trusted environments.
- Build monitoring and alerting to detect anomalous movement, policy violations, and potential leakage.
- Apply privacy-preserving computation techniques, including aggregation-before-export, pseudonymization and tokenization, differential privacy concepts, and privacy-aware reporting.
- Implement encryption, key management, and secure data handling using cloud-native services.
- Document trust boundaries, data contracts, lineage, and permitted data movement between zones.
- Support audits, compliance requirements, governance reviews, and secure data-sharing initiatives.
- Participate in architecture and design reviews for new data products to incorporate governance, privacy, lineage, and trust-boundary requirements.
- Contribute to engineering standards and best practices for secure data processing and privacy-preserving analytics.
Experience Requirements:
- 10+ years of data engineering with deep Scala and extensive Apache Spark for large-scale processing on AWS and/or GCP.
- Strong Python for pipelines, tooling, automation, and infrastructure modules.
- Advanced SQL across relational, cloud warehouses, and lakehouse platforms with TB-scale datasets.
- Designing, building, and maintaining batch and streaming pipelines.
- Strong understanding of warehousing, dimensional modeling, data quality, partitioning, and performance optimization.
- Experience with lakehouse architectures such as Databricks, Delta Lake, or equivalent.
- Operating distributed data platforms at scale.
- Workflow orchestration with Airflow, Databricks Workflows, AWS Step Functions, or equivalent.
- Source control and testing practices including unit, integration, and automation.
- Cloud-native development on AWS and/or GCP.
- Strong engineering practices including CI/CD, code reviews, observability, and production support.
- Proven ability to set technical direction across teams and mentor senior engineers.
- Trusted environment execution: clean rooms and secure data-sharing platforms, handling PII and regulated data, fine-grained access controls, and policy-based enforcement.
- Working knowledge of data classification frameworks including PII, PCI, regulated data, and sensitivity-tier models.
- Familiarity with tokenization, pseudonymization, aggregation-before-export, and differential privacy concepts.
- Experience with clean-room, measurement, attribution, audience analytics, partner data-sharing, or privacy-preserving reporting.
- Lineage and governance tooling such as Unity Catalog, AWS Glue Data Catalog, Apache Atlas, or OpenLineage.
- Understanding of trust boundaries, secure sharing patterns, and zero-trust data architecture principles.
- Documenting data contracts, flows, lineage, and permitted movement between security zones and domains.
- Encryption, key management, and secure handling using cloud-native security services.
- Designing observability and alerting controls to detect anomalous movement, policy violations, and potential leakage.
- Experience in environments where only aggregated, anonymized, tokenized, or privacy-protected outputs may exit trusted processing environments.
- Strong understanding of cloud-native security and governance.
- Good to have: Databricks ecosystem, AWS Clean Rooms, advertising measurement and attribution platforms, collaboration with Security and Compliance in regulated settings, ELK, Grafana, OpenTelemetry, Docker, Kubernetes, and security architecture or threat modeling.
- Strong written and verbal English communication skills.
- Good understanding of Agile and SCRUM within cross-functional teams.
Skills, experience, and other compensable factors will be considered when determining pay rate. The pay range provided in this posting reflects a W2 hourly rate; other employment options may be available that may result in pay outside of the provided range.
W2 employees of Eliassen Group who are regularly scheduled to work 30 or more hours per week are eligible for the following benefits: medical (choice of 3 plans), dental, vision, pre-tax accounts, other voluntary benefits including life and disability insurance, 401(k) with match, and sick time if required by law in the worked-in state/locality.
If anyone reaches out to you about an open position connected with Eliassen Group, please ensure that you are working directly with us by confirming the following:
· When you work with Eliassen Group, all email communication will come from an Eliassen.com address, never Gmail, Yahoo, etc.
· Eliassen Group will never ask you for personal information (home address, bank account, or check routing number) until you have worked with someone clearly associated with Eliassen Group.
If you have any indication of fraudulent activity, please contact fraud@eliassen.com.
-
Senior Data Engineer
Chicago, IL
Description: Hybrid At least 2 days per week in office in Chicago, IL Our client seeks a Senior Data Engineer to design, build, and operate large-scale data processing for attribution, measurement, forecasting, and privacy-preserving analytics. You wil...
Date Posted: 07/15/2026 Recommended
-
Lead Data Engineer
Chicago, IL
Description: Hybrid At least 2 days per week in office in Chicago, IL Our client seeks a Lead Data Engineer to design, build, and optimize large-scale data processing for attribution, measurement, forecasting, and privacy-preserving analytics. The role...
Date Posted: 07/11/2026 Recommended
-
Lead Software Engineer
Irving, TX
Description: Hybrid At least 2 days per week in office in Irving, TX Our client seeks a Lead Software Engineer to drive design, delivery, and technical strategy for cloud-native data platforms that power data-driven, omnichannel experiences. You will l...
Date Posted: 07/11/2026 Recommended
-
Cloud Database Engineer
Westlake, TX
Description: On-site Every Other Week onsite / 5 days in Westlake, TX Our client seeks a Cloud Database Engineer to design, implement, and operate scalable, resilient, and secure database platforms supporting operational and analytical workloads. The r...
Date Posted: 07/17/2026 Recommended
-
Lead Software Engineer
Chicago, IL
Description: Hybrid At least 2 days per week in office in Chicago, IL Our client seeks a Lead Software Engineer to drive the design, delivery, and technical strategy for cloud-native data platforms within a Customer Data Platform team. The role will de...
Date Posted: 07/11/2026 Recommended
-
Senior Software Engineer
Chicago, IL
Description: Hybrid At least 2 days per week in office in Chicago, IL Our client seeks a Senior Software Engineer for the CDP team to drive design, delivery, technical strategy, and execution for cloud-native data platforms. You will define engineering...
Date Posted: 07/15/2026 Recommended
-
Senior AI Security Architect – Data Security & DSPM
Anywhere
Description: Remote Our client has an excellent opportunity for a Senior AI Security Architect to own the end-to-end security architecture for the Agentic AI platform and lead the data security mandate for AI. This combined senior role unifies Agentic ...
Date Posted: 06/25/2026 Recommended
-
Lead Software Engineer
Boston, MA
Description: Hybrid at least 2 days per week in office in Wakefield, MA Our client seeks a Lead Software Engineer for the CDP team to drive design, delivery, technical strategy, and execution for critical data systems. You will define engineering stand...
Date Posted: 07/11/2026 Recommended
-
Product Data Analyst
Westlake, TX
Description: Hybrid Every Other Week onsite / 5 days in either Westlake, TX or Merrimack, NH or Durham, NC Our client has an opportunity for a Product Data Analyst to support Stock Plan Services Customer Solutions. The role blends business analysis, da...
Date Posted: 07/12/2026 Recommended
-
Cloud & Network Infrastructure Engineer
Washington, DC
Description: On-site 5 days/week in Washington, DC Our client requires a senior Cloud and Network Infrastructure Engineer to lead cloud and on‑premises network design, implementation, and operations. The role focuses on hybrid connectivity, security, a...
Date Posted: 07/08/2026 Recommended
-
Senior Software Engineer - Risk Team
Los Angeles, CA
Description: Hybrid in Los Angeles, CA Our client seeks a Senior Software Engineer for the Risk Team to design and deliver scalable, secure, and resilient backend services supporting fraud detection, risk scoring, and compliance automation. The role re...
Date Posted: 07/10/2026 Recommended
-
Infrastructure Engineer
Washington, DC
Description: Hybrid 3 days onsite / 2 days remote in Washington, DC Our client seeks a Senior Infrastructure Engineer to manage and optimize on-premises and cloud network environments. The role supports network hardware, monitoring tools, and core serv...
Date Posted: 07/19/2026 Recommended
-
Senior Systems Engineer
Fort Meade, MD
Description: On-site in Fort Meade, MD Our client seeks a Senior Systems Engineer to support the Defense Information Systems Agency Citadel mission. The role requires advanced engineering, integration, administration, and operational support for enterp...
Date Posted: 07/18/2026 Recommended
-
Cybersecurity Firewall Analyst
Charlotte, NC
Description: Hybrid 3 on in Charlotte, NC Our client seeks a Cybersecurity Firewall Analyst to support next-generation firewall administration, operations, and policy governance across enterprise and regulated environments. The role includes incident r...
Date Posted: 07/20/2026 Recommended
-
Google Cloud Platform Engineer
Washington, DC
Description: On-site in Washington, DC Our client seeks a Google Cloud Platform Engineer with hands-on experience across cloud administration, automation, integration, and application support. The role supports production GCP environments and contribut...
Date Posted: 07/01/2026 Recommended
-
Senior Cloud/DevOps Engineer
Fort Meade, MD
Description: On-site in Fort Meade, MD Our client seeks a Senior Cloud/DevOps Engineer to design, automate, secure, and operate enterprise cloud and hybrid platforms in support of a DISA Citadel mission within a secure DoD environment. The role impleme...
Date Posted: 07/18/2026 Recommended
-
Operating Systems Engineer
Anywhere
Description: Remote Our client seeks an Operating Systems Engineer to design, develop, and maintain platform adapters that extend the Darkstrike execution control platform to Microsoft Windows and macOS. The role collaborates with software engineering,...
Date Posted: 07/08/2026 Recommended
-
Mid Systems Engineer, CyberArk
Washington, DC
Description: Onsite in Washington, DC Our client seeks a Mid Systems Engineer, CyberArk, to support a Privileged Access Management program for a federal environment. The role will assist senior engineers with CyberArk deployment, operations, reporting,...
Date Posted: 07/11/2026 Recommended
-
Senior Systems Analyst
Westlake, TX
Description: On-site in either Westlake, TX or Merrimack, NH or Smithfield, RI Our client seeks a Senior Systems Analyst within Corporate Services Technology supporting Legal Strategy & Shared Services. You will plan, coordinate, and deliver technology...
Date Posted: 06/25/2026 Recommended
-
DevOps Engineer – Database Security Monitoring
Merrimack, NH
Description: On-site every other week / 5 days in Merrimack, NH or Westlake, TX Our client is seeking a DevOps Engineer to join a cybersecurity-focused team supporting database activity monitoring across critical platforms. This role will focus on clou...
Date Posted: 06/22/2026 Recommended
-
Product Manager - Lead
Richmond, VA
Description: Hybrid Must sit onsite in RVA, McLean, or Chicago in Richmond, VA Our client seeks a Product Manager to lead development of an internal Model Governance platform within Enterprise Risk. The role will define and deliver end-to-end user expe...
Date Posted: 07/08/2026 Recommended
-
Sr. Mobile Developer
Anywhere
Description: Remote in either Washington, DC As a Sr. Mobile Developer (Cross-Platform), you will design, build, and sustain secure iOS and Android applications to deliver mission-critical mobile capabilities for federal stakeholders. Our engineering t...
Date Posted: 06/30/2026 Recommended
-
Business Process Lead — SAP S/4HANA Data (MDG & Migration)
Duluth, GA
Description: Hybrid 3 days a week onsite in Duluth, GA Our client, a leader in the manufacturing and infrastructure space, has an excellent opportunity for a Business Process Lead — SAP S/4HANA Data to work on a 12‑month + contract position working in ...
Date Posted: 07/16/2026 Recommended
-
Data Engineer
Westlake, TX
Description:Hybrid Every Other Week onsite / 5 days in Westlake, TX Our client has an opportunity for a Data Engineer on a 12+ month contract. The role will design, develop, and support enterprise-scale data solutions that enable reporting, analytics, ...
Date Posted: 07/13/2026 Recommended
-
Associate Cybersecurity Engineer
Fenton Cary, NC
Description: Hybrid Tuesday – Thursday in Cary, NC Our client is seeking an Associate Cybersecurity Engineer to support the protection of data and systems across core platforms. The role focuses on vigilant monitoring, operation, and continuous optimiz...
Date Posted: 07/05/2026 Recommended
-
Mid Systems Engineer, PAM (CyberArk)
Washington, DC
Description: On-site in Washington, DC Step into the role of a Senior-Level CyberArk Engineer supporting our client at the Department of Transportation. You will contribute to deploying and maintaining a Privileged Access Management solution across dev...
Date Posted: 07/16/2026 Recommended
-
Platform Architect
Anywhere
Description: Remote Our client seeks a ServiceNow Platform Architect to lead architecture, governance, and strategic evolution of the enterprise ServiceNow platform. The role defines platform direction, ensures scalable and secure implementations, and ...
Date Posted: 07/15/2026 Recommended
-
Senior Full Stack Engineer
Westlake, TX
Description: On-site Every Other Week onsite / 5 days in Westlake, TX Our client seeks a Senior Full Stack Engineer to build omni-channel user experiences and platform capabilities that support a broad digital presence for millions of customers. You wi...
Date Posted: 07/15/2026 Recommended
-
Security Engineer
New York, NY
Description: Hybrid 4 days on site in New York, NY Our client is building secure and scalable infrastructure for institutional digital finance. The team focuses on security engineering, cryptography, blockchain, and research-backed engineering that bri...
Date Posted: 07/05/2026 Recommended
-
Senior Performance Engineer
Jersey City, NJ
Description: On-site in Jersey City, NJ Our client seeks a Senior Performance Engineer to define strategy, establish performance excellence, and create systems that make performance a default outcome for software delivery within digital asset platforms...
Date Posted: 07/09/2026 Recommended
Eliassen Group is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status. Eliassen Group’s Affirmative Action Plan (AAP) is available for inspection by any employee or applicant for employment upon request, during normal business hours of Monday through Friday, 8:30am to 5:30pm EST. Interested persons should contact Phaedra Wells at pwells@eliassen.com for assistance. It is unlawful in Massachusetts and Maryland to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
Applicants with disabilities that believe they require an accommodation or assistance with a position, please email our HR team at hradmin@eliassen.com. This email inbox is designed exclusively to assist job seekers whose disability prevents them from being able to apply online. Emails sent for other purposes will not receive a response.
Please be advised that a number of fraudulent job postings have been released under the Eliassen Group brand.
Unfortunately, fraudulent job postings can happen. If anyone reaches out to you about an open position connected with Eliassen Group, never provide personal or financial information to anyone who is not clearly associated with Eliassen Group
If anyone seemingly from Eliassen Group has ever requested this personal information in the past or does so in the future, please contact fraud@eliassen.com.
Please ensure that you are working directly with us by confirming the following:
- When you work with Eliassen Group, all email communication will come from an Eliassen.com address, never Gmail, Yahoo, etc.
- Eliassen Group will never ask you for personal information (home address, bank account, or check routing number) until you have worked with someone clearly associated with Eliassen Group, as indicated above