Job Description

Lensa is a career site that helps job seekers find great jobs in the US. We are not a staffing firm or agency. Lensa does not hire directly for these jobs, but promotes jobs on LinkedIn on behalf of its direct clients, recruitment ad agencies, and marketing partners. Lensa partners with DirectEmployers to promote this job for Insight Global. Clicking "Apply Now" or "Read more" on Lensa redirects you to the job board/employer site. Any information collected there is subject to their terms and privacy notice.

• Translate business strategy into technical execution, ensuring data platforms enable both real-time and batch analytics at scale.
• Lead, mentor, and grow teams of data engineers and engineering managers with diverse experience levels.
• Acting as a lead/SME to the team building large scale, real time data pipelines with spark on Databricks.
• Working with cross functional teams and analysts to understand business requirements leverage our data.
• Translate complex business requirements into scalable, high-impact data solutions
• Build and align teams through resource planning, team member allocation, and
staffing projections
• Define and communicate a forward-looking technical roadmap for Data Engineering spanning multiple quarters to one year
  

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

• 5+ years of experience Spark, working in SparkSQL and/or DataFrames/Datasets API (with emphasis on DataFrames) to query and perform data manipulation
• 5+ years experience functioning as a lead or senior resource on a team.
• Experience building large scale Spark applications, ideally with either Batch processing and/or Streaming processing
• 3+ Python would be ideal but a solid knowledge of Scala is also acceptable
• 4+ Experience with cloud computing platforms, we use AWS (Kinesis, S3, Lambda)
• Has experience with ANSI SQL relational database (Oracle, SQL, Postgres, MySQL)
• Linux common working knowledge, including navigating through the file system and simple bash scripting
• General knowledge of distributed systems and distributed data processing frameworks
• Knowledge about agile software processes

If you have questions about this posting, please contact support@lensa.com

Job Tags

Similar Jobs