Job Title: Cloud Engineer
Required Skills - AWS, Snowflake, Python / Pyspark, SQL, ETL & Data warehousing
Good to Have Skills - Data Modelling, Dimensional Modelling
Experience: 5 - 12 years
Location: Greater Noida
We at Coforge are hiring Cloud Engineers with the following skillset:
- Must be extremely proficient in Data Warehouse ETL Design / Architecture & dimensional / relational data modelling.
- Experience in at least one ETL development project, writing / analyzing complex stored procedures.
- Should have entry - level / intermediate experience in Python / PySpark – working knowledge of Spark / pandas dataframe, Spark multi - threading, exception handling, familiarity with different boto3 libraries, data transformation & ingestion methods, ability to write UDF.
- Snowflake – Familiarity with stages & external tables, commands in Snowflake like copy, unload data to / from S3, working knowledge of variant data type, flattening nested structure through SQL, familiarity with marketplace integrations, role - based masking, pipes, data cloning, logs, user & role management is nice to have.
- Familiarity with Coalesce / dbt is an added advantage for this job.
- Collibra integration experience for Data Quality and Governance in ETL pipeline development is nice to have.
- AWS - Should have hands - on experience with S3, Glue (jobs, triggers, workflow, catalog, connectors, crawlers), CloudWatch, RDS & secrets manager.
- AWS - VPC, IAM, Lambda, SNS, SQS & MWAA is nice to have.
- Should have hands - on experience with version control tools like GitHub, working knowledge of configuring & setting up CI / CD pipelines using YAML, pip files.
- Streaming Services - Familiarity with Confluent Kafka or Spark streaming, or Kinesis (or equivalent) is nice to have.
- Data Vault 2.0 (hubs, satellite links) will be a plus.
- Highly proficient in Publisher, PowerPoint, SharePoint, Visio, Confluence & Azure DevOps.
- Working knowledge of best practices in value - driven development (requirements management, prototyping, hypothesis - driven development & usability testing).
- Good communicator with problem - solving mindset & a focus on process improvement.
- Consistently demonstrates clear & concise written and verbal communication skills.
- Good interpersonal skills, ability to interact with Senior Management.
- Highly self - motivated with a strong sense of initiative.
- Excellent multitasking skills & task management strategies
- Ability to work well in a team environment, meet deadlines, demonstrate good time management, & multi - task in a fast-paced project environment.
- Partner with business stakeholders to gather requirements & translate them into technical specifications and process documentation for IT counterparts (onshore and offshore).
- Highly proficient in the architecture and development of an event - driven data warehouse; streaming, batch, data modeling & storage.
- Advanced database knowledge, creating / optimizing SQL queries, stored procedures, functions, partitioning data, indexing & reading execution plans.
- Skilled experience in writing & troubleshooting Python / PySpark scripts to generate extracts, cleanse, conform & deliver data for consumption.
- Expert level of understanding & implementing ETL architecture, data profiling, process flow, metric logging & error handling.
- Support continuous improvement by investigating & presenting alternatives to processes and technologies to an architectural review board.
- Develop & ensure adherence to published system architectural decisions and development standards.
- Multi - task across several ongoing projects and daily duties of varying priorities as required.
- Interact with global technical teams to communicate business requirements & collaboratively build data solutions.
