This posting has been closed |
Job Description: Data Architect - Data Warehouse/Data Marts
This position will be responsible for setting up the correct architecture and framework, developing, testing, and implementing tools/processes related to data acquisition/transfer, integration, and management using Talend as ETL tool. Experience deploying and managing data in an AWS and Snowflake environment is required.
Responsibilities:
• Set up processes to acquire data in structured/semi-structured format, and implement tools/processes to consume data received, as a batch process or real-time using Talend.
• Analyze data, design and build S3 folder structure and Snowflake/Redshift tables to capture all data elements, and load data, while maintaining data lineage and operational tracking.
• Use APIs or source XML Type columns to dynamically extract, integrate and load data in target schema using Talend
• Should be responsible for building extraction and mapping rules for loading data from multiple sources for greenfield data warehouse implementation based on AWS and Snowflake.
• Continuously optimize, enhance, monitor, support and maintain all Talend data integration processes and should be an expert in Talend Big Data jobs.
• Should have experience working with multiple file formats especially with Parquet and Avro
• Should have experience with moving data into S3 folder structures and working with Talend Spark jobs on AWS EMR.
• Should contribute to logical data model for data warehousing and making data available for downstream consumption
• Provide guidance or assistance to client/vendor on setting up processes to extract and deliver data for consumption.
• Maintain documentation, manage source code and deployments, and implement best-practices
• Lead on aspects of day to day delivery in agile and devops delivery environments
• Lead and guide other developers, coordinate and oversee assignments, timing and quality of their deliverables
Requirements:
• 10+ years of total experience in IT and seeking previous 7+ years with Talend ETL / Data Integration / Big Data / Cloud (AWS) Technologies experience
• Knowledge of Oracle, Unix/Linux Shell scripting, Autosys scheduling tool and version control Tools
• Must have hands on experience on AWS S3/EMR/Lambda/Kinesis/Batch
• Hybrid/Multi Cloud implementation experience in Data Warehouse
• Experience on Snowflake/ Redshift
• Experience with CI/CD pipeline setup using Jenkins
• Nice to have experience with P&C Insurance
• AWS/ Talend certification preferred