Must Have Technical/Functional Skills

 ESSENTIAL QUALIFICATIONS

• Experience: Minimum of 5+ years of experience as a Data Engineer or ETL Developer in complex, large-scale data environments.

• Databricks Expertise: Strong hands-on experience working with Databricks, including using Apache Spark for data processing and optimization.

• ETL Tools: Proficient with DataBricks ETL tools and frameworks 

• Big Data Technologies: In-depth knowledge of big data processing frameworks like Spark, Kafka, etc.

• NoSQL/MongoDB: Expertise in working with NoSQL databases, especially MongoDB, for large-scale data storage, retrieval, and optimization.

• Programming Skills: Proficient in Python or Java, Power shell for building data pipelines.

• NoSQL Proficiency: Experience working with both relational databases and NoSQL databases like MongoDB.

• Data Modeling: Expertise in designing and implementing data models, including OLAP, OLTP, dimensional, and document-based models (NoSQL).

• Data warehousing: Data warehousing is a key part of ETL process, as it stores data from multiple sources in an organized manner and will be needed to build a repeatable ETL workflow to support many different data sources.

• Redesign and refactor legacy custom ETL processes to reusable ETL workflows that can ingest diverse data sources to normalize data to standard JSON / XML output.

• Data Governance: Knowledge of data governance, security standards, and best practices for managing sensitive data.

• Version Control: Experience with Git or other version control systems for code management.

• Certification: Databricks certification, MongoDB certification, or other relevant certifications in data engineering, cloud platforms, or big data technologies.

• Soft Skills: Strong problem-solving skills, excellent communication, and the ability to work in a collaborative team environment.

• Analytical Mindset: Ability to translate business requirements into scalable, efficient, and reliable ETL solutions.

• Solid understanding of legacy communication protocols and migration strategies.

• Experience with cloud platforms like AWS, Azure, or Google Cloud

Roles & Responsibilities

• ETL Design and Development: Develop, maintain, and optimize ETL processes for data ingestion, transformation, and data warehousing across multiple platforms, including NoSQL databases.

• Data Pipelines: Design, build, and manage scalable data pipelines using technologies like Databricks, Apache Spark, Python, and NoSQL databases.

• NoSQL/MongoDB Expertise: Work with MongoDB to design efficient document schemas, implement query optimization, and handle large-scale unstructured data.

• Data Integration: Collaborate with cross-functional teams to ensure seamless data integration between different sources such as databases (both relational and NoSQL), APIs, and external files. 

• Performance Optimization: Implement and monitor performance metrics, optimize data processing performance, and manage ETL job scheduling and dependencies.

• Data Quality: Ensure data quality and integrity across ETL pipelines, implementing processes for data validation, cleansing, and enrichment.

• Automation: Automate repeatable ETL tasks and data processing workflows to improve efficiency and accuracy.

• Collaboration: Work closely with data architects, analysts, and business stakeholders to gather and understand data requirements.

• Cloud Platforms: Leverage cloud services (Azure, GCP) for data storage, processing, and infrastructure management, ensuring scalability and cost efficiency.

• Best Practices: Maintain documentation, adhere to data governance, and best practices in data management, including security and compliance.

• Build Microservices APIS to expose ETL services .

Generic Managerial Skills, If any

 Leading the team and driving the projects for closure

Salary Range: $90,000-$120,000 a year

TCS Employee Benefits Summary:

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

#LI-SP1