GYAbout Gaurav
Experienced Data Engineer specializing in Big Data, ETL, and cloud solutions. Skilled in Python, PySpark, AWS, and Apache Iceberg. I build scalable data pipelines for analytics and business insights.
English
Native or bilingual
Can work on-site
Dubai City (up to 50km)
Experience
- Airbus Group India Private LimitedSoftware EngineerDecember 2022 - Today (3 years and 6 months)Bengaluru, KA, IndiaSkills Used: Palantir Foundry, Python, Machine Learning, Apache Spark, AWS Lambda, AWS SQS, Automation, API Development, Data Optimization, Project Management• Developed a machine learning model to identify optimal profiles for Spark jobs, reducing resource usage by 30% and aligning with Cloudera optimization best practices.• Reduced costs by 25% through automation of manual processes using AWS Lambda functions, showcasing strong scripting and orchestration skills.• Leveraged AWS Lambda and SQS to enable concurrent API execution, cutting job processing time by 40%.• Created and optimized Spark jobs, achieving a 35% reduction in resource wastage and a 30% cost saving, demonstrating advanced PySpark proficiency with DataFrames and RDDs.• Led the end-to-end implementation of 5 high-impact projects, improving project delivery time by 30% using PySpark and Cloudera tools.• Designed a machine learning model to predict resolver group assignments for incidents, enhancing response time by 20% through efficient data utilization and automation.
- Larsen & Toubro Infotech LtdGraduate Engineering TraineeNovember 2016 - January 2017 (2 months)Chennai, Tamil Nadu, India• Big Data Technologies: Hadoop, HDFS, Spark (PySpark), Hive• Database Management: MySQL, Database Scripting• Cloud Platforms: Cloudera Manager
- IBM India Pvt LtdSenior Big Data DeveloperJuly 2017 - November 2022 (5 years and 4 months)Pune, Maharashtra, IndiaSkills Used: PySpark, Cloudera Data Platform (CDP), Hadoop, Hive, Sqoop, Impala, HDFS, Data Warehousing, Apache Oozie, Linux Scripting, Kafka, ETL Development• Led a team of 8 professionals, delivering 3 critical projects within budget and on schedule, utilizing Cloudera CDP component s such as Hive, Impala, and HDFS.• Spearheaded the integration of a new billing system into the client's data lake using Hadoop, Spark, and HDFS, reducing operational inefficiencies by 25%.• Engineered a framework to transfer data to DB2, optimizing ETL workflows and reducing data transfer time by 30% with PySpark and Cloudera tools.• Transformed the client's billing process with Hive and Sqoop, achieving a 40% efficiency improvement and a 30% increase in data accuracy.• Enhanced orchestration workflows using Apache Oozie, streamlining task scheduling and improving overall project timelines by 20%.• Streamlined data lake infrastructure to improve data accessibility and data-driven decision-making by 20%.• Demonstrated advanced proficiency in PySpark by optimizing Spark jobs and applying RDD/DataFrame transformations.
Recommendations
Be the first to recommend Gaurav
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- Bachelor of EngineeringBansal Institute of Research Technology and Science2016Bachelors of engineering, Computer Science
- HSC, MathematicsST. THOMAS H.S. SCHOOL JABALPUR2009HSC, Mathematics