Summary
Overview
Work History
Education
Skills
Certification
Languages
Timeline
Generic

Harshavardhan Reddy Tanamala

Rome,RM

Summary

Big Data Engineer with 5+ years of experience specializing in the design and implementation of large-scale data processing systems. Expert in Hadoop, Spark, and Kafka, with a strong command of Python, Scala, and Java for developing robust data pipelines. Proven track record in optimizing ETL workflows, data integration, and real-time data streaming. Hands-on experience with cloud platforms like AWS and Google Cloud, utilizing services such as EMR, S3, Redshift, and BigQuery. Successfully led data projects that reduced processing times by 50% and improved data accuracy by 30%. Passionate about big data analytics and driving innovation through data-driven solutions.

Overview

5
5
years of professional experience
1
1
Certification

Work History

Big Data Engineer

Almaviva Spa
09.2022 - Current
  • Evaluated emerging big data technologies to ensure relevance with evolving industry trends, maintaining competitive advantage over peers.
  • Migrated legacy systems to modern cloud-based platforms for increased efficiency and scalability.
  • Conducted thorough performance testing to optimize system configurations and maximize resource utilization.
  • Enhanced data quality by developing robust validation strategies to identify and correct inconsistencies.
  • Proactively addressed potential bottlenecks in the ETL process through regular monitoring, enabling seamless workflow operations.
  • Optimized data processing by implementing Hadoop and Spark frameworks for big data management.
  • Designed scalable ETL pipelines for improved data ingestion, processing, and storage.
  • Developed custom visualization tools for better interpretation of complex datasets, aiding in strategic decision making.
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Compiled, cleaned and manipulated data for proper handling.
  • Supported business objectives by developing custom reporting tools using Python libraries like Pandas and Matplotlib.
  • Designed web-based assets for deployment on cloud computing platforms.
  • Generated workflow reports to test proposed change executions.
  • Streamlined the software development process with Agile methodologies and effective communication among team members.
  • Integrated machine learning models into Python applications for advanced data analysis and predictions.
  • Investigated new technologies like Docker or Kubernetes to streamline deployment processes while maintaining a consistent dev/prod environment.
  • Achieved substantial cost savings by migrating on-premises infrastructure to cloud-based services such as AWS or GCP.
  • Automated data ingestion, transformation, and aggregation processes leveraging Apache Hadoop, Spark, and Hive on Amazon EMR clusters.
  • Managed data flow between EMR and data lakes/warehouses like Amazon Redshift ensuring seamless data access and storage.
  • Developed CI/CD pipelines for deploying and updating EMR jobs using tools like Jenkins and CodePipeline.
  • Used Nifi and Python to create data flows and transformations.
  • Created team strategy for SDLC automation, configuration management and release management.
  • Managed AWS assets and integrated multiple AWS resources into solutions appropriate for company projects.
  • Built databases and table structures for web applications.
  • Tested and deployed scalable and highly available software products.
  • Performed routine maintenance tasks such as backups, index rebuilds, and updating statistics to maintain optimal system health.
  • Implemented effective database indexing strategies, enhancing query execution speed and overall application performance.
  • Participated in the review and selection process of new database technologies, ensuring alignment with organizational goals and long-term strategic plans.
  • Troubleshot critical production issues related to SQL Server databases, providing timely resolutions to minimize downtime.
  • Assisted in database migration projects, ensuring accurate data transfer between systems.
  • Optimized database performance by analyzing and fine-tuning complex SQL queries.
  • Developed scalable, high-performance applications using Hadoop ecosystem tools such as MapReduce, Pig, and Hive.
  • Optimized data storage and retrieval processes by implementing compression algorithms, reducing storage costs and improving query response times.
  • Enhanced system performance by optimizing Hadoop clusters and implementing efficient data processing strategies.
  • Ensured application security by implementing encryption protocols, access controls, and other measures to protect sensitive information from unauthorized access or tampering.
  • Migrated legacy systems to Hadoop-based platforms, improving data storage efficiency and accessibility.
  • Improved query performance through effective partitioning schemes, indexing strategies, and optimized database structures in HBase.
  • Implemented advanced analytics algorithms on large datasets using Spark, yielding valuable insights for business stakeholders.
  • Reduced data processing time for large-scale projects by streamlining ETL processes and leveraging distributed computing techniques.
  • Streamlined data ingestion processes by automating tasks and integrating various data sources into the Hadoop ecosystem.
  • Designed and maintained ETL/ELT workflows to ingest data into Snowflake from various sources, including cloud storage (AWS S3, Azure Blob Storage, GCP), streaming data sources, and third-party APIs.
  • Integrated Snowflake with tools like Apache Airflow, Talend, and Informatica for automated data ingestion and transformation.
  • Developed Python and SQL scripts to automate data ingestion, transformation, and loading processes within Snowflake.
  • Provided clean and structured data from Snowflake to business intelligence tools like Power BI for visualization and reporting purposes.
  • Leveraged Snowpipe and COPY command to efficiently load large datasets into Snowflake with minimal downtime.
  • Scripting mediated by Kafka topics.
  • Extensive experience in importing and exporting the data using stream processing platforms like Flume and Kafka
  • Developed, managed, and optimized complex ETL pipelines using Apache Airflow to automate data workflows.
  • Monitored the execution of DAGs in Airflow, ensuring timely and successful data pipeline execution.
  • Optimized Airflow DAGs to improve execution times and reduce resource consumption.
  • Integrated data from various sources (e.g., databases, APIs, flat files) into a centralized data warehouse using Airflow.
  • Ticketing system experience in Jira platform
  • Have extensive knowledge of Agile/scrum practices

Software Developer

Valley Infosystems
08.2019 - 08.2022
  • Using AWS Redshift, extracted, transformed, and loaded data from various heterogeneous data sources and destinations.
  • Performed data analysis and design, and created and maintained large, complex logical and physical data models, and metadata repositories using ERWIN and MB MDR.
  • Designed and managed databases using SQL or NoSQL technologies like MySQL, PostgreSQL.
  • Integrated third-party APIs and services into the application.
  • Conducted unit testing, integration testing, and automated testing to ensure software reliability.
  • Worked on the tuning of SQL Queries to bring down run time by working on Indexes and Execution Plans.
  • Automated build processes and implemented continuous integration practices.
  • Improved software efficiency by troubleshooting and resolving coding issues.
  • Saved time and resources by identifying and fixing bugs before product deployment.
  • Has an extensive experience in power bi and created visually impact dashboards in Excel and power bi for data reporting.
  • Collaborated with cross-functional teams to deliver high-quality products on tight deadlines.
  • Enhanced user experience through designing and implementing user-friendly interfaces.
  • Monitored application performance and deployed updates with minimal downtime.
  • Works with company management to ensure mission-critical reporting and business intelligence reporting are being developed and delivered.
  • .Achieved faster development cycles using Agile methodologies, including Scrum or Kanban processes.
  • Tested and deployed scalable and highly available software products.
  • Coordinated deployments of new software, feature updates and fixes.
  • Utilized caching strategies, database indexing, and other techniques to improve system performance.
  • Analyzed work to generate logic for new systems, procedures and tests.
  • Conducted data modeling, performance and integration testing.
  • Participated in Agile/Scrum meetings, including daily stand-ups, sprint planning, and retrospectives.

Education

BACHELORS DEGREE - MECHNICAL ENGINEERING

Jawaharlal Nehru Technological University
Pulivendula
05.2019

DIPLOMA DEGREE - MECHNICAL ENGINEERING

STATE BOARD OF TECHNICAL EDUCATION AND TRAINING
Nellore
04.2016

Skills

  • Agile Methodology
  • Git Version Control
  • Python Programming
  • Data Warehousing
  • Data Modeling
  • NoSQL Databases
  • Apache Flink
  • Data Pipeline Design
  • Kubernetes Deployment
  • ETL development
  • RESTful APIs
  • Spark Development
  • Data Lake Management
  • Data Migration
  • Apache Kafka
  • Stream Processing
  • Amazon Web Services
  • Big Data Analytics
  • Hadoop Ecosystem
  • MapReduce Development
  • Apache HBase
  • SQL and Databases
  • Data programming
  • Snowflake
  • Data Visualization

Certification

  • Certified Developer Basic, Moovaacademy ,Almaviva Spa.
  • Certified Architect basic,Moovaacademy ,Almaviva Spa.
  • Java Course , Udemy
  • Java script,HTML,CSS Courses, Udemy
  • Python developer , Udemy
  • SQL Developer ,Udemy

Languages

English
Advanced (C1)
Italian
Upper intermediate (B2)
Hindi
Upper intermediate (B2)
Telugu
Bilingual or Proficient (C2)

Timeline

Big Data Engineer

Almaviva Spa
09.2022 - Current

Software Developer

Valley Infosystems
08.2019 - 08.2022

BACHELORS DEGREE - MECHNICAL ENGINEERING

Jawaharlal Nehru Technological University

DIPLOMA DEGREE - MECHNICAL ENGINEERING

STATE BOARD OF TECHNICAL EDUCATION AND TRAINING

Similar Profiles

CREATE PROFILE
Harshavardhan Reddy Tanamala