Joao Edivirges
São Paulo
Summary
Diligent Data Engineer Lead with a strong track record in leading data engineering teams. Skilled in designing scalable data pipelines and overseeing intricate ETL processes. Proven expertise in utilizing SQL and Python to optimize data workflows while also providing guidance to junior engineers.
Overview
7
7
years of professional experience
Work History
Data Engineering Lead
Novo Nordisk
05.2024 - 12.2024
- Reduced sales databases downtime by proactively identifying potential issues through pharmaceutical auditions and quality monitoring, leading to prompt resolution on materialized views.
- Championed the use of Snowflake and AWS tools within the organization, leading to solve business modeling problems among the business units.
- Collaborated with cross-functional international projects(Latam and India)
- Automated repetitive tasks using Snowpark scripts, saving time and resources for higher-priority projects.
- Product Owner
Data Engineering Lead
Novo Nordisk
05.2024 - 12.2024
- Reduced sales databases downtime by proactively identifying potential issues through pharmaceutical auditions and quality monitoring, leading to prompt resolution on materialized views.
- Championed the use of Snowflake and AWS tools within the organization, leading to solve business modeling problems among the business units.
- Collaborated with cross-functional international projects(Latam and India)
- Automated repetitive tasks using Snowpark scripts, saving time and resources for higher-priority projects.
- Product Owner
Data Engineering Lead
Novo Nordisk
05.2024 - 12.2024
- Reduced sales databases downtime by proactively identifying potential issues through pharmaceutical auditions and quality monitoring, leading to prompt resolution on materialized views.
- Championed the use of Snowflake and AWS tools within the organization, leading to solve business modeling problems among the business units.
- Collaborated with cross-functional international projects(Latam and India)
- Automated repetitive tasks using Snowpark scripts, saving time and resources for higher-priority projects.
- Product Owner
Data Integration Specialist
Amicci
01.2023 - 11.2023
- Coordinated data ingestion operations from the company's data sources into Azure Data Lake
- Supported demand for infrastructure by delivering automated deploys for Lakehouse data ingestion
- Balanced data access requirements to the company's cloud resources against cost, security, and structural constraints
- Pioneered the implementation of multi cloud-based storage solutions for scalable growth when handling large volumes of integrated datasets(GPC, Azure and AWS)
Senior Data Engineer
Remessa Online
06.2022 - 11.2022
- Created and maintained data pipelines for stakeholders and consumers Metabase
- Implemented abstractions and packages to make easier the Lakehouse development environment
- Enhanced data systems continuously by working closely with data and analytics experts on greater functionality, like deploy agents, autoloaders, etc
- Collaborated in open-source projects, absorbing some of them into the company's data ingestion strategy
Senior Data Engineer
Picpay
08.2021 - 05.2022
- As a developer, the main tasks were to provide Customer Experience and Operations data, capture from the company's data lake and load it into the curated layer
- Designed and implemented a layered architecture microservice for the bank customer satisfaction forms and submits the answers from the campaign webview, bringing more veracity to the company's score, from 1k reviews monthly up to 10k/weekly subscriptions.
Data Engineer
Social Miner
11.2020 - 06.2021
- Maintained data warehouse systems with a focus on ingestion throughout, mostly with Mongo DB and Cassandra for the non-transactional data
- Developed orchestrated pipelines with Airflow and Pyspark and MySQL on the relational layer
- Supported monitoring BI operations with Kibana
Data Engineer
Semantix
11.2019 - 07.2020
- Developed pipelines using Python and AWS to capture and load into company's Lake
- Tested and monitor data pipelines using SonarQ
- First experience with CI/CD (Jenkins) and virtualization
- Worked with the company's legal team to create a semantic layer for the internal products
- Utilized advanced statistical techniques for more accurate forecasting of judge-made law sentences
Data Analyst Trainee
BIBOP
06.2018 - 09.2019
- Responsible for micro-task deliveries and management, using kanban framework
- Quality API data control and Q&A consultant for customer integrations that capture and load information from Brazil's law system
- Trainee with Postgres and Mongo administration
Education
Associate of Applied Science - Big Data Analytics
FIAP
Sao Paulo01-2020
Skills
- Data pipeline design
- Kafka streaming
- Python programming
- Spark development
- ETL development
- Big data processing
- Data security
- Infrastructure as Code
- Data modeling
- Performance tuning
- API development
- Distributed systems
Timeline
Data Engineering Lead
Novo Nordisk
05.2024 - 12.2024
Data Engineering Lead
Novo Nordisk
05.2024 - 12.2024
Data Engineering Lead
Novo Nordisk
05.2024 - 12.2024
Data Integration Specialist
Amicci
01.2023 - 11.2023
Senior Data Engineer
Remessa Online
06.2022 - 11.2022
Senior Data Engineer
Picpay
08.2021 - 05.2022
Data Engineer
Social Miner
11.2020 - 06.2021
Data Engineer
Semantix
11.2019 - 07.2020
Data Analyst Trainee
BIBOP
06.2018 - 09.2019
Associate of Applied Science - Big Data Analytics
FIAP
Similar Profiles
Elizabeth H. SmithElizabeth H. Smith
Diabetes Educator at Novo NordiskDiabetes Educator at Novo Nordisk
TRISH NOBLETRISH NOBLE
Executive Diabetes Care Specialist at Novo NordiskExecutive Diabetes Care Specialist at Novo Nordisk
Elaine TakahashiElaine Takahashi
Global Market Access Director (secondment) at Novo NordiskGlobal Market Access Director (secondment) at Novo Nordisk