Mukkanteswara

Work with on-site and internal teams to gather data requirements for a project that evaluates promotions and offers effectiveness for a major Canadian supermarket chain. Design and develop data pipelines to cater to the same and transform the legacy data into a Standard Data Model.

• Participated in TDD(technical design document) and helped scope out the project.
• Worked on EAD(Efforts Assessment Document).
• Worked as a single point of contact for implementing end-to-end data pipelines.
• Implemented data pipelines using various GCP big data services DataProc (Spark), DataFlow (Apache Beam), BigQuery, Cloud Function, etc.
• Implemented cloud functions using Python for customization.

Technologies:  TDD, EAD, GCP, Spark, Apache Beam, BigQuery, Cloud Function, Python, API, KMS, Airflow, SourceRepositories

Work as a Senior Data Engineer to implement the eSentire AWS Anomaly Detection Data platform. eSentire is a Managed Detection and Response (MDR) service provider, that keeps organizations safe from constantly evolving cyber-attacks.

• Gathered requirements and provided design documents.
• Provided estimates for different data pipelines.
• Implemented Kinesis Firehose set up to ingest real-time events to S3 buckets.
• Implemented Glue flows to ingest data from relational databases.
• Implemented business transformations using Glue.
• Implemented Glue flow to ingest transformed data to Redshift tables.
• Validated the business logic by performing descriptive data analysis.
• Involved in daily calls with the team to track the progress of the project.
• Involved in post-production support and documentation.

Technologies:  AWS, Kinesis Firehose, S3, Glue, Redshift

Work with internal and client team members to implement the Azure Analytical Data platform. AEG was an American worldwide sporting and music entertainment presenter who wanted to leverage analytics with Azure Platform for insights about customers, ticket sales, marketing, etc.

• Ensured code integrity (by unit and integration testing) and code version control using GIT and supported production deployment of code with appropriate documentation.
• Implemented data ingestion to load data from Azure Data lake to Staging Tables.
• Implemented data ingestion layer with NIFI data flows
• Implemented various business transformations with Azure Databricks Spark cluster and loaded to Azure Warehouse.
• Implemented UDFs in Python for custom-specific logic
• Implemented an in-house analytical platform for automating data pipelines.

Technologies: Git, Azure Data Lakes, NIFI, Spark, Data Bricks, Python, Airflow

Led a team of 3 developers at the client location to help deliver near-real-time data for a dynamic dashboard used by Rogers’ senior management to get an instant insight into the overall status of wireless and cable operations, network stability, department-wise sales targets, and customer query categories so they could make proactive, preemptive decisions.

• Assisted in sprint planning, provided effort estimates
• Developed data processing framework using Spark(Python), Hive, and Unix shell scripting to load data into Hadoop Data Lake.
• Developed a rule-based data quality framework using Pyspark to evaluate the KPI conformance of data loaded into the Data Lake that enabled the production support team to proactively identify data issues and make available a seamless service to the Senior management.
• Implemented Batch and near real-time pipelines using Pyspark.
• Implemented Python UDFs for customized code.
• Implemented Complex Spark SQLs for the end user data analysis.
• Documented implementation details to hand over to support teams.
• Involved in KT sessions with different stakeholders in the project.

Technologies:  PySpark, Python, Batch, SQL, Data Lakes, Hadoop Data Lakes, Spark, Hive, Unix Shell

Led a team of 5 members to deliver AML(Anti Money Laundering) use cases. Hadoop Analytical Platform was built to capture various transactions into the system and validates the data by running various scenarios (like a customer cannot deposit more than 10K dollars per day into his account etc.) to find the fraudulent transactions. If any of the transactions were faulty, an alert would be generated and the concerned team would start looking into the details of the transaction and see whether it was a legitimate transaction or would file a case against the customer.

• Worked on real-time data streams by integrating Kafka and Spark
• Worked on Hbase and Cassandra NoSQL databases to load huge amounts of data.
• Developed Spark SQLS to analyze data
• Worked on optimization like partitioning, bucketing, and concepts in Hive.
• Prepared unit test cases and validated results.
• Participated in Prod migration activities and extended support till the warranty period.

Technologies:  Kafka, Spark, Hbase, Cassandra NoSQL, SQL, Bucketing, Hive, Hadoop Analytical Platform

Worked as a big data engineer to build data pipelines for CCAR applications. The Comprehensive Capital Analysis and Review (CCAR) is an annual exercise by the Federal Reserve to ensure that institutions have well-defined and forward-looking capital planning processes that account for their unique risks and sufficient capital to continue operations through times of economic and financial stress. Particular users for this project are the U.S Federal Reserve body and the internal Risk management compliance team.

• Involved in complete SDLC – Requirement Analysis, Development, System Integration Testing, Performance Testing, code promotion, and deployment CI/CD and DevOps.
• Created several Pyspark jobs loading data from CSV files and Oracle server.
• Involved in creating Hive-managed and external tables, and then applied HiveQL on those tables for data validation.
• Developed PySpark HiveQL code to integrate with Hive for analyzing Historical data using In-memory computing to speed up report generation.
• Worked on code check-ins in the GIT repository.
• Support given during SIT and UAT cycles. Fixed issues reported by the QA team.
• Participated in Prod migration activities and extended support till the warranty period.

Technologies:  DevOps, CI/CD, PySpark, Oracle, HiveQL, Git, QA

Worked as an ETL Developer to build ETL flows for the Customer Behavior Discovery Analytical Platform. Customer Discovery Platform is Vodafone’s customer analytics platform to analyze customer data and make decisions. Vodafone drives some analytics like new activations, deactivations, customer usage based on geographical locations, different promotions, various plans, etc.

• Designed ETL process flows for the entire DWH application and developed data mapping spreadsheets to define transformation rules for each stage of the ETL process.
• Developed Informatica ETL code using various mappings and transformations for the transport of data from legacy extract files to data mart as per the business requirements.
• Responsible for migration of Informatica mappings, and workflows between different environments.
• Developed robust Informatica mappings and fine-tuned them to process millions of put records with estimated throughput.
• Development of Bteq / FastLoad / Multiload script for loading purposes.
• Prepared the technical design document and estimated the efforts required for the build.
• Prepared Unit Test Case scenarios and verified the results.
• Participated in PROD migration activities and extended support till the warranty  phase

Technologies:  ETL, Informatica, Bteq, FastLoad, Unit Testing

Worked as an ETL Developer to build ETL flows for Marketing the  Campaign management Analytical Platform. It was developed to analyze the effectiveness of the marketing campaigns for the incentives and offerings offered by Vodafone. The ETL process involved extracting the data from SQL Server and Oracle sources and loading them into Teradata Data Warehouse. The data was used to analyze customer opinions, loyalty, customer profiles, and customer satisfaction.

• Created Informatica workflows and Bteq scripts to move the data from source to Raw, Stage, and finally Target.
• Created appropriate indexes depending on the table situation and requirement.
• Did performance checks and tuning opportunities like a partition on table level, compression, joining indexes and collecting statistics, etc.
• Prepared Unit Test Case scenarios and verified the results.
• Involved in Development support during the deployment process and post-production support as well.

Technologies:  ETL, SQL Server, Oracle, Informatica, Bteq, Teradata