SHIVAM VAISHNAV

Aspiring Data Engineer | Full Stack Developer Intern

LinkedIn | GitHub

About

Aspiring Data Engineer with a robust foundation in Python and SQL, specializing in crafting efficient data workflows that have improved processing speeds by up to 30%. Eager to leverage expertise in real-time data challenges and advanced analytics to drive impactful solutions in a dynamic tech environment.

Work Experience

Full Stack Developer Intern

Dev Information Technology Pvt. Ltd.

Feb 2025 - Aug 2025

Ahmedabad, Gujarat, IN

Currently contributing to data engineering and automation initiatives for a leading IT solutions provider, focusing on real-time data ingestion and workflow optimization.

  • Engineered and deployed real-time data ingestion pipelines using Apache NiFi, significantly reducing manual data processing by 60%.
  • Optimized complex SQL queries, enhancing data retrieval speed by 50% for critical data analysis and reporting.
  • Developed and implemented Python-based automation scripts for data workflows, minimizing manual processing effort and improving operational efficiency.
  • Managed Linux system administration tasks, including user access, system updates, and performance monitoring, ensuring stable data infrastructure.
  • Leveraged Hadoop and HDFS for robust batch processing of large-scale datasets, supporting critical data analytics initiatives.

Education

Computer Engineering

Gujarat Technological University (GTU)

8.96 CGPA

Aug 2021 - May 2025

Ahmedabad, Gujarat, IN

Science

Uma Shikshan Tirth

91% Overall Percentage

Apr 2020 - May 2021

Ahmedabad, Gujarat, IN

Courses

  • Mathematics: 93%

Science

N. V. Patel Vidya Mandir

91.60% Overall Percentage

Apr 2018 - Mar 2019

Ahmedabad, Gujarat, IN

Courses

  • Mathematics: 90%

Certificates

Software Development Internship Certificate

Dev Information Technology Ltd.

Aug 2025

PySpark - Apache Spark Programming in Python for Beginners

Udemy (Instructor: Prashant Kumar Pandey)

Aug 2025

Google Analytics Certification

Google

Aug 2025

Projects

YouTube Trending Data Pipeline

Jan 2024 - Jan 2025

Built a serverless pipeline using AWS S3, Glue, Lambda, Athena, and Step Functions to process YouTube trending data. Improved data quality by 35% using automated schema detection, ETL via Glue Crawlers, and delivered insights through QuickSight and Power BI.

S3 Auto-Upload with Email Alerts

Jan 2024 - Jan 2025

Automated S3 uploads by scanning local directories using Python (boto3 + os.walk) and saving files to timestamped subfolders. Configured Apache NiFi to send email notifications after each successful upload, enabling real-time visibility and backup reliability.

Skills

Programming Languages

  • Python
  • MySQL
  • SQL

Cloud Platforms

  • AWS S3
  • AWS EC2
  • AWS RDS
  • AWS DynamoDB
  • AWS Lambda
  • AWS IAM
  • Snowflake

Big Data Technologies

  • Databricks
  • Pyspark
  • Apache Hadoop
  • Apache Spark
  • Apache NiFi
  • MongoDB
  • HDFS

Operating Systems

  • Linux

Business Intelligence & Visualization

  • Power BI
  • QuickSight

Professional Skills

  • Communication
  • Problem-Solving
  • Time Management
  • Adaptability
  • Mentoring
  • Data Workflow Automation
  • ETL
  • Schema Detection
  • Serverless Architecture