DEV Community

Karan Pulyala profile picture

Karan Pulyala

Certified data engineer (Azure & AWS). I write about Azure Databricks, PySpark and Spark performance. Interview prep packs at pulyala.gumroad.com

Location Greater Toronto Area, Canada Joined Joined on  Personal website https://pulyala.gumroad.com

Work

Data Engineer

PySpark Window Functions on Databricks: Latest-Record Dedup, Running Totals, Sessionisation, and the Frame Traps
karanpulyala profile

PySpark Window Functions on Databricks: Latest-Record Dedup, Running Totals, Sessionisation, and the Frame Traps

#spark #databricks #pyspark #dataengineering
Comments
5 min read

Want to connect with Karan Pulyala?

Create an account to connect with Karan Pulyala. You can also sign in below to proceed if you already have an account.

Create Account
Already have an account? Sign in
Spark Join Strategies on Databricks: Broadcast, Sort-Merge, Shuffle Hash, and What AQE Changes
karanpulyala profile

Spark Join Strategies on Databricks: Broadcast, Sort-Merge, Shuffle Hash, and What AQE Changes

#databricks #pyspark #spark #dataengineering
Comments
6 min read
Delta Lake Table Maintenance on Databricks: OPTIMIZE, Z-ORDER vs Liquid Clustering, and VACUUM
karanpulyala profile

Delta Lake Table Maintenance on Databricks: OPTIMIZE, Z-ORDER vs Liquid Clustering, and VACUUM

#databricks #deltalake #dataengineering #pyspark
Comments
6 min read
Auto Loader and Structured Streaming on Azure Databricks: Checkpoints, Schema Evolution, and Exactly-Once
karanpulyala profile

Auto Loader and Structured Streaming on Azure Databricks: Checkpoints, Schema Evolution, and Exactly-Once

#databricks #pyspark #dataengineering #azure
Comments
6 min read
Delta Lake MERGE in PySpark: Upserts, SCD Type 2, and the Traps Interviewers Ask About
karanpulyala profile

Delta Lake MERGE in PySpark: Upserts, SCD Type 2, and the Traps Interviewers Ask About

#databricks #pyspark #dataengineering #deltalake
2
Comments
5 min read
Diagnosing and Fixing Data Skew in PySpark on Azure Databricks (2026 Edition)
karanpulyala profile

Diagnosing and Fixing Data Skew in PySpark on Azure Databricks (2026 Edition)

#databricks #pyspark #dataengineering #spark
Comments
8 min read
loading...