eBay Network ETL

The Problem

eBay's network engineering team needed to manage a deep, complex hierarchy of networking devices across data centers. The existing manual processes were error-prone and couldn't keep pace with infrastructure growth, leading to blind spots in network visibility.

The Solution

Designed and built a custom ETL solution in Go that ingests, normalizes, and maintains the full hierarchy of networking devices. The pipeline handles complex parent-child relationships, detects topology changes, and feeds a notification system that alerts network engineers to relevant updates. Deployed on AWS with Docker.

Results

• Production deployment managing thousands of network devices
• Automated hierarchy tracking with change detection
• User-friendly notification system for network engineers
• Reduced manual tracking effort from hours to zero

Technologies