Developing a Custom Load Balancer to serve traffic at scale



Load Balancing is a critical piece in the infrastructure or design of any distributed system which allows you to scale horizontally.  Load balancing supports multiple ways of distributing traffic load on multiple backend targets some of which are based on round robin or weighted round robin. It can be performed on HTTP traffic at layer 7 or TCP traffic at layer 4.

Common options for LoadBalancing are cloud native ones like Application Load Balancer (ALB) in AWS or App Gateway in Azure and the equivalents in other cloud providers. Naturally being a critical component, it comes priced expensively and with limitations around how you can configure them trying to tie a user to a cloud provider.

We can overcome this by designing a Load Balancer that is developed using an Open Source component like Envoy Proxy which can be scaled as per traffic needs where-in this is deployed on a custom machine with the specifications as per the targeted traffic expectations thus saving cost for the organisation. This makes it more flexible and scalable as per needs without getting tied up into a particular cloud provider.

About the speaker

Sandeep Bhat

Staff Engineer at Harness. Working on Go for over 5 years. Worked extensively with Load Balancing technologies in Walmart and Harness.

Want to discuss?
Post it here, our mentors will help you out.