Boosting Database Performance using RDS Proxy

DPDzero, the go-to Collections platform for financial institutions, is making waves in the world of tech optimization. Here’s how we harnessed the power of RDS Proxy to supercharge our services and slash costs.

Understanding DPDzero

DPDzero is your trusted partner in scaling collections effortlessly for banks, fintechs, NBFCs, and MFIs. We facilitate the recovery of capital across a wide range of loan products and stages, making it easier for financial institutions to achieve their goals.

The Challenge

Our core function involves sending a massive volume of SMS, WhatsApp, and IVR messages daily, backed by substantial data. Each data record corresponds to a borrower, creating a need for efficient concurrency management. Below diagram shows DPDzero communicates with various communication channels

To achieve this, we employ a multitude of worker processes that retrieve messages from a RabbitMQ queue. Each message is dispatched via a separate Celery task, powered by the Python distributed task management system.

Our database operates on RDS, and each Celery task demands the creation of a new database connection. Additionally, we receive numerous webhooks for all the outgoing communications. During the initial days of each month, our workload spikes significantly, causing our database to become a bottleneck. Each new connection exacts a high CPU toll, and we initially tried to mitigate this by continually upgrading our machine.

However, this approach became prohibitively expensive as we quickly scaled from db.m6g.medium to db.m6g.8xlarge instances, skyrocketing our database costs.

A Lifesaver from AWS

Fortunately, AWS took notice of our situation and proactively reached out to help us cut costs. Their dedication to customer success was impressive, and we were eager to explore their recommendations.

The Solution: RDS Proxy

After consulting with AWS’s database experts, we pinpointed connection pooling as the ideal solution. Although we had initially considered using pgBouncer, we wanted to avoid increasing DevOps complexity.

Enter RDS Proxy—a fully managed connection pooling service. Once we enabled RDS Proxy, our database’s CPU usage plummeted.

As a result, we were able to downgrade our RDS PostgreSQL instance from db.m6g.8xlarge to db.m6g.4xlarge, significantly reducing our costs while maintaining exceptional performance.

Conclusion

DPDzero’s journey with RDS Proxy serves as a testament to the power of cutting-edge technology in streamlining operations and optimizing costs. Our partnership with AWS has allowed us to continue offering top-notch services to our valued clients while keeping expenses in check.

At DPDzero, we’re not just collections experts; we’re tech-savvy trailblazers. And we are hiring talented Full stack Engineers, QA professionals, Devops engineers and AI/ML engineers. Check out our careers page at https://dpdzero.com/careers/

Join us on our journey to reshape the future of collections with innovation and efficiency at its core.