From Squad Creation to Real-time Leaderboards: Scalable and Efficient Architecture with Apache Spark for IPL Fantasy League
As millions of cricket enthusiasts across the world gear up for the IPL season, the IPL Fantasy League is one of the most popular fan engagement activities. The official IPL website and mobile application host this fantasy game, which receives more than a lakh of registrations on the day of launch and attracts more than a million users during the middle of the season. With such massive traffic and user engagement, it is crucial to have a scalable and efficient architecture to handle the load and provide a seamless experience for the fantasy gaming users.
To provide a seamless experience for the users, we chose a serverless architecture with a combination of AWS services.
User Squad Creation:
User squad creation receives massive concurrent requests as users create squads before the 10 mins of the match starts once the playing 11 is announced. To handle these requests, we used a Node.js-based API that is deployed in Lambda, a serverless compute service provided by AWS. The data is stored in DynamoDB, a highly scalable NoSQL database provided by AWS.
Once the match starts, an AWS data pipeline gets initiated, which exports the user squad and moves it to S3, a highly scalable object storage service provided by AWS.
To compute the user points in real-time, we used Apache Spark, an open-source, distributed computing system. The Apache Spark job was executed on an EC2 instance, a scalable compute service provided by AWS. The user points were then updated back in DynamoDB.
Serving the Leaderboard:
API Gateway and Lambda, both provided by AWS, were used to serve the leaderboard and API requests. API Gateway acts as the entry point for all API requests and provides security, throttling, and monitoring. Lambda functions are used to process these requests and provide the necessary response.
The serverless architecture is highly scalable and can handle massive concurrent requests and compute real-time leaderboard with ease. The AWS services like DynamoDB, S3, API Gateway, Lambda, and EC2 instances are designed to scale up and down as traffic fluctuates.
The entire process of migrating the squad, computing the real-time leaderboard, and announcing the winners is automated. This reduces manual intervention and increases efficiency.
Cost-effective: With the serverless architecture, the cost is based on usage, making it highly cost-effective. This is because you only pay for the services that you use, and there are no upfront costs or minimum fees.
Engagement during the IPL:
During the IPL match, the engagement generally happens before the match once the team is picked. However, we introduced a feature where users can change their power player for the second innings, increasing engagement during the match. We set up a pipeline to export the updated squad data during the start of the second innings, providing real-time updates for the users.
We are thrilled to report that our serverless architecture, powered by a combination of AWS services and Apache Spark, achieved 100% uptime during the entire IPL season (2017 and 2018) , providing a seamless and uninterrupted experience for millions of users. Our team worked tirelessly to ensure that the architecture was highly scalable and efficient, capable of handling the massive traffic and user engagement that the IPL Fantasy League generates.
We are extremely proud to have been a part of this massive fan engagement activity and to have received overwhelmingly positive feedback from users who enjoyed the real-time updates, innovative features, and overall experience of the IPL Fantasy League. We look forward to continuing to innovate and enhance the Fantasy Gaming Experience for users by collaborating with more platforms providing fantasy gaming service.