Our Client needed to build a Program Guide Services infrastructure, which would collect and store all Broadcast and On Demand Content metadata and provide advanced query and search features to request data with a versatile API. The system architecture would require efficiently storing and tracking changes of voluminous amounts of data like Programs, Videos, Channels, Scheduling Information, Images, Posters, Reviews and Celebrity Information. Despite the amount of data and complex relations among objects, the system provides low latency high TPS guarantees, load distribution, geographical high availability and fault tolerance.
Large TV Providers aggregate copious amounts of data related to Video Broadcast, Video On Demand and Video Distribution services. Video Content itself cannot be imagined now without accompanying information like time when it is broadcast, which channel it is on, its episode title, duration, parental restrictions and ratings, recommendations and cast crews, trailers, etc. This information becomes even more important for large Video Libraries to search for a particular movie title or category.
Traditionally TV Providers publish the Program Guide for two weeks into the future, and when a day ends, all information expires and becomes unavailable. There are many business cases when historical metadata is useful, including analytical processing, technical attributes modifications tracking, incorrect or invalid data audit, access to old video data from existing applications and others.
IntroPro was selected as the delivery team for Architectural Design, Implementation and Quality Assurance due to our experience designing and building Distributed Systems with advanced algorithmic complexity and low latency high throughput requirements. The proposed architecture heavily relied on In-Memory Data Grid capabilities to provide caching and low latency query and search processing layer deployed at Point of Presence geographical locations whereas the core storage utilized Relational Database with custom written DAO for efficient and optimized query execution.
The solution included Free Text Search functionality which was designed and implemented on top of Distributed Grid utilizing Map-Reduce processing, Distributed Functions execution, Replicated and Partitioned Caches joining operations, Cluster update notifications and other advanced features.
- Gemfire Distributed Grid
- Oracle Database
- Spring Framework
- REST Web Services (JSON, XML)
- Apache Cassandra (D-Node prototype)
- VerticaDB (D-Node prototype)
- Scala (utility tools and prototypes)
- Extensive experience with In-Memory Data Grids
- Track record of Distributed System Design and Implementation
- Business Domain knowledge
- Robust development process
- Quality assurance for functional requirements, data consistency, load and performance via automation
- Automated Performance, Stress and Functional Full System testing
- Continuous Integration for constant tracking of project quality
- "Product style" delivery including Software Quality Assurance phase
Yuriy Korotyshev, Senior Project Manager