Turning mess into a new application: the story of one checker
In this piece we will share our personal experience connected with quick search and analysis of media database.
If you work with content you probably know that files pile up at the speed of light, and it is hard to put them in use. Just imagine how much content we’ve received from our clients for more than 30 years being developers of software products for working with different media content. This is a huge database (DB) with all kinds of video, audio, and media container formats.
Our engineers usually use samples during development or testing processes.
It would be easier if they were able to find necessary samples in the DB on-the-fly.
However, the storage was a real mess — TS as MP4s, not valid vs. valid files, video and audio formats, etc. Finally, Elecard engineers wrote their own scripts to search for particular files.
Of course, there were some existing tools, but we’ve decided to follow an unbeaten track. Engineers identified a common architecture: they deployed a “checker probe” in the network and anyone in the office network could manage it via the web interface.
First, our engineers created an application — checker — that sorted out the files by containers, and video/audio codecs. That helped us clear out the entire DB.
Then, we added additional functionality based on internal QA requirements.
The new task came out quickly, engineers selected 4k HEVC files for live hevc decoder test performance. With the next iteration, we added a feature to check resolution, as a result we got a list of 4k HEVC files in the TS container.
Then we faced two challenges:
1. Decoder performance depended on the video bitrate — live HEVC decoder consumed more resources while decoding files with high bitrates. Thus, we added a bitrate checker feature to sort and get files with high bitrates only.
2. Only files with a CBR rate control went under tests because VBR files can’t be broadcasted accurately, which would add additional circumstances.
As an app turned out so much useful, we decided to use it not only for test samples searching and clearing out the media DB, but also for debugging. It helped us when the packetizer wrote incomplete HLS chunks after the transcoder. Engineers ran a task for searching all chunks with no audio. The probe detected all the problem files. The pattern allowed engineers to find out the root of the problem quickly.
The tool is now our internal lifesaver, and our idea is to develop it into a full-fledged solution for quick search and analysis of media DB.
We would appreciate the feedback to improve our software and to better understand market requirements. Feel free to leave comments or contact us!