Google Summer of Code 2023 Work Product Submission
Student: Soham Gupta
Organisation: VideoLAN
Project: Improve Chromecast Support (subtitles)
Introduction
Over the past 3 months, I had the opportunity to work on the VLC project for Google Summer of Code 2023. It was an invaluable learning experience that helped me grow my skills in software development and open source contribution. In this blog, I'll summarize my work and key learnings.
About the Project
π₯ The project involved enhancing the capabilities of VLC's HTTP Live Streaming (HLS) module by:
- Implementing support for H.265/HEVC video codec
- Improving performance of video casting to Chromecast
- Enabling blending of subtitles with videos
I worked primarily on the chromecast-hls
branch under the guidance of my mentor Alaric SΓ©nat and org admins Thomas Guillem and Jean-Baptiste Kempf.
Work Summary
Here are some highlights of what I was able to accomplish over the course of the program:
Transcoding Bug Fix
- π Fixed a bug in the transcode module to resolve synchronization issues when using
transcode:display
with paced outputs(UDP, HLS and display) - β Successfully closed issue #25959 through this bug fix
H.265/HEVC Video Codec Implementation
- π Researched details of implementing H.265 support
- π§ Added code to generate H.265 codec information in
codecs.c
- β Transcoded sample videos to H.265 using ffmpeg and tested with hls module
- β Faced challenges manipulating profile/level values to match H.265 codec syntax
Chromecast Performance Improvements
- π Troubleshot various connectivity issues between VLC and Chromecast device
- π· Rebuilt code frequently to address streaming problems
- πΆ Worked around WSL2 network limitations to get video casting working
- π€ Reduced lag and improved synchronization of audio/video casting
Subtitle Blending
- π¬ Successfully blended SRT subtitles with videos using soverlay parameter
- βͺ Fixed timing issues causing subtitles to be out of sync
- π Explored advanced subtitle formats like ASS, WebVTT but faced rendering issues at times
Testing Subtitle Blending (Windows Build)
SRT Format
- Used the following command to blend SRT subtitles:
./vlc.exe video.mp4 --input-slave=sub.srt --sout="chromecast{ip=[ip address]}" --demux-filter=cc_demux --no-plugins-cache -vv | cat -
ASS Format
- Used the following command to blend ASS subtitles:
./vlc.exe video.mp4 --input-slave=sub.ass --sout="chromecast{ip=[ip address]}" --demux-filter=cc_demux --no-plugins-cache -vv | cat -
WebVTT Format
- Used the following command to blend WebVTT subtitles:
./vlc.exe video.mp4 --input-slave=sub.vtt --sout="chromecast{ip=[ip address]}" --demux-filter=cc_demux --no-plugins-cache -vv | cat -
Tracing Flush sout callback
β Trace module did not log Flush events in the JSON file in the Windows build
β° Added timestamps for Flush events to enable tracing
π Traced video frames to analyze timestamp issues
Contributions
Over the course of GSoC 2023, I made the following contributions to the VLC project:
- Created 3 merge requests over the course of the program π
- Transcoding Bux Fix
- HEVC Video Codec Implementation
- Traced
Flush
events - Patches for issues like HTTPD URL deletion, GnuTLS, etc
- Clear documentation doubts through daily logs and merge request descriptions
- Responded to code reviews and made changes as per suggestions
Learning Outcomes
Through this project, I greatly improved my skills in areas like:
- π§ Tooling - gcc, clang, docker, ffmpeg
- π» Application development - VLC architecture, FFmpeg transcoding, Chromecast protocols
- Open Source Workflow - version control, build troubleshooting, submitting merge requests
- Git Version Control -->
git commit --amend
,git stash
,git rebase
,git reflog
, splitting commits, changing authors & many more in the process
- Git Version Control -->
- βοΈ Technical communication - daily logs, merge request descriptions, collaborating with mentor
Future Scope
While I was able to complete core objectives, some areas of improvement remain:
- Add support of fragmented mp4 in the HLS server
- Achieve 1:1 matching of H.265 codec profile/level values
- Reduce audio/video sync issues during Chromecast streaming
- Support text subtitle formats like WebVTT
- Improve modularisation and memory management
I'm excited to continue contributing to VLC and build on the work done during GSoC 2023.
Challenges Faced π₯
Throughout the program, I encountered various challenges that tested my problem-solving skills:
π Connectivity issues between VLC and the Chromecast device due to network limitations of WSL2. This required extensive troubleshooting.
π§ Frequent rebuild errors during cross-compilation that needed debugging of outdated contribs and prebuilt contribs.
β Manipulating the profile and level values to match the H.265 codec syntax was difficult without proper documentation.
βͺ Synchronization problems between audio and video streams during Chromecast casting. Identifying the root cause took time.
π€ Limitations on video segment duration and buffers caused intermittent casting failures.
π The initial part of videos without audio failed to render in the transcoded stream.
β Absence of logs and abrupt UI closures made it hard to pinpoint errors.
β Frequent merge conflicts and overwrite errors while rebasing branches caused delays.
By persevering through these roadblocks, I improved my debugging skills and learned how to approach problems methodically. Special thanks to my mentor for their invaluable guidance when I was stuck.
Final Weeks' Work
βοΈ Added delays using
vlc_tick_sleep
to extend video castingπ Debugged asserts and crashes during Chromecast streaming
β Got WebVTT subtitle blending working with Chromecast
π¦ Committed and pushed all code changes to respective branches which I made throughout the project timeline
Branches Worked On
Branch Overview Commits chromecast-hls Initial work on Chromecast streaming View rebased/chromecast-hls.2 Final work on Chromecast streaming View sout-hls.4 H.265 codec implementation View fix/transcode Transcoding synchronization fix View trace Added tracing module View Acknowledgments
I'm grateful to my mentor Alaric SΓ©nat for his invaluable guidance and support throughout this enriching journey.
π And a big thanks to the VideoLAN's VLC project maintainers and Google Summer of Code program for giving me this opportunity to learn and grow as a developerπ¨π»!