Fighting WebRTC Audio Sync Issues
Subtitle:  Mainly with Chrome

Audio Sync issues are when the audio and video get out of “lip-sync.”  The sound does not line up with what you see.  Many folks call it a “lag.”  (We also have heard people call it a “latency” problem, but that is an incorrect description.)  This problem can happen with any broadcasting software, but it appears to happen more often on Chrome streaming with WebRTC.  This does not mean, “Firefox is better than Chrome,” or any one browser is superior.  We are only saying this one specific problem happens to a few broadcasters while on Chrome.

The guts of how WebRTC streams, also known as the encoder, processes the audio and video separately. They are streamed separately, they travel across the internet separately, and it is your computer at home that has to put it back together in-sync.  When a moment in time of the audio and video are sent across the internet too far apart from one another, the result is audio that does not match video.  That’s just the way it works!

Processing audio is easy from a computer’s point of view, there is less data to manage.  Video, on the other hand, takes massive amount of data and processing to produce a video signal.  The CPU (central processing unit) of the computer, a.k.a. the brain, can only handle so many calculations at one time.  When there is too much data to process when streaming, it is generally the video that gets processed slower than the audio.

The result: the audio is encoded first, and the video lags behind.  Look closely at a sync problem broadcast – you will almost always hear the sound / voice first, then witness the video catch up.

How to fix this problem?  Start by understanding that anything that can free up your computer’s CPU from tasks, or reduce the amount of video data to process, will help:

• Make sure your browser is running the latest version. We will say this a 1,000 times… it’s very important with WebRTC to be on the latest version because the browser makers release significant changes with each new release.  To check / update Firefox or Chrome, go to Help >> About.
• Turn off any software running on your computer you do not need for streaming.  Anything to free up your CPU will help.  Close multiple browser windows and tabs, close music recording software, close any games and video intensive software, and so on.  And close dang Facebook!
• Choose 4:3 SD (standard def) streaming on StreetJelly.  Obviously, 720p HD takes way more CPU cycles to stream.
• Adjust your webcam to the lowest decent quality.  Your webcam may be set at very high HD settings. The encoder then has to convert that video to lower quality for streaming.  That takes a lot of CPU power to do that.  In the hardware settings for your camera (not on StreetJelly):
  • Match the screen size of the webcam to the screen size chosen for broadcasting on StreetJelly.  For example: 640 x 480
  • Set the frame rate (if available) to 15fps, frames per second.  The WebRTC encoder defaults to 15fps, so it will get converted anyway.
  • Set quality (if available) to medium, or somewhere around 80%.  Different webcam vendors approach this setting differently, but the key is to dial it off the very top, but don’t go all the way to the bottom.
• Remove any webcam effects from your broadcast.  This includes extra graphics embedded into the video (snowflakes, alien heads, swirly do-dahs, etc). All these require massive CPU processing to generate.
• Remove any lighting effects in your home studio.  Yes, we mean swirly lights in your room in the real world.  This affects compression. The more areas within your video frame that do not change over time, the more your video gets compressed into smaller data chunks.  This reduces the amount of bandwidth needed to stream and process.
• In your computer’s video settings (at the hardware level):  make sure any “hardware acceleration” settings are turned on.  I’m sure that sounds vague, but there are far too many video card drivers, manufacturers, scenarios, etc. to write about in a single blog.  The key here is to maximize the CPU and GPU (graphics processing unit) settings.
• Get a faster computer! 🙂

UPDATE APRIL 19th, 2017

So we know Chrome can be a CPU hog, and these issues can get the audio out of sync.  One thing to try – and definitely check – is the Chrome “use hardware acceleration” setting.  Hardware Acceleration is a feature on most computers to use the GPU (graphics processing unit) for video processing. In Chrome, this can be controlled with an Advanced Setting called:  “Use hardware acceleration when available.”

Normally, this setting should be checked.  Check it now with these instructions:  http://www.pcadvisor.co.uk/how-to/internet/how-turn-off-gpu-hardware-acceleration-in-google-chrome-3605455/

For some with audio sync issues, UNCHECKING this setting may help.  It’s not a guarantee, but worth a try.  Sometimes a computer’s video card drivers could be out of whack (I literally had that problem with an old Lenevo laptop).  Using software to process the video software can possibly fix (or avoid) a hardware issue.

For a bit more detail about Chrome and Hardware Acceleration, here’s another nice article:  https://www.lifewire.com/hardware-acceleration-in-chrome-4125122