Jump to content

Photo

Fix Lip Sync


  • Please log in to reply
426 replies to this topic

#1
Site Bot

Site Bot

    Fear me

  • Members
  • PipPipPipPipPip
  • 2,248,079 posts
Fix Lip Sync

SKSE plugin that fixes desynchronization of sound and lip movement on NPCs.

#2
Darkangelmn

Darkangelmn

    Fan

  • Supporter
  • PipPipPip
  • 434 posts
This is unexpected, I have to try this...

Earlier methods were just ploppin' default meshes from the BSA and back put them in the data folder, that seemed to work for some people or it was placebo at best.

The de-sync happens whenever the game can't get target FPS when using FPSclamp in the .ini (which I nor does S.T.E.P guide recommend using) or just when generally there's a lot going on in the engine at one time.

Edited by Darkangelmn, 28 May 2016 - 07:37 PM.


#3
meh321

meh321

    Faithful poster

  • Premium Member
  • 2,478 posts
In response to post #38594785.


Spoiler

lintonthepenguin had a better explanation. Perhaps he will post it here.

To be honest I don't even understand it myself. :D I just got lucky with this one.

It has something to do with negative keyframes in the facial animation, they were not played at the right time.

#4
lasurarkinshade

lasurarkinshade

    Enthusiast

  • Premium Member
  • 202 posts
In response to post #38594785. #38595095 is also a reply to the same post.


Spoiler

That isn't actually the cause, as far as Meh and I were able to gather.

The lip sync bug is due to a faulty Patch 1.9 optimisation that caused an (intentional) audio/lip sync playback delay feature that had been in the game since launch to malfunction. Some lines of dialogue have always been subject to an audio and lip sync playback delay designed to ensure that the FaceFX (lip sync middleware) integration has time to properly morph the face so it can keep up with the start of the audio file. Pre-Patch 1.9 this worked fine, and the delay on the lip sync was the same as the delay on the audio. Post-Patch 1.9, the lip sync delay became around 2x the length of the audio delay, causing the infamous desync.

While I'm not entirely sure of the specifics of how Meh's code changes actually work, they basically prevent the delay from being multiplied 2x, thus ensuring that lip sync delay remains proportional to the audio delay and preventing the desync. It basically restores the behaviour to how it was before the 1.9 Patch that introduced the bug.

The bug was never due to issues with resource load, as evidenced by the fact that the same lines (those that trigger the FaceFX delay feature) displayed the bug consistently. It wasn't random or situational, the bug was 100% reproducible on each line, each time, to the same degree.

In case you're curious, the reason why some lines displayed the bug moreso than others is due to the fact that the issue was essentially caused by the aforementioned 2x multiplication of the delay. Those lines which only had a short delay instituted by the FaceFX delay feature obviously don't display as much desync when multiplied by 2 as those that have a longer, more drastic FaceFX delay.

And the reason why some lines never displayed the bug? Those lines weren't targeted by the FaceFX negative keyframe delay feature since the engine didn't need to delay their playback in order to have time to properly morph the face.

Edited by lintonthepenguin, 28 May 2016 - 07:51 PM.


#5

    Resident poster

  • Account closed
  • PipPipPipPipPip
  • 3,173 posts
Mr. meh321!!!
Dude you need some kind of award...
You have my deep thanks!
Cheers on fixing this great game left broken by beth!

#6
Darkangelmn

Darkangelmn

    Fan

  • Supporter
  • PipPipPip
  • 434 posts
Yeah, it's once again modders picking up the torch after Beth just left the game in such an un-immersive state.

#7
cloudedtruth

cloudedtruth

    Old hand

  • Premium Member
  • 820 posts
Thank you! I was constantly having to re-generate LIP files in an attempt to fix this problem for many lines of dialogue I was creating. Now, I don't have to :D
I tried this out on known problematic dialogue and it works perfectly. Thank you very much for this!

Edited by cloudedtruth, 28 May 2016 - 07:57 PM.


#8
Darkangelmn

Darkangelmn

    Fan

  • Supporter
  • PipPipPip
  • 434 posts
In response to post #38594785. #38595095, #38595245, #38595875 are all replies on the same post.


Spoiler

Thanks for the in-depth explanation. And after testing for a few minutes, the patch works wonders, although for the current follower that I have with me (Sofia), sometimes I perceive her lines being said slightly before the lips open.

That's maybe because I got so used to the bug, it makes me think there's something wrong now! :)

It just so happens that I tried the other lip sync fix yesterday and it didn't work at all. (I wasn't looking for a fix to this since Skyrim launched.) I stopped testing at the point where my game CTD'd twice (not related to this mod), I posted the results in CF thread. Kinda lucky this comes out today.

You and the guys in the credits deserve a beer.

Holy balls though, has anyone ever tested how many .dlls can SKSE handle in a stable way? I'm now running 19 .dlls in total. I have always wondered about this and there's yet any comments from anyone how much stress is put on the system by having a load of .dlls that use SKSE's loader.

Edited by Darkangelmn, 28 May 2016 - 08:12 PM.


#9
Bosmeroya

Bosmeroya

    Enthusiast

  • Premium Member
  • 171 posts
If this actually works.. You deserve a nobel prize.

#10
PowerFactor80

PowerFactor80

    Journeyman

  • Premium Member
  • 40 posts
You are in danger of achieving "wizard" or "wise sage" status.

Awesome.




IPB skins by Skinbox
Page loaded in: 1.087 seconds