音视频同步-主要代码逻辑

也就是你追我敢的过程

因为音频格式sample channel rate 都是固定的 所以 某一段音频时间是可以确定的，所以一音频为基准 视频同步到音频

常规做法： 展示一帧视频帧后，得到下一帧视频帧的PTS, 和当前音频的时钟锁一个比较，如果在之前那就立即播放，如果在之后那就设置一个delay ,然后设置进定时器，当定时器超时后，刷新新的视频帧。

 av_frame_get_best_effort_timestamp(pFrame)) 可以直接获得一个对应AvFrame 中获得一个合适pts
 pts转成秒需要乘以time_base 
 然后对pts做一定计算 如果可以用就直接 is->video_clock = pts 就行

音视频同步主要具体代码逻辑

在主要结构体里面增加5个参数
audio_clock 当前音频帧时间
video_clock 当前视频帧的下一帧的时间
frame_timer 下次要回调的timer
frame_last_pts 视频上一帧的pts
frame_last_delay 上一帧视频播放的delay

  double          audio_clock;
  double          video_clock; ///<pts of last decoded frame / predicted pts of next decoded frame

  double          frame_timer;  //视频帧回调的时间
  double          frame_last_pts;
  double          frame_last_delay;

简单逻辑就是！
1 获取 当前真和前面一帧之间的 delay

delay = vp->pts - is->frame_last_pts 

2判断delay 并且代替前一步

delay = vp->pts - is->frame_last_pts; /* the pts from last time */
  if(delay <= 0 || delay >= 1.0) {
	delay = is->frame_last_delay;
  }
  is->frame_last_delay = delay;
  is->frame_last_pts = vp->pts;

3 获得当前音频帧的PTS 和 正在播放帧的下一帧（也就是当前处理帧）计算他们之间的diff

ref_clock = get_audio_clock(is);
  diff = vp->pts - ref_clock;

4 sync_threshold 不能让帧小于音频的最小帧率
if(fabs(diff) < AV_NOSYNC_THRESHOLD) 如果前面得到的距离 满足最小帧处理要求那就同步处理
如果diff <= -sync_threshold 那么说明当前帧时间以及过了 delay 设置0 快点播放
如果 diff >= sync_threshold 说明距离播放还有一段时间吧delay 增长

 sync_threshold = (delay > AV_SYNC_THRESHOLD) ? delay : AV_SYNC_THRESHOLD;
  if(fabs(diff) < AV_NOSYNC_THRESHOLD) {
	if(diff <= -sync_threshold) {
  delay = 0;
	} else if(diff >= sync_threshold) {
	  delay = 2 * delay;
	}

5 在当前回调时间基础上 加上上面计划好的delay时间

is->frame_timer += delay;

6 计算实际时间中的delay 保证最小值不低于音频帧的帧率
最后传入实际需要时间 去刷新 视频渲染 +0.5是加一个偏执

actual_delay = is->frame_timer - (av_gettime() / 1000000.0);
  if(actual_delay < 0.010) {
/* Really it should skip the picture instead */
actual_delay = 0.010;
  }
  schedule_refresh(is, (int)(actual_delay * 1000 + 0.5));

void video_refresh_timer(void *userdata) {

  VideoState *is = (VideoState *)userdata;
  VideoPicture *vp;
  double actual_delay, delay, sync_threshold, ref_clock, diff;
  
  if(is->video_st) {
    if(is->pictq_size == 0) {
      schedule_refresh(is, 1);
    } else {
      vp = &is->pictq[is->pictq_rindex];

      delay = vp->pts - is->frame_last_pts; /* the pts from last time */
      if(delay <= 0 || delay >= 1.0) {
	/* if incorrect delay, use previous one */
	delay = is->frame_last_delay;
      }
      /* save for next time */
      is->frame_last_delay = delay;
      is->frame_last_pts = vp->pts;

      /* update delay to sync to audio */
      ref_clock = get_audio_clock(is);
      diff = vp->pts - ref_clock;

      /* Skip or repeat the frame. Take delay into account
	 FFPlay still doesn't "know if this is the best guess." */
      sync_threshold = (delay > AV_SYNC_THRESHOLD) ? delay : AV_SYNC_THRESHOLD;
      if(fabs(diff) < AV_NOSYNC_THRESHOLD) {
	if(diff <= -sync_threshold) {
	  delay = 0;
	} else if(diff >= sync_threshold) {
	  delay = 2 * delay;
	}
      }
      is->frame_timer += delay;
      /* computer the REAL delay */
      actual_delay = is->frame_timer - (av_gettime() / 1000000.0);
      if(actual_delay < 0.010) {
	/* Really it should skip the picture instead */
	actual_delay = 0.010;
      }
      schedule_refresh(is, (int)(actual_delay * 1000 + 0.5));
      
      /* show the picture! */
      video_display(is);
      
      /* update queue for next picture! */
      if(++is->pictq_rindex == VIDEO_PICTURE_QUEUE_SIZE) {
	is->pictq_rindex = 0;
      }
      SDL_LockMutex(is->pictq_mutex);
      is->pictq_size--;
      SDL_CondSignal(is->pictq_cond);
      SDL_UnlockMutex(is->pictq_mutex);
    }
  } else {
    schedule_refresh(is, 100);
  }
}