Gstreamer Time & Sync

GstClock

Frist some introduce words:

absolute_time: the current time get from GstClock (monotonically increasing time)

running_time:the total time spent in the PLAYING state

base_time:defined as the absolute_time minus the running_time at the time when the pipeline is set to PLAYING

stream_time:the position in the stream(between 0 and the total duration)

GStreamer can use different clocks:

• system time (with g_get_current_time() and with microsecond accuracy)
  
• soundcards and other devices time(better, use get_clock in some elements to get a clock)
  
• a network source based on packets received + timestamps in those packets (a typical example is an RTP source)
  

time is always expessed in nanoseconds, so I think there must have a conversion between different time types.

pipeline state and running_time:

• NULL/READY, the running_time is undefined
  
• PAUSED, the running_time remains at the time when it was last PAUSED
  
• PLAYING, the running_time is the delta between the absolute_time and the base time
  
• after a flushing seek, the running_time is set to 0(redistribute a new base_time to implement)
  
  running_time = absolute_time - base_time(In the PLAYING state)
  

GstBuffer

The GstBuffer timestamps and the preceeding NEW_SEGMENT event define a transformation of the buffer timestamps to running_time

B: GstBuffer

• B.timestamp = buffer timestamp (GST_BUFFER_TIMESTAMP)
  

NS: NEWSEGMENT event preceeding the buffers

• NS.start: start field in the NEWSEGMENT event
  
• NS.stop: stop field in the NEWSEGMENT event
  
• NS.rate: rate field of NEWSEGMENT event
  
• NS.abs_rate: absolute value of rate field of NEWSEGMENT event(By default a pipeline will play from position 0 to
  the total duration of the media at a rate of 1.0)
  
• NS.time: time field in the NEWSEGMENT event
  
• NS.accum: total accumulated time of all previous NEWSEGMENT events. This field is kept in the GstSegment structure
  
  
  

  if (NS.rate > 0.0)
      B.running_time = (B.timestamp - NS.start) / NS.abs_rate + NS.accum
  else
      B.running_time = (NS.stop - B.timestamp) / NS.abs_rate + NS.accum
  

  
  

• And we also got:
  
  stream_time = (B.timestamp - NS.start) * NS.abs_applied_rate + NS.time
  

• This formula is typically used in sinks to report the current position in
  an accurate and efficient way:
  
  stream_time = (absolute_time - base_time - NS.accum) * NS.abs_rate * NS.abs_applied_rate + NS.time
  

Synchronisation

There are two ways to get the running_time:

• using the clock and the element's base_time with:
  
  C.running_time = absolute_time - base_time
  

• using the buffer timestamp and the preceeding NEWSEGMENT event as (assuming positive playback rate):
  
  B.running_time = (B.timestamp - NS.start) / NS.abs_rate + NS.accum
  

• For synchronisation the following must hold:
  
  B.running_time = C.running_time
  

• expaning:
  
  B.running_time = absolute_time - base_time
  

• or:
  
  absolute_time = B.running_time + base_time
  

• The absolute_time when a buffer with B.running_time should be played is noted with B.sync_time. Thus:
  
  B.sync_time = B.running_time + base_time