Here's an idea; You enter a Twitter hashtag (or any search, I guess), and have the resulting tweets replayed in real time as they would have appeared had you been following the hashtag when the event occurred. That way you get the full Twitter-augmented experience when watching anything after the fact.
A feature to speed up and slow down is probably a good idea as syncing is going to be a challenge, but overall it doesn't have to be synced with millisecond precision.
It would also be interesting if you could add your own tweets to the timeshifted stream after the fact, but I am not entirely sure how that'd work. Perhaps adding a machine tag indicating a timeshift factor or timestamp could handle it. It would be pretty confusing for your followers seeing your out of context tweets appearing, though.