Well it's a topic I wanted to put out for a while, and knowing that it's difficult and has various sets of opinions coming to it I have been putting it off too. But anyway here goes.
Why sync combat log-related data?
Well the combat log of an individual player doesn't show everything he wants to know. Here are examples:
*) See my own contribution accurately
*) See other's contribution accurately
*) See my threat or other derived information accurately
*) See other's threat or other derived information accurately
The short answer to the question why we sync is: Because the combat log doesn't give a player all the info.
There are different types of info that may be missing:
1) Inherent non-transparency of the combat log. The combat log simply does not inherently show (a) an event or )(b)a relationship.
(a) For example currently (patch 2.4.1) does not show deaths of player units.
(b) For example Prayer of Mending. The combat log does not show the originator of the spell when it heals.
Here there are various ways of trying to cope.
*) First is non combat log info available to the player. E.g. one could do death tracking by checking group for death status via their unitid and UnitIsDead().
*) Second one is that another player has that information. Then we can "sync" that information. Those type of syncs that create a functionality that overcomes an inherent non-transparency of the combat log I tend to call "functionality syncs". I.e. syncs to enable a functionality that is otherwise not possible.
A concrete CL example are Greater Elementals of shamans. These do not offer a SPELL_SUMMON that will link them to their totem and hence to their actual originator, the shaman. There are two ways to overcome this:
*) The combat log offers a flag if a guardian belongs to you. If you detect a Greater Elemental event that is yours you sync that information to others in your group. This is tricky handling because others have to wait for that extra information to properly attribute that elemental to its owner.
*) This approach I call Cryect's trick, because he proposed this. When casting an elemental totem, the greater elemental summoned after it seems to have the next NPC sequence number in its GUID. This is fairly robust but not 100%. Hence given a totem SPELL_SUMMON one can check for a valid tooltip with the Greater Elemental's constructed GUID and this derive it's owner.
Both these approaches are heuristic and can fail in pathological situations. This is not unusual for functionality/restore or sync solutions.
2) Combat log "range" based. The combat log in principle shows events, but they are for some reason out of the reported range.
This range is nowadays large, roughly 150 yards, but the exact mechanism how units are seen or not seen is not documented. Since Slouken's remark that the combat log shows everything that you can see I operate on the assumption that the combatlog range coincides with UnitIsVisible() though there is no definite confirmation for that.
This affects both (a) self damage and (b) other damage.
(a) One can end up out of range of ones own event and hence for example no longer see ones own DoT or HoT. I have yet to have definite confirmation of this and am actually working on the assumption that combat log events relevant to self are always shown, but this may be incorrect.
In any case it is a very good heuristic to assume that one sees ones own damage because one is certainly always visible to oneself.
One can try to sync for this information if it's missing.
(b) This is more frequent, it happens in very large area fight or in "split" fights. However a lot of fights should have no or minimal need for this because certainly everything is visible to everyone.
Approaches to syncing:
*) Sync everything
Send all combat log events (or at least those that one wants sync'd to everybody), run a running diff to determine missing events. This is extremely expensive in addon channel traffic and I have never seen anyone actually try this.
*) Sync frequently
This is the model that heavy syncing solutions use. Try to update frequently to reduce the error of the inherent heuristic of not syncing every event. This approach is very expensive on the addon channel but will sync at reasonably high quality. This approach does not guarantee exactness. The original sync of Recount used this model.
*) Sync sparsely
This is the model that syncs less frequently to keep addon bandwith in check trading off for a larger possible error from the inherent heuristics.
*) Sync self only
This approach assumes that one is very good at inspecting ones own events. By that assumption it cuts the addon bandwith. Lazy Sync in Recount uses this model in principle, combined with another one. This approach will get more accurate the more players participate in syncing, in the limit reaching the comparable accuracy of frequent syncing but at much reduced bandwidth (because not everybody syncs every group member).
*) Sync out of combat
Self syncing allows one to hold off syncing for long periods as one assumes that self always sees everything (leave pathological situations) and hence one can wait until the end of combat to sync. This is also used in Lazy Sync of Recount. This moves addon traffic when it doesn't hurt, comparable to the garbage collection mechanism in WoW moving collection cost to out of combat times.
*) Other schemes
Theoretical considerations of syncing:
*) Unneeded syncs: A sync is unneeded if it turns out that the data was the same after the sync. Lazy Sync uses UnitIsVisible() and testing if you actually created data to determine if syncing is at all needed and only syncs if there is an actual chance that the other person needed a sync. This still can be unneeded as we do not know if the other person already saw the events.
*) Missed needed sync: A sync that was needed but not sent. Someone missed data that is available elsewhere but wasn't sent. Lazy Sync and all self-only syncs accept this for anyone who does not contribute their own sync. It can also happen if the sync-need code is not exact but heuristic.
*) More heuristics for detecting misses: One can try to checksum combat log and sync those to detect missed data. This is a trade-off. Spend sync traffic to determine misses in order to not send traffic that may be unneeded. If the traffic to determine misses is on the order of the blind solution this approach is not worth it. I know of no data that gives indication about this in typical, best and worst combat situations.
Why sync again?
This time lets look at why sync from a blizz side.
*) Threat faitfulness. The reason why PoM, Lifebloom and Earthshield heals show target as source is because that reflects the threat mechanism of these spells. These are examples where the combat log is designed to be faithful to threat rather than originator of the event. This is not needed per se. A threat meter could know that PoM, LB and ES heals always cause threat to target not source, but this is what we have as official explanation why these events are the way they are.
*) Visibility faithfulness. CL shows what you can see. Why that is hasn't been explained, it could be due to technicalities of the underlaying engine, it could be other reasons. Certainly a combat log that shows everybody in a group would be much preferable from a damage analysis point of view. If this behavior is needed for threat faithfulness I don't know.
*) Bandwidth. The current CL will have at times use less bandwidth than if it serviced everybody in the group. But as in other cases it does have to service everybody anyway this is unlikely the real reason for the current CL behavior.
"Range" syncing certainly would go away if blizz decided that CL of group members are fully served independent of range.
"Functionality" syncs will go away if attribution to owners of events is always transparent through the combat log. 2.4. has much improved this and some things that were completely impossible (same-named pet, totems) are now solved, and some have good heuristics (like LB, ES, PoM) though not perfect.
So we try to sync because we don't have those things.
I don't really know what your post is supposed to achieve Elsia.
We already know that
1) Syncing is hard
2) Syncing is made harder by the fact that there is
a) Limited bandwidth
b) Limited CPU usage (some people have really bad CPUs)
c) Lack of understanding of how the combat log works with regards to what it tells us (range, events, etc)
d) Incomplete information as presented to us.
You have poured in a lot of effort into Recount and keeping it alive. I think a lot of people, including me, can appreciate this.
This brings me back to my point though, is this post just to express your thoughts and feelings about it? Or is it meant to stir up a discussion about syncing methods to come up with new methods to produce more accurate numbers and figures, while yet using as little bandwidth and CPU usage as possible? Or a bit of both maybe? My response is as-such, because the post is somewhat directed at me and Cryect, as pointed out in the Recount thread.
I don't really have a new syncing solution to offer, other than what is already employed by today's damage meters. Maybe I'll come up with something...
It's in only vaguely a response to you and cryptmagic (not cryect) in the Recount thread. Mostly if people really want to discuss syncing I rather have that debate separately from the Recount thread where it would be constantly interspersed with bug reports etc.
I wanted discussion on the topic for a good while anyway (before I decided to try ooc syncing, which goes back to January at least). The purpose is to invite people to think about how to do this, because as you said it's a hard topic. Also I wanted to expose my assumption for critique, and yes I do make assumptions. I make no claims in having the best solution for syncing, or if I know any way to even arrive at a judgement what the best solution would be.
But to be clear, I do not mean this to be about syncing in Recount. It's an example that I know most about so that'll color my perception and lead me to use it as an example, but that's it. Forget Recount or Threat-2.0. I really just care to discuss what ideas one can come up with to do this. Initially I wanted to think about a combat log sync library that would standardize how to do this, but so far I have not acted on that idea because there are too many open question what the right approach would be.