• 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    Hmm. Well, I'm not sure I agree, but that's fine. I do appreciate your opinion!

    I think I've put together enough ideas now that I'll write up a requirements/spec/api doc. When I get that done I'll post it here if anyone is interested.
    Posted in: Addon Ideas
  • 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    Elkano - I hadn't considered data merging. Hmmm.
    Posted in: Addon Ideas
  • 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    OrionShock - are the examples I gave in the post not sufficient? Sure, some could be done with more specific logic (rather than a separate library), but perhaps there is performance optimization to be gained here.
    Posted in: Addon Ideas
  • 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    How about a token-ring topology? Like Adirelle mentioned, it would give a "Master" responsible for all outbound communication, and then clients would pass their tokens around at specified intervals. It allows for maximum throughput with only one sender at a time, plus the small overhead involved in joining the topology.
    Posted in: Addon Ideas
  • 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    Thanks for the heads up sparky, I'd have missed that if you hadn't mentioned it. Thank goodness that that's finally going in to the core UI - it's very much needed. (Your add-ons have done a lot for me in that regard, so many thanks there!)

    Today I've been going back and forth and back and forth one exactly how "smart" I want the library to be. On the smartest end, I want it to be a standalone mod with its own PersistentVars, tracking the latest X updates (X configurable per-feed?). There are a lot of advantages to this including reducing how much traffic needs to be generated, and how many callbacks need to be fired. Of course there are serious issues too - duplicated data, possible memory issues, a more confusing install process (well, I mean, relatively speaking.)

    I think a lot of the work that I'll be doing for this library isn't directly related to time-stamping, but is actually related to how the cloud updates itself. I'm thinking that polls would be done in a two-round phase - first ask everyone what their current state is, and then only poll the states you're interested in. (ie. "Give me the values assosciated with timestamps X, Y, and Z"). I'm also thinking that I'll make clients wait a random amount of time before responding, so that they don't all respond at once, and so they can stop themselves from responding if someone else beat them to it. There's a name for that tactic, does anyone know the name so I can do research? I think it's used in some LAN topologies.
    Posted in: Addon Ideas
  • 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    Of course IMMEDIATLY after making this post I find http://wow.curseforge.com/addons/messageboard/ - I'll see if my ideas are different enough to warrant a new addon, or not.
    Posted in: Addon Ideas
  • 0

    posted a message on AceComm-enabled data synchronization across subscription clouds
    Hi everyone,

    I'm working on a fairly ambitious project idea that all hinges on a library I will create that I am tentatively calling "LibASync" (sic), standing for "Asynchronous Synchronization". The apparent oxymoron is intended, and makes sense once the purpose of the library is clear. While I draft the overarching project's design, I wanted to see what people had to say about LibASync - about feasibility, about implementation, about application.

    So, what is ASync? ASync is a library that will perform best-effort data synchronization across clients (via AceComm) using a "word of mouth" network. Clients will 'subscribe' to a data feed, and receive or post updates to that feed, which ASync will then attempt to communicate and synchronize across however many users exist in the subscription cloud. Let me give an example.

    Alice, Bob, and Eve all subscribe to a cloud. The subscription cloud is identified by a printable string name, which is also used as the prefix in the AceComm library to send the serialized data. Alice updates her local variable and then pushes that update to the cloud. ASynch serializes the variable via AceSerialize, and also prepends a field to the serialized message representing a UNIX timestamp, and then sends it to the cloud. Bob and Eve get the message, and their copies of ASynch see that they have no previous record for this subscription, and so triggers a callback to Bob and Eve's client saying that a subscription with the given name was updated at the given time, sent by Alice.

    Now, Bob logs off. Afterwards, Eve pushes a newer update to the cloud. Alice gets the update, records it, including that it was sent by Eve. Eve logs off, and Bob logs back on. His client, as soon as it has finished loading, queries every subscription pool he belongs to, saying "I have this version, is there an update?" Alice's client responds by pushing Eve's update, and Bob records the update. Importantly, Alice doesn't say that Eve was the originator of the message - even if Alice did say that, Bob would have no way of trusting Alice. If the originator of the message is important, it could be encoded in the serialized data, but that data would remain untrustworthy.

    I expect that this will be a fairly simple library to write with some hefty overhead concerns. Certainly at the very least, any large subscriptions could cause chat throttling to hamper communication. There's also the comparatively smaller concern that every subscription's state will need to be tracked both in ASync as well as in any addon using that subscription.

    It's important to note that ASync specifies no security or data injection prevention. Such concerns are strictly the domain of the clients. If true data protection is required, this scheme should be fully compatible with a PGP-like web of trust, though I would be very surprised if that was ever needed by anyone.

    So, the project I'm planning will be doing a lot of cool things with this technology, but I'd love to hear what else you guys can come up with. This project will be focusing on a suite of guild tools, but any ideas are welcome.

    1. "Shout box" - the first client for ASync will be a very simple proof of concept. The client will track (at first) a single string-type variable across the cloud. People can push to the cloud, creating a new "shout". The most recent shout gets displayed through a LibBroker interface. A more advanced version that may require enhanced functionality from Async (or may not, I haven't decided) will track the last N shouts.
    2. "Notice" - a replacement to the extremely boring default Guild Information tab. By default, will only allow guild officers to push to the cloud. The first test of a 'security policy' (ie, ignore pushes from people less privileged than an officer).
    3. "Bulletin board" - the logical next step from the shout box is a bulletin board system for guild communication. Post new topics, reply to posted topics, ignore and drop topics that haven't been updated in N days. Officers can sticky topics that shouldn't ever expire. The first real, heavyweight application.
    4. "Yellowpages" - several addons already try and do this, but I think with ASync they can do it better. Synchronize guild crafting links across the guild, so that you can find out exactly who crafts what.
    5. "DKP" - Track item purchasing points (DKP) so that everyone can see running DKP totals. This module would probably also have an item disbursement and officer management interface, but that's a whole 'nuther ball game.

    Well, that's the gist of it. What do people think? Am I re-inventing something that already exists? Anyone want to help?


    And, by the way, I hereby license this work, "AceComm-enabled data synchronization across subscription clouds", as copyright Erich Blume 2010, released to the public under the Creative Commons Attribution 3.0 Unported License. See http://creativecommons.org/licenses/by/3.0/ for license information. Share and enjoy.
    Posted in: Addon Ideas
  • To post a comment, please or register a new account.