• 0

    posted a message on Data Exchange Manager (Need Feedback)
    Thanks. That would be fine already. I think we've highlighted just about every corner of the idea and that's a lot more than I ever expected from this thread. So thanks for bearing with me.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    It is obvious that I tried to find an example to illustrate an idea. Of course I outrageously oversimplified how Omen is supposed to work. Not to mention what happens if the person having the threat meter master addon gets disconnected and everyone's display freezes. Try finding a good example. Basically all I'm saying is that almost any addon that uses SendAddonMessage may benefit from my idea. In some cases you really do need a full install of an addon on everyone's client so those would be cases where it's less useful.

    But it's great that you told us something about how Omen works and where the problems are. :D You can learn something new every day. And the WoW API is really quite large. I think that's one advantage of a community of developers, there aren't many problems that haven't been discussed already.

    @Xinhuan: I'm sorry that I haven't been paying attention to WotLK development. I guess between playing the game, raiding, developing addons and real life obligations I just didn't find time to educate myself enough about what's to come in a few months. Maybe if I stopped brushing my teeth I could spend those 5 minutes on more important things :D
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    I agree. But I can't very well talk about this on Curse since they rely on advertisement and this idea might indirectly hurt them if it worked as intended. WoW is a big business and I prefer free sites that have no such restrictions. Maybe I'll look around to other sites but I thought I'd start here. After all, you do have good developers and I don't know if differences to other communities are really all that big.

    What I still don't understand is how you cannot find this useful. I didn't want to bring this up at first because of its obvious security issues:

    For some specially signed modules, you can lower security restrictions, allowing them to be executed immediately after clicking yes on a dialog box instead of having to install them. These modules are very small and only temporary in nature.


    Example 1:

    Lets assume that every WoW user had my base addon installed. When somebody asked in the chat where quest XYZ in Nagrand can be completed, you wouldn't have to tell them the coordinates. Having a ping-sender addon installed, you would simply right click on your map and choose "send location to user". The user gets the coordinates along with a tiny portion of signed code that displays a ping on their map.

    Having the base addon installed means you are able to process any data of any addon that will ever be made in the future because that data will come with its own handler which the base addon can run.


    Example 2:

    Let's take another example that should be closer to your community: Omen. I know it's been mentioned before but you probably only considered distribution of updates. I'm asking why does everyone have to have Omen installed in the first place? Wouldn't it be enough to have one user download the latest version and send a small stub to everyone else in the raid that is able to display bars and data in a window? That single user could calculate threat values and send them to everybody else which would drastically reduce the number of packets that have to be sent. It would also save memory (and maybe even CPU time on the whole) because only one user has to have the threat library installed on their system. Granted, you can also do that using a client/master approach where all other users need to have at least a small client installed.

    Don't you think developers will be glad if they don't have to wait a couple of months until people finally notice how useful their addon is. With my idea, all it takes is just one guy in a raid or even only one guy on an entire server who has the addon to make it immediately work for everyone else he comes in contact with! There's no better promotion for your addon than others seeing it in action. With Omen, there's just nothing to see unless you have it.


    Example 3:

    I'm an enchanter and want to advertise my wares. If a user is interested, why can't I just send them a graphical window that will pop up on their screen and display all I have to offer including prices and such? Yes, it also works via the chat but not as well. The client has it all, boxes and whatever, they're just not available to another user.


    What I propose is an universal Turing machine, a dynamic scripting engine on each client, akin to scripting for webbrowsers. I agree that there are many security issues with this even when only accepting signed modules. Security experts recommend completely disabling scripting in webbrowsers because it's inherently a bad idea. But nobody ever said it wasn't useful. So fine, tell me that my idea is dangerous and we'll discuss it. But don't tell me you don't see any use for it. That's just driving me nuts! ;D

    That would be like visiting the IBM site and they ask you to download ibm.exe first or the site can't be displayed. After all, if the user wants to use the IBM website for a longer period of time then they won't mind installing the EXE, right?

    Maybe it has to do with the mindset. I feel like explaining to a couple of old HTML guys why they need Web 2.0. Maybe they don't. You can do everything with static pages and everything else is overkill. Who needs all that stuff in an online game? Maybe you don't. But don't tell me you can't see how this might be useful.


    PS: I am grateful for all the comments so far. I will not keep on trying to convince you if there are no more comments. So if anyone is tired of my stubborn refusal to give up this project, just stop answering. I can't force anyone to accept my idea but I will try as long as I think it's not entirely pointless :D
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    @Ringleron:

    SavedVariables aren't protected in any way from being read or written to by other addons on the same client. This is usually not a problem since nobody does it anyway (why would you) and it'll only cause problems later on if you did. But in my case, the base addon can register two variables in its own TOC file which it doesn't need itself but which it provides to modules for saving data. You might access data in this fashion: DEM_SaveVarPerChar["MyModule"]["x"] = 3. You could also use some load/restore handlers or mapping metatables to make these variables more accessible (shorter variable name).

    You make it sound awfully childish. :D Of course I'm not asking everyone to agree with my idea, far from it. But any thread where one person is trying to advocate an idea and 2-3 people are strongly and relentlessly opposed is doomed. I just didn't expect that much resistance. Normally people will tell you if something is a bad idea or not useful but not actually fight you every step of the way. This just made me think that maybe not everyone is completely objective about this. And if that was the case I would be right to feel disappointed. But seeing as how the idea still gets lots of negative feedback even after I've given up, maybe people are objective and really do just think it's as bad as they say. I'm new here so I don't really know the community yet.

    @lilsparky:

    A friend of mine has about 200 addons which consume ~40MB of memory. He tells me that if he didn't use WoWAce it would very well use twice as much or more. I'd never want to completely replace the current addon system which works really well. And there's really no point in trying to load all 200 addons via my idea and double the memory requirements. This is intended for maybe 20-30 small addons, each about 10 to 20KB in size.

    @Xinhuan:

    I've done some research and it seems that any global code loaded by RunScript() somewhere that produces an error will result in an error message that includes the first line of the loaded code. So you could include a comment such as "-- Module:MyMod --" that points in the right direction. It would look like this:

    Date: 2008-09-01 09:17:35
    ID: 1
    Error occured in: Global
    Count: 1
    Message: [string "-- Module:LFGVoyeur --..."] line 69:
     attempt to call global 'LFGVoyeur_Pull' (a nil value)
    Debug:
     [C]: LFGVoyeur_Pull()
     [string "-- Module:LFGVoyeur --..."]:69: LFGVoyeur_Refresh()


    Using loadstring() you can specify a second parameter naming the code block as described on WoWWiki. Thus there shouldn't be any difference whether code has been loaded from a Lua file or by loadstring(). Who knows, maybe the client itself uses loadstring() to load code from files specifying the filename as second parameter.

    If an error occurs, the base addon could even process the error message directly and display the offending portion of code in an ingame editor/debugger. That's again something you can't do with normal addons.

    Why am I always the one who has to point out these things? :D
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    SavedVariables aren't a problem. There are always ways to getting them saved even if modules lack TOC files because the base addon has one and can provide tables if necessary.

    I don't know if error reporting can somehow be solved but I will certainly look into it.

    I still think this idea could turn into a great solution for very small snippets and mini addons. But I guess I shouldn't have described it as an alternative solution to the current addon system in general. People here seem to feel threatened because they have a great system in place already (e.g. Ace Updater) and because my idea might cause some inconvenience later on even though there's not much dispute that it may work in principle.

    So, better to squash the whole idea exactly because it could work. I'm sure there are much smaller addons that aren't all that useful that haven't been dissed in that way.

    I've even signalled that I'd be willing to adapt the idea to something the community might feel more comfortable with without getting any positive responses. I just find it hard to accept that people think the idea is THAT bad even with all the technology in place to make sure things don't get out of hand.

    Of course everyone is entitled to their own opinion. But I guess I'm just a bit disappointed. People are usually so inventive. For any other addon, they may spot possible problems and even give hints as to how they might be solved later on. Not so here.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    Oh well, I see I'm the only one who believes in the idea and that there doesn't seem to be any interest to pursue this any further. No matter, I've already spent enough time on this so I might as well finish it properly. I'll just start converting addons into modules to get a good package and then let users decide whether they like it or not. That way it'll resolve itself.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    Why don't we just try to find a common denominator? You don't like sending addons ingame, fine. I can put that on the shelve and try to slip it in later. If it catches on fine, if not then fine too.

    So what should we (or I) then start working on? A general way to exchange addon data? I've seen in the API that TOC information is available at runtime. Maybe one could find out which save variables are used and pack them into a string? This sounds more like a problem that could be solved more easily with one of your libraries.

    Just put some data exchange interface into your profile DB lib thingy and BAAAM, all Ace addons are able to export settings to other users ingame. It's probably more complicated than that but still... you don't need me for that.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    @Pastamancer:
    Modules should be convenient for users, not for developers. Developers can test their code and exchange it with friends even when it's unsigned, although users have to jump through hoops to accept unsigned modules. But that's the price of security.

    I propose that a module has to be posted on a public site for review for one week before it's getting a signature. Several responsible groups could have their individual private key that works for signing. I will obviously have one and e.g. the Ace moderators could have one etc.


    @Xinhuan: There's no worm effect here. It is possible to breach the security defenses of one person by modifying the code that is supposed to protect the user. But that doesn't mean the code is spreading now. Each user has their own defenses that have to be breached. As long as your very own defenses aren't breached, the signature checking code is in place and will keep you from getting anything dangerous.

    Anyway, once bad code is in your system, all bets are off. Your system is compromised. It doesn't matter if the bad code changes my addon to allow further evil code to get in or if it comes with it's own copy of my routines (a general problem with open source) or even with its very own download stub (which isn't hard to code). All of this is possible with normal addons too.


    @Kagaro: Now I think you're being unfair. :D The signing process makes sure addons have to be reviewed by the public before they can be spread which is exactly how it works with real addons that are uploaded to a site. Suggesting that the public might mistakenly sign a module is the same as the public missing bad code in a publicly posted addon.

    One thing that is true is that it's easy to delete a bad addon even if you miss it for one week. Having already signed a bad addon means that you have to revoke a signature by spreading some kind of emergency update to put it on a blacklist (designing such a revocation system won't be hard having the rest already in place). It's more complicated but it's possible. You are right that we have to be extra careful to make sure no bad code is signed by mistake.

    But even if it does happen, users still have to accept every incoming transmission. So there's still no worm effect. I have to admit that users may accept addons more quickly when they're signed because that seems to be a guarantee that they're safe. But the same applies to addon sites that only post addons that were reviewed.

    Modules might indeed spread faster than normal addons because the whole system is designed to spread any addon very quickly. But you won't sell your car just because walking is safer, right?


    I admit that this all seems very complicated. Maybe too complicated to implement properly. But most of it is already done. And these technologies weren't invented by me. They're being widely used in the industry and have been proven to work. I've been interested in this kind of stuff for a while so this seems easy to me.

    I'm really glad about the many comments so far. Like I said, I will go ahead with this project no matter what, but if I could win the support of a few people here who think the project isn't all that bad, this would really be a big help. There's still a lot of work to do, too...
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    @Pastamancer concerning the sharing of data:

    First I wanted to name my project Addon Exchange Manager. But actually you can exchange any kind of data with it, addons just happen to be executable. I've designed the header to be as extendable and independent of my implementation as possible. Who knows, if my idea works out then other developers might do things with it too and it makes sense to define some kind of standard format to avoid conversions later on. Even if you don't agree with my idea, some of my work may be useful to you too since it touches many different areas. Let me show you some of the problems concerning the header format and how I tried to solve them. It looks like this:

    Id: "DEM1"
    Hash: <40 character hex value>
    Signature: <256 character hex value>
    Fields: "NEVDACF"

    Name: "Test Module"
    Extension: "exe"
    Version: "1.0"
    Description: "This is a test module!"
    Author: "Lenja"
    Category: "Raid"
    File: <string containing addon or data>

    A valid module must contain the first four entries (id,hash,sig,fields). Fields specifies what and in which order additional fields appear in the header.

    Fields are separated by an underline "_". So a valid header will look like this:

    DEM1_637EF7048617C2E72D9D71755BCA35397C65C3D7_C0E7
    C47EAE056A5410B677815D53991C68C28E2076553D5E8833A9
    CFB77214E6DF04A003BD49F1A4A37983FAA01EDA10D9AA5586
    8F9864F810F576E601C5BF491547A51ACE4C2AFEAFE097F3B7
    FD2152FBDDC176C442B28D42E76482BA5F890181FE48C8FEBE
    EFD708404C162F7EFA03F5C0735AC74F6B1CAA02D4D66D4B6E
    4_NEVDACF_ (additional fields following...)

    You see that the character "_" is used to separate fields, so naturally that character may not occur in any fields themselves. Zero characters may cause problems in C strings and won't work with SendAddonMessage() so those cannot appear as well (which is really bad if you want to include binary data). Any special characters above ASCII 127 (depending on language settings) cannot be reliably used to copy&paste from and to edit boxes inside WoW because of UTF8 conversion, which is inconvenient if you want the user to be able to import modules without having to write data to save files in the WTF folder.

    For all fields except the first four, I've chosen the Base64 encoding standard which uses 64 unproblematic characters that are also used in email transmission. This will increase the size by one third but that can't be helped. We can shrink the fields again once they're loaded in memory.

    We gain the following by this proposed format:

    - A module can be identified by the first five characters "DEM1_". Future formats that differ from this specification will have a different ID so you can hold them apart and implement different handlers.

    - The SHA1 hash value guarantees that the message has not been changed by a transmission error. (better than the Sig for error detection because it's faster)

    - The RSA signature guarantees that the following data has not been tampered with and has been given an "OK" by someone in possession of the private RSA-1024 key. If I have implemented it correctly, there is no way how you can crack this unless you use 1mio PCs and let them work 1 year on it. If you find a way, tell me how. I'm sure many mathematicians and intelligence agencies around the world would also like to talk to you. Meaning: You won't crack this.

    - The "fields" field allows us to omit or add more fields in the future in any order without breaking the basic format. Future modules should be downwards compatible.

    - All other fields are encoded in a way that is compatible with Lua, Lua save files, WoW edit boxes and Internet message boards (which tend to truncate space and tab characters).

    So I hope you see, I'm not just some guy who wanted to code something together that works. I've been thinking on this for weeks. I want to set standards and get a rock solid solution. I'm sure you guys with your wonderful Ace libraries can appreciate that.

    I'm hoping you might think my idea is worthwhile and if anyone wants to participate...

    The first implementation of this model is already done and working so far. I'm sure it contains a few bugs and inaccuracies so I'll have to rewrite some portions of the code. But the concept stands and it would be wonderful if this could be turned into a community effort if anyone's interested.

    Since there weren't any crypto libraries for hashing and RSA available in plain Lua, I had to implement them myself which wasn't trivial. I've published them on LuaForge, so they're freely available here: http://luaforge.net/projects/sha1-rsa/

    I think many of you fail to see how cool cryptography is and how it may help you in your projects, addons or otherwise. For instance, you all know the problem of bad code causing all sorts of problems. I've read somewhere that people should not worry about the quality of the Ace libraries but about the code that uses them. Anyone can take your libraries and do whatever evil things they want. Luckily the community is there to tell them just how crappy their code is.

    But I won't even have that problem. I may simply choose not to sign badly coded modules. And that's the end of it. No chance to distribute your module if it's not signed even if I have no direct control over the distribution system. Signed data can take care of itself. I don't know if that's a good or bad thing depending on how picky the holder of the key is, but you can't deny it's really cool from a technical standpoint.

    Coming back to your Grid idea: The addons that receive such data have to be aware it comes from a potentially unsafe source and sanitize all input. I'm sure someone will find a way to design a Grid config file in such a way that the recipient's WoW will crash. So in that respect, immutable executable addons are saver than changeable data that can't be signed.

    @HunterZ:
    You're right. Although signatures guarantee that bad people cannot send you evil stuff, you really want some kind of filtering to keep random people from flooding your inbox. It is another deterrent for abuse since you can configure it to only accept data from friends. So somebody would have to get on your friendslist first just to be allowed to send you anything at all. Once in your inbox, you'll still have to give your "OK" to install something.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    The main problem is getting people to install the base addon. Once that's on many players' computers, the system would start working. Yeah, it very much suffers from the same chicken and egg problem it tries to solve.

    But I don't think it's a lost battle. Maybe the concept has to be changed a bit. I've identified two points I can work on:

    1. Creating modules has to be extremely easy. Developers will stick to the old way of creating addons using XML and not just switch to another system that depends on a framework. A converter that reads a toc file and spits out a module would be extremely helpful.

    2. An ingame browser that lets you download shared addons from other people. That way, even if only one person who is online has the base addon and a few cool modules it'll be a very impressive show effect if a user who is trying the base addon comes online and sees a whole list of other addons to get.

    @Pusikas: I'd say Blizz has implemented a very solid addon system and that's as far as they'll go. Any further innovation beyond that has to come from players. I can see this work within the current bounds set by Blizzard if it's done right. And I'm willing to see this through no matter how much work is involved.

    @tekkub: Winzip isn't all that complicated. But it's still one step less to take and a whole lot faster if you don't have to restart the client. I'll admit that many problems are yet to be solved once this system gets used more widely (e.g. giving users a good overview over all modules they have installed and keeping memory usage in check etc). But it's not really a complicated system. You pack an addon into a single string, append a well documented header and that's it. The base addon is responsible for managing them and making them visible to users. But apart from that, modules don't really differ from normal addons. They are in a special form but not changed or restricted in any other way (apart from not being able to include special graphics). A badly coded addon will be just as bad if inside a module but it won't mess anything up beyond that just because it's a module.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    @Astaldo: I don't know about starting a new trend. I think the current way of addon distribution via the web is very nice, just in some cases it doesn't work very well. There are some ideas for addons in my head that just won't work unless there's a faster way to install them.

    E.g. a minigame like chess or nine men's morris. If you're waiting outside an instance and want to play a game or two, good luck trying to convince your party members that they need to download it from some website now and restart their client just for those 10 minutes.

    Some addons just won't work unless everybody has them and the reason why they don't get them is because they're useless because nobody else has them. Chicken and egg problem...

    I do have a first addon module in place that works. I don't know about its practical value since the normal version is only downloaded by 3-4 users from Curse each day. It is posted as Base64 block in my Modules forum.

    I agree about the need to have a converter. It's not too much fun translating XML files from existing addons into Lua code. Maybe something like that already exists? I really don't feel like writing yet another XML parser.

    @OrionShock: What's the difference with real addons? You'll probably say: "The difference is that publicly posted addons can be scrutinized by knowledgeable users so unsafe addons like your 1c hack can be identified and removed."

    But the same holds true with signed addons. You can't just make a bad addon and send it to people because it'll lack a digital signature. You need to post your addon publicly where it can be checked to get it signed by me or some other party. Once it is signed you can send it to people ingame but cannot change even a single byte without invalidating the signature.

    And that's why I hope my idea will work in the end. I would never even have started without my own Lua crypto libs in place.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    Thanks for the heads up. I've just tested it with a 1MB string and there seems to be no limit. I agree that keeping the memory footprint small would be a major design goal.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    Well, I was hoping it would go beyond raiding.

    I've implemented RSA and SHA in Lua just for the security aspect of this idea. Using digital signatures, I'm sure I could make it fairly save. So no screwing of users. It's basically up to module authors what they want to implement and the addon tries not to restrict them in any way.

    You are right of course, storing addons in variables takes twice as much memory as normal addons do. Once because it's in a variable and again when it gets loaded.

    But for many smaller addons with only a few kilobytes, this wouldn't be too great a problem, right? The memory requirement for each addon could be displayed to the user so they could decide for themselves which addon is too wasteful for their taste.

    With the digital signatures in place, the community could even decide not to sign badly coded modules.

    Thanks for the comment about RDX. I will definitely take a look at openrdx.
    Posted in: Addon Ideas
  • 0

    posted a message on Data Exchange Manager (Need Feedback)
    I thougt I'd start a thread here in the hope that a few people might take a look at my new project and maybe provide some feedback.

    I'm almost positive that there isn't anything like it out there yet, at least I haven't seen anything similar except for the automated external installers from Curse and WowAce.

    In a nutshell: It is an addon that allows you to send and receive other addons via ingame chat channels without having to visit a website or run an external program. You don't even need to restart your client. Simply share addons ingame with your friends.

    I've written some stuff on how it all works here: http://www.lenja.ch/forum/viewforum.php?f=2 (shamelessly posting link to my forums)

    If some of you can spare 5 minutes and take a look at it... I'd be really grateful for any feedback you can give.
    Posted in: Addon Ideas
  • To post a comment, please or register a new account.