I'm sure some of you are already aware of this, but it's a frequently asked question among inexperienced addon authors. I posted this in the official UI and Macros forum in hopes that some of the potential abuses or loopholes mentioned here might get addressed by Blizzard.
In the meantime, if this functionality is never changed, it provides a "legitimate" means for queued inter-process messaging and communication.
It's possible to get limited amounts of textual data into and out of the game (while not in-combat) without reloading the UI by hijacking one or more of the character's or account's macro slots for your addon's use and using the BackupMacros() and RestoreMacros() API. This will create immediate file changes in the filesystem. Again, this only works out of combat.
This is how the Twitter WoW addon should have been made, to be honest. Even though the APIs for editing, saving, and loading macros are protected, they can still have an abusive impact, or accomplish tasks that are counter to the game's UI design philosophy.
Imagine, an addon which at the beginning of an arena match saves the name of each opponent in a macro, and backs up the macros. A helper utility monitoring the macro cache file notices the change and downloads armory details for each of the players and saves pertinent information in the macro cache backup file. The in-game addon periodically calls the RestoreMacros() API until the player enters combat, and when it finds new information in the macro, it presents those data to the player. (This may not be the best example of abuse, but I'm having a hard time thinking of one.)
Besides the obvious Twitter example, other potentially "cool" uses of this would be a GPS-style web site which shows where your character is in near-real time. An application could also query an online database and provide small amounts of contextual information in-game, preventing the need for shipping large, frequently updated but relatively sparesely used databases within addons.
It's possible to get smaller pieces of textual data out of the game (but not into it) even while in combat, by modifying guild member attributes and using the /saveguildroster macro command. This obviously requires the character to be in a guild and have the permission to modify one or more guild member attributes.
The /saveguildroster macro command really needs to be secured, since it's the only current loophole for getting data out of the game while in-combat.
This could even be used to poll an addon site and alert the user to new versions in-game, without relying on addon communication channel spam and having other players online with newer versions.
Edit: Okay, I've been up all night. Obviously since this technique requires a helper application, it would make more sense to alert the player before the player enters the game. :) But if a player has already been online for 6 hours and gets an in-game alert that a new addon version is available, perhaps they will take the time to leave the game and update their addons before joining that night's big raid.
ok seriously, what is the armory going to tell you about your opponents that your not going to find out in the first 5 seconds of an arena game? and seriously, if your a good player then you simply don't need that information.
I think I mentioned that the arena/armory example was not a particularly "abusive" example, but I was having a hard time thinking of one.
With 255 characters, you can look up mob loot drops or spell usage from Wowhead and display it in-game, without the need for a huge database. You can send and receive Tweets (if you are disposed to using Twitter; I'm not). You can query global AH prices for an item from a web site, or even store your own AH data and statistics locally in a real database using a helper application (slowly) instead of saved variables. You could maybe improve some lazy synchronization aspects for multiboxers. You could be on IRC from within the client.
Edit: Also, if your addon (or framework developed for this purpose) is particularly disrepsectful of the player's and account's macros and is resilient to any other addon's attempted use of those macros, 13,770 characters are available at any given time.
I'd also like to add that all of these uses (as cool or uncool as they may sound) could easily be seen as exploitative, since they involve using the macro body data and backup and restore APIs for purposes they were not intended for.
I still can't see any truly exploitative behavior coming from this. Unintended, certainly, but nothing malicious or against the spirit of the game (...yet).
Not that I'm opposed to limiting their used to prevent this, of course. I'm apathetic on the issue. Seems like a lot of hub-bub for a little something though. :)
True, usually in order for something to be seen as an exploit, it has to convey some kind of advantage to the user, and not merely a convenience. Perhaps my concerns were unwarranted. (I still think it has the potential for some pretty cool applications.)
Although it's not mentioned anywhere, the backup and restore functions (just like the edit function) don't appear to be usable in combat, so any use that could convey an advantage in combat already seems to be impossible.
If Blizzard disagreed with some of the simply convenient uses of this tactic, they could just impose a limitation on how often macro backups and restores can be done. This would have minimal impact on the intended use of these functions, but would largely disrupt any application that tried to use them for near real-time inter-process messaging.
Edit: Yes, making them protected functions would be a very straightforward way for Blizzard to prevent this usage altogether. It seems unlikely that an addon would ever have a legitimate reason to backup and restore macros that wasn't user-initiated, and Blizzard could provide those options in their own macro UI.
The only thing I can see that would be of a major drawback would be an addon that gets this information and broadcasts to others using an addon message channel.
Such as a DKP addon, where an admin would get the information, then distribute it to several members.
Though I don't see this as being very intrusive beyond the scope of what bliz has intended Although it may be a risk factor to introduce "new" data to the client, which could doubtfully, but possibly, cause an overflow of information.
Over-stressing end-user resources, and possibly bliz server resources.
Using your arena example, couldn't you, in theory, have something generate macros for you based on your opponents before combat starts? Granted, the same could be done without outside communications if you have whatever armory data you need cached in an addon before you log in.
Yeah, I'm really scratching my head to think of anything that could give a player a competitive advantage in PvE, PvP, economically, etc. Most of the applications seem to represent more of a convenience (not having to alt-tab or relog) than an advantage.
I've been considering addons that do large amounts of data analysis (AH addons, combat log analyzers). If players have systems with extra CPU cores not being utilized by WoW, the processing burden of such addons could be shifted outside the game into an application running platform-native compiled code.
I haven't done any testing to see what kind of overhead is involved when BackupMacros() and RestoreMacros() gets called.
Another application which comes to mind involves guild loot-distribution addons that everyone seems to freak out about if the raid leader's computer crashes. Getting data out of the game in near real time provides a means of data protection in such cases that isn't offered by saved variables.
Probably the biggest concern is that any of these applications would require an executable helper application to be running at the same time as WoW. If such applications were to become widespread or popular, the potential for malware distribution alongside addons would skyrocket.
Right now it's easy to tell players, "Addons can't contain viruses because they only contain script that runs within WoW." If one or more killer apps surface, players would gradually become accustomed to running executables alongside the game.
Eventually, with this social conditioning, some hacker would come along and gain players' trust with an addon/executable combo that does really cool things for a few months. Then once the hacker has amassed a large enough user base, he would sneak a keylogger on them in an update. In a worst case scenario, tens or hundreds of thousands of accounts could be compromised.
I think this would be Blizzard's biggest concern, and the biggest reason for them to close this loophole. It's also a good reason for addon authors not to use this method, since every innocent use builds the players' comfort level for the hacker to take advantage of later.
I'm confused as to how it would be shifted outside the game? 255 characters isn't exactly that much data if you're talking about a scale at which you would need to shift it outside the game for it to matter which isn't counting the overhead of reading/writing.
If someone mentions it to Blizzard I would imagine it'll get fixed fairly quickly anyway and they'll just change it so macros are buffered and written on UI Reload or protect it as Tekkub said.
Using your arena example, couldn't you, in theory, have something generate macros for you based on your opponents before combat starts? Granted, the same could be done without outside communications if you have whatever armory data you need cached in an addon before you log in.
People have been doing that for years with "/target <blah>" such as with Proximo.
People have been doing that for years with "/target <blah>" such as with Proximo.
Yea, I meant something more specific... like oh there's a (class x) in the enemy group, so use (spell y) instead... i.e. something that tailors your macros based on the makeup of the group...
About 13k of data can be read or written in one restore or backup operation. (14k + 54 more 10-bit integers if you include the macro name text and decode the icon index.)
Edit: Compression schemes could increase this amount, since these limits apply to the length of UTF-8 encoded strings, not their byte count. We're really talking about 13k/14k characters, not bytes. Each of those characters can encode roughly 16 (I think) bits.
Edit again: This also makes me curious how strict WoW's UTF-8 encoding and decoding is. What happens when I feed it a string of octets that don't comprise valid UTF-8, or would comprise overlong encodings of an otherwise valid codepoint? Time for some testing! Just how many bits can I cram into a "255-character" string and get the same bits back out of it? :)
Edit yet again: As an interesting discovery, I've found that when restoring macros, I can read in strings of up to 63 "characters" in length and the macro icon name is not limited to actual texture file names in the Interface\Icons folder. Instead, it is limited to 243 "characters" in length. Additionally, the macro body appears to be larger. The potential message size for getting data into WoW is significantly larger than getting data out of WoW.
As an interesting side effect, I've noticed that the UTF-8 decoding in WoW tends to choke and introduce errors on long strings of characters like "" repeated many times over. The same exact decoding errors occur in Notepad++ version 5.3.1 for 32-bit Windows. However, Windows Notepad handles these characters just fine, regardless of the encoding. (Who knows how these will show up in your browser. These are U+25CF <BLACK CIRCLE>, U+25A1 <BLACK SMALL SQUARE>, and U+25A2 <WHITE SMALL SQUARE>.) These decoding errors may actually only be present when attempting to display the UTF-8 encoded string using a FontString in WoW. Looking at the actual bytes present in the string variable, they appear to be correct.
Iriel has mentioned that Blizzard is aware of these loopholes. I assume this means that even if they don't get fixed, actually using them would risk the wrath of fiery retribution.
Edit: Just as a final coda to this discussion: Blizzard has removed the BackupMacros(), RestoreMacros(), and SaveGuildRoster() global API functions (and the /saveguildroster command) in patch 3.2.2.
In the meantime, if this functionality is never changed, it provides a "legitimate" means for queued inter-process messaging and communication.
It's possible to get limited amounts of textual data into and out of the game (while not in-combat) without reloading the UI by hijacking one or more of the character's or account's macro slots for your addon's use and using the BackupMacros() and RestoreMacros() API. This will create immediate file changes in the filesystem. Again, this only works out of combat.
This is how the Twitter WoW addon should have been made, to be honest. Even though the APIs for editing, saving, and loading macros are protected, they can still have an abusive impact, or accomplish tasks that are counter to the game's UI design philosophy.
Imagine, an addon which at the beginning of an arena match saves the name of each opponent in a macro, and backs up the macros. A helper utility monitoring the macro cache file notices the change and downloads armory details for each of the players and saves pertinent information in the macro cache backup file. The in-game addon periodically calls the RestoreMacros() API until the player enters combat, and when it finds new information in the macro, it presents those data to the player. (This may not be the best example of abuse, but I'm having a hard time thinking of one.)
Besides the obvious Twitter example, other potentially "cool" uses of this would be a GPS-style web site which shows where your character is in near-real time. An application could also query an online database and provide small amounts of contextual information in-game, preventing the need for shipping large, frequently updated but relatively sparesely used databases within addons.
It's possible to get smaller pieces of textual data out of the game (but not into it) even while in combat, by modifying guild member attributes and using the /saveguildroster macro command. This obviously requires the character to be in a guild and have the permission to modify one or more guild member attributes.
The /saveguildroster macro command really needs to be secured, since it's the only current loophole for getting data out of the game while in-combat.
Edit: Okay, I've been up all night. Obviously since this technique requires a helper application, it would make more sense to alert the player before the player enters the game. :) But if a player has already been online for 6 hours and gets an in-game alert that a new addon version is available, perhaps they will take the time to leave the game and update their addons before joining that night's big raid.
With 255 characters, you can look up mob loot drops or spell usage from Wowhead and display it in-game, without the need for a huge database. You can send and receive Tweets (if you are disposed to using Twitter; I'm not). You can query global AH prices for an item from a web site, or even store your own AH data and statistics locally in a real database using a helper application (slowly) instead of saved variables. You could maybe improve some lazy synchronization aspects for multiboxers. You could be on IRC from within the client.
Edit: Also, if your addon (or framework developed for this purpose) is particularly disrepsectful of the player's and account's macros and is resilient to any other addon's attempted use of those macros, 13,770 characters are available at any given time.
I'd also like to add that all of these uses (as cool or uncool as they may sound) could easily be seen as exploitative, since they involve using the macro body data and backup and restore APIs for purposes they were not intended for.
Not that I'm opposed to limiting their used to prevent this, of course. I'm apathetic on the issue. Seems like a lot of hub-bub for a little something though. :)
If Blizzard disagreed with some of the simply convenient uses of this tactic, they could just impose a limitation on how often macro backups and restores can be done. This would have minimal impact on the intended use of these functions, but would largely disrupt any application that tried to use them for near real-time inter-process messaging.
Edit: Yes, making them protected functions would be a very straightforward way for Blizzard to prevent this usage altogether. It seems unlikely that an addon would ever have a legitimate reason to backup and restore macros that wasn't user-initiated, and Blizzard could provide those options in their own macro UI.
Such as a DKP addon, where an admin would get the information, then distribute it to several members.
Though I don't see this as being very intrusive beyond the scope of what bliz has intended Although it may be a risk factor to introduce "new" data to the client, which could doubtfully, but possibly, cause an overflow of information.
Over-stressing end-user resources, and possibly bliz server resources.
I've been considering addons that do large amounts of data analysis (AH addons, combat log analyzers). If players have systems with extra CPU cores not being utilized by WoW, the processing burden of such addons could be shifted outside the game into an application running platform-native compiled code.
I haven't done any testing to see what kind of overhead is involved when BackupMacros() and RestoreMacros() gets called.
Another application which comes to mind involves guild loot-distribution addons that everyone seems to freak out about if the raid leader's computer crashes. Getting data out of the game in near real time provides a means of data protection in such cases that isn't offered by saved variables.
Probably the biggest concern is that any of these applications would require an executable helper application to be running at the same time as WoW. If such applications were to become widespread or popular, the potential for malware distribution alongside addons would skyrocket.
Right now it's easy to tell players, "Addons can't contain viruses because they only contain script that runs within WoW." If one or more killer apps surface, players would gradually become accustomed to running executables alongside the game.
Eventually, with this social conditioning, some hacker would come along and gain players' trust with an addon/executable combo that does really cool things for a few months. Then once the hacker has amassed a large enough user base, he would sneak a keylogger on them in an update. In a worst case scenario, tens or hundreds of thousands of accounts could be compromised.
I think this would be Blizzard's biggest concern, and the biggest reason for them to close this loophole. It's also a good reason for addon authors not to use this method, since every innocent use builds the players' comfort level for the hacker to take advantage of later.
If someone mentions it to Blizzard I would imagine it'll get fixed fairly quickly anyway and they'll just change it so macros are buffered and written on UI Reload or protect it as Tekkub said.
People have been doing that for years with "/target <blah>" such as with Proximo.
Yea, I meant something more specific... like oh there's a (class x) in the enemy group, so use (spell y) instead... i.e. something that tailors your macros based on the makeup of the group...
Edit: Compression schemes could increase this amount, since these limits apply to the length of UTF-8 encoded strings, not their byte count. We're really talking about 13k/14k characters, not bytes. Each of those characters can encode roughly 16 (I think) bits.
Edit again: This also makes me curious how strict WoW's UTF-8 encoding and decoding is. What happens when I feed it a string of octets that don't comprise valid UTF-8, or would comprise overlong encodings of an otherwise valid codepoint? Time for some testing! Just how many bits can I cram into a "255-character" string and get the same bits back out of it? :)
Edit yet again: As an interesting discovery, I've found that when restoring macros, I can read in strings of up to 63 "characters" in length and the macro icon name is not limited to actual texture file names in the Interface\Icons folder. Instead, it is limited to 243 "characters" in length. Additionally, the macro body appears to be larger. The potential message size for getting data into WoW is significantly larger than getting data out of WoW.
As an interesting side effect, I've noticed that the UTF-8 decoding in WoW tends to choke and introduce errors on long strings of characters like "" repeated many times over. The same exact decoding errors occur in Notepad++ version 5.3.1 for 32-bit Windows. However, Windows Notepad handles these characters just fine, regardless of the encoding. (Who knows how these will show up in your browser. These are U+25CF <BLACK CIRCLE>, U+25A1 <BLACK SMALL SQUARE>, and U+25A2 <WHITE SMALL SQUARE>.) These decoding errors may actually only be present when attempting to display the UTF-8 encoded string using a FontString in WoW. Looking at the actual bytes present in the string variable, they appear to be correct.
Edit: Just as a final coda to this discussion: Blizzard has removed the BackupMacros(), RestoreMacros(), and SaveGuildRoster() global API functions (and the /saveguildroster command) in patch 3.2.2.