Wasn't the itemid used in order to be able to map items to tradeskills?
my guess is that the GetTradeSkillRecipeLink command came after this system of identifying tradeskills by itemid. keeping in mind that enchanting also used to use a whole different api (the old Crafting api). so for enchants, you could get an enchant link (probably why it was called an enchant link, now that i think about it) while for the other trade skills the only id you could get was an item id from the crafted item.
in that context, the system makes good sense. but once a generic recipe link system came to be that gives a unique id to each skill, then retaining this system is purely for compatibility.
you are right, tho, that as-is you can attach info to crafted items without having to use any of the tradeskill api. so there is a bit of utility here (otherwise you'd need another table of spellid to itemid) so maybe it's fine to keep. not sure if that's outweighed by the extra effort it takes to generate a secondary index for libpt inside a tradeskill mod that uses spellids internally...
regarding tradeskill data... why are we still using the weird itemID/-spellID format for tradeskill data? i realize mods probably are used to that system, but it seems really quirky and kinda of out-dated, no? tradeskills all have unique id's. why not use them instead of itemID for some and -spellID for others (those that produce no items)?
i'd like to add data to connect enchant spell ids to the resulting scroll id.
where do you think it should go? also, the tradeskill indices are all based on the quirky itemID/-spellID system (as opposed to explicitly being spellID). should the index for such a table be -spellID just to remain consistent?
i think it would be a mistake to try to make a "complete" record of each quest. if we did that, we'd have to include a lot more data, imo. plus, any mod that is interested in one bit of the data would need to decode the whole string just to learn what level a quest is or what items might come from it.
i think it would be better to break the data up into related chunks that could be added as needed. additional tables could be mined for quest start/end locations, chains, reputation gains, etc. the only mods that would need to know about that data would be the ones that care.
splitting it up by quest type would be good, tho. not entirely sure how to mine that from wowhead. they have a "daily" indicator but i haven't seen a clear "raid" or "dungeon" indicator. could be one of the "category" flags...
i sort of based my .Horde, .Alliance, .Both scheme on the tradeskill tables. the tradeskills with specializations are broken up to such that there's a .Basic table alongside the specialist tables. so querying all the recipes a spellcloth tailor could know would include two table queries -- .Spellcloth and .Basic.
i see the logic of making .Horde include the .Both table, tho.
maybe that .Both should be changed to .Common, btw. Both sounds like it could be the union rather than the intersection.
i dunno if it would be getting too wooly, but my usage of this data has already got me wondering if some kind of rough level breakout would be worthwhile as well. i can only think of highly arbitrary methods, tho. like level1-10, level11-20, etc. but then we'd have 8 times the number of tables which already could be kind of large.
the nice thing would be that a mod could specify the range of levels its interested in and skip all the data it doesn't need. in my case, i'm working on a mod to do ingame item upgrade suggestions. at level 50 you're probably not going to find any upgrades in any quest below level 40 and you can't run quests at level 60+ yet, so that's a lot of data i can outright.
perhaps that's not really a big issue, tho, if the level data access is fast enough (another reason to keep the data fields in separate tables). first thing to do is to check levels of quests to figure out the list that's of interest before delving any deeper.
i've got the dataminer working already, so it's just a matter of figuring out what the data format should be...
for quest chains, would you just identify the previous questID? like .Chain = QuestID:previousQuestID, ...
you'd have to chace the chain yourself, but it would make for the smallest data set.
the A/B/C would mean one of A or B or C and the + would mean you get to choose a few different items. i don't think there are really ever three choices (like my example) -- heck, i'm not even sure you ever get two choices, but i figure it's worth thinking about it more generically from the start. i think the most common would be A/B/C/D and A/B+C.
(edit: need to account for times when you recieve multiples of the same item -- maybe Ax#?)
i'd also like to add
Quest.Levels.Horde/Alliance/Both = "QuestID:minLevel/questLevel,..."
minLevel being the minimum level you need to accept the quest and the questLevel being the reported level of the quest. not sure if it would be best to indicate things like raid or group or pvp here or if in another table...
Quest.Type.Horde/Alliance/Both = "QuestID:type,"
type = R for raid, G for group, D for daily, S for seasonal... if it's not in the list, it's just a normal quest. could add the # for the suggested group size perhaps.
or this table could be broken down like:
Quest.Raid.Horde/Alliance/Both = "QuestID"
Quest.Group.Horde/Alliance/Both = "QuestID"
not sure it's worth doing with the other tables cuz then looking for a quest reward would require knowing what kind of quest it is, unless there's some way to also stuff that data into a multitable (which i don't exactly understand, tbh).
Do you mean mine a PT3 set dynamically? Or that some mods can skip the PT3 sets for trade and do their own mining?
no, i actually meant mine the data using the wow client directly instead of wowhead or any other online database that may or may not be up to date. the rest of the process would be the same. the user wouldn't really see any difference except that the data would be 100% accurate for each patch as soon as somebody ran the update and uploaded the new data sets. i dunno, might not be worth it, it was just a thought. there's still a lot of data that the api can't tell you...
with tradelinks being able to access pretty much the entire database of trade skills, i wonder if some of the data mining could be reworked to run on the wow client. things like accumulating what items are used in which trade skills, what items are created by the different trade skills (and what spell creates them), etc. those can all be mined directly now since anybody can simply create a trade link that contains all recipes for any trade skill ('cept smelting oddly). is it worth looking into? you'd have to play with saved vars files, so it'd be kind of breaking the mold a bit...
edit: looks like you can also mine the levels at which a skill turns yellow, green, and gray (but not orange, sadly).
That said, space used is still significant to some degree. So:
*Please split it into a separate file so you only burden people who actually need it.
*Go with "/" so somebody can just print it straight out if they want to.
*Do not compress, so someone can go in and read and understand the data straight up without a math degree and look stuff up on wowhead to detect errors etc.
maybe he should simply stick to # as seperator. As you said, / is used for "x per y". But ; is used to seperate a list where order doesn't matter. In this case however, order does matter (although it's implied by ascending numbers).
Also I vote for not sticking it into PT-3.1-Tradeskill but into a spereate file. This will allow addons to only load that part of data.
in regards to the "/" vs ";" vs "#" do you guys think it makes a big difference? libpt doesn't actually DO anything with the data, right? so it's really up to the mod that uses the data to understand what it's getting. it's sort of inconsequential since most people will probably go ahead and split up the values, but "100/110/120/130" is the only form that could be used as-is, it seems to me, so that's why i favored it.
and i see your point about splitting it up into a different file. TradeskillLevels (no dot) would probably be better then.
on the table still, i suppose, is the idea of encoding the data a bit more to shrink it down. in that case the human readability goes out the window so "#" would be perfectly fine. also, the data might become small enough to slip into the normal Tradeskill file. i'll poke around and see just what i come up with.
one simple thing would be to change from absolutes to deltas -- how many levels to get to the next color. eg 100#10#10#10
many skills follow a pretty linear progression so i could change that to 100#10 implying that each subsequent level break is 10 levels away. or 50#40#20 for skills that have a longer orange than yellow or green (which is common).
i just noticed that wowhead as some skills without an orange value, so i'll have to fix my code.
so compressing by changing to deltas and clipping off redundant values (only clip the 4th or the 3rd and 4th) the textfile size goes from 68k to 48k (58k if i just do the deltas and not the redundancy check). the data is screaming for more compression, but i suspect this is about as much as makes sense from a usability standpoint. i think maybe "+" would work in this method as a delimiter.
i think you could explain the data.lua process a little more. it seems a bit counter-intuitive that data.lua is both generated and editable. i think maybe instead of describing the dataminer as scraping wowhead for data, perhaps you could describe it as scanning data.lua and updating tables that is has been explicitly programmed to update while leaving untouched all other tables.
i'm a little vague as to what happens between the data.lua file and the individual breakdowns. is it a hold-over from the old system where data was compressed?
i think it would also be nice if there was a requirement to provide a description of the data somewhere when editing/generating tables so people can know what the table is and how to use it.