30 réponses à ce sujet

[MrFob]

Ok, thought I'd make a new thread about this because we were really derailing Helix's JAM thread over there. Just to bring everyone up to speed, I am basically continuing the discussion we had here. I'll just quote giftfish's last post since the nested quoting system on the new BSN boards is pretty messed up:
 

As the situation exists now, yes. But, this whole discussion is in the context of users who extract their DLC, and the presumption that the content modders will now be able to make WIndows installers for those DLC files (if they choose). If that's the case, MEHEM/TM doesn't need to be installed to the sfar anymore; it can treat the unpacked DLC files as any other game file.

Your comment here seems to assume that MEHEM/TM would still be installed to the sfar via VPatch or a .mod file. But, for TM, if I'm going to have to make windows installers for folks who unpack their DLC, then what I'd like to do is only have windows installers, and not need to make .mod files any longer. Saves me some work, ugh.

On a related note, what's your strategy if a user unpacks their DLC (either via DLCEd2 or Texplorer) *then* decides they want to install MEHEM?

You've mentioned that you and KFreon have an ongoing discussion about this, and I'm aware you disagree with him. But, it seems to me that it's important everyone is on the same page about this stuff. Right now, KFreon and Ottemis are still telling everyone to scan on vanilla files. It's basically become a mantra, and anyone who spends any amount of time on ME3Exp knows this. It's also in the ME3Exp Set Up Guide that Ottemis mainstains. So, when users start to hear from someone else that they should scan on modded files, it's just going to create confusion, especially for new users.

Maybe WV should create a "private" section on the forum similar to what TM has (or something)? That way ME3Exp developers and major mod creators can talk amongst themselves and try to figure out the best way to handle difficult issues like this? I can make the suggestion to WV via PM if you like the idea. If you think it's not necessary, then that's fine, too. Just tossing the idea out there.

Here's the thread on ME3Exp, btw, where WV, JohnP and I have been talking about some of this.

And here is my reply. Sorry it got a bit lengthy :


Well, I don't see a reason to install content mods after you unpack your DLC at this point.

The main reason for unpacking the DLC is for texture and possibly mesh mods.These mods should be installed after story mods anyway.

KFreon (and Ottemis, I think, correct me if I ma wrong) say that they only support tree-scan of vanilla files. That only means that they cannot predict what will happen if you scan on modded files (which is fair enough).

I hope KFreon is ok with me posting this here but here is a quote from one of is pms on the matter:
 

As long as the replacing pcc has the same exports as the original, all will be fine. Clarification, the treescan looks for textures in the pcc's, and adds the file and the export ID inside the file, to the tree. So if you add entries to the export list, that won't work after scanning (i.e. Texplorer won't see anything new) but if you just edit entries, all should be well.

I bolded the important part here. tree-scan checks the pccs an the basis of their export lists and saves that data. As long as you mod your files before the scan, things should always work (provided that the modded file works in the first place).

Maybe it would be a good idea if it could be made a bit more clear (especially in any tutorials) that tree-scan on modded files is possible, just not supported by the texture modders and that people have to deal with problems that arise from this on their own or with our help.

What I can tell you for sure is that if you do the tree scan on a vanilla game and then install MEHEM (or CEM and probably also TM), you will break your game 99% of the time because all of these mods modify export lists and change pcc files without regard for the texture mods.

I hope this makes things a bit more clear. It is a tricky issue since you have multiple components interacting here, which don't really "see each other".

As for my strategy, regarding MEHEM, for now, I am rather contempt with the v0.4 installer as it is because it works if used properly and, as I said above, I don't really see the advantage of installing the mod on anything but a vanilla game before extracting DLC.

That said, if possible, I am happy to try and accommodate people with the the next version as much as possible. We need to have a discussion with wavion on this (haven't seen him in a while) but since I am mostly traveling next month, I won't be able to update anything essential until July or August (I know I keep procrastinating all things modding more and more but this year is just a bit crazy for me ).

Ok, sorry for that wall of text. Would be good to get Otte's input on his, too I guess.

[giftfish]

Did Kfreon clarify for you the issue about when/how the offsets get changed? This is the problem people always have when they scan on modified files, so it's an extremely important point.

 

If the offset associated with the texture changes and then the tpf files people download from Ottemis (or whomever) won't find the texture in the game files because the offset is different. Then folks jump onto ME3Exp complaining that this or that texture mod doesn't work, and KFreon/Ottemis spend hours and days helping them. Seriously, that is 90% of what they troubleshoot.

 

Upon scanning, does the offset only change if the texture they are trying to replace has been modded? Or, will the offset change if the pcc is modded at all (i.e., with a content mod)?

 

And yeah, Kfreon and Ottemis should definitely be a part of this discussion. They both have more knowledge about this than us, so we should probably bring it onto ME3Exp.

 

EDIT: Just noticed Ottemis is in the Dev group here, lol. Is KFreon here under a different handle?

[JohnP]

So I have good/big news on this front. I was able to add the JAM mod as DLC! So no modding of the Extended Cut SFAR, no overwriting of base game files. The files in the new "DLC" simply take precedence over the others. I need to do some more testing, but I just ran through JAM with this method and it worked perfectly.

 

I'll give some more details, but it's after 1am here and I'm tired. 

[giftfish]

The frack!?

 

Holy crap. That's sort of funny. As I was combing the coalesced in search of codex answers I found an 'Override" file path referenced left over from development (I'm guessing). Which got me thinking about how cool it would be if we could simply plop all these mods in an override folder like DAO.

 

While what your saying doesn't apply to main game files, even DLC would be a vast improvement. Do tell

[JohnP]

The thing is, it DOES apply to main game files. DLC files override files of the same name, regardless if they're in the base game folder or an earlier DLC.

 

I have a completely vanilla install and placed all my mod files in the new "DLC" folder, including the base game files I modded for Miranda, Jack, Kelly, and Samara to appear at the end. It all works with no copying over of any files. I just dropped the DLC I created into the BIOGame\DLC folder.

[MrFob]

That sounds really cool JohnP.

I am not entirely sure if that will solve our problem with the compatibility with texture mods though. Don't really know enough about texture mods to be sure, so here is the question:

When a texture mod is installed, what exactly happens to the tfc files and the pcc files? My guess is, one of these 3 things happens:

1. A new texture is concatenated at the end of an existing tfc and all the rferences in all pcc files are adjusted, so that they link to the new texuter. The old tecture stays intact this way.

2. A new tfc file with the new texture is adde to the game and the references in all pcc files is adjusted to reference the new texture. The old texture in the old tfc files remain intact and can still be accessed.

3. The new texture is put in place of the old one in tfc file. offsets for all textures in the files change and references in all pcc files have to be adjusted. The old offsets don't work anymore.

 

If either situation 1 or 2 are the case, than a new DLC should work very well. The only "downside" would be that the pcc files would still reference the old textures and therefore any texture mods would revert to the vanilla textures within the mod (which is probably not a big deal).

If situation 3 is the case, than there is a problem because the references in the old pcc files break the game.

 

I am a bit confused because on the one hand, I can't imagine that situation 3 is the case. It sounds too complicated to set it up that way. On the other hand, if situation 1 or 2 were the case and we could still access the old textures, shouldn't just overwriting the pcc files work?

Basically, introducing a whole new DLC is the same as pcc file replacement (like during a manual install of a mod), right?

[JohnP]

Number 2 is correct.

 

This method makes installing and uninstalling the mods much easier (IMO) but yes it will still cause textures to revert if the story mod is installed AFTER the texture mods. The user would still need to do a tree scan on the story mod so it would treat it the same as the rest of the DLC.

 

I'm about to run a scan now to test.

[giftfish]

Wow. Ok, so basically you are saying installation to the DLC folder acts as an override folder?

 

What about files in the \Movies and \Binaries folders, JohnP? Do those still need to be installed to their respective locations?

 

And do we know the answer to my question above about the offsets hashes? All texture mods installed with a tpf file via the TPF Tool require the user to have a specific offset hash for the texture in question (the vanilla value). So, will scanning on content-modded files change these offsets hashes or no? I guess you should be able to tell us shortly, JohnP.

 

EDIT: Got my terminology garbled. Corrections made and explained in post below.

[JohnP]

Wow. Ok, so basically you are saying installation to the DLC folder acts as an override folder?

 

What about files in the \Movies and \Binaries folders, JohnP? Do those still need to be installed to their respective locations?

 

And do we know the answer to my question above about the offsets? All texture mods installed with a tpf file via the TPF Tool require the user to have a specific offset for the texture in question (the vanilla value). So, will scanning on content-modded files change these offsets or no? I guess you should be able to tell us shortly, JohnP.

 

1. Pretty much yes.

2. Movies work the same way as PCCs. The DLLs will still need to go to Binaries. The only main game files you need to worry about are BIOGame.tlk and Conditionals.cnd. You can't put those in DLC; it doesn't work I tried.

3. I was under the impression that texture mods looked at the texture/mesh names and not raw offsets (I thought that was the whole reason of scanning the files: to build an index). If that's not the case, then that would pretty much nix the idea of installing them on top of content mods.

[MrFob]

Hey JohnP, about the tlk files, have you tried introducing a new tlk for your new "DLC"? I assume you have to give the new DLC some name (just like the EC is called DLC_CON_END for example). Let's say you call your new one "DLC_MY_DLC". Have you tried adding a DLC_MY_DLC.tlk or e.g. a DLC_MY_DLC.bin or something like that?

 

Well, maybe we shouldn't ask all this stuff one after the other and wait until you can compile some sort of tutorial. If you want to do that, just ignore my question. Looking forward to seeing this in action.

[giftfish]

2. Movies work the same way as PCCs. The DLLs will still need to go to Binaries. The only main game files you need to worry about are BIOGame.tlk and Conditionals.cnd. You can't put those in DLC; it doesn't work I tried.
3. I was under the impression that texture mods looked at the texture/mesh names and not raw offsets (I thought that was the whole reason of scanning the files: to build an index). If that's not the case, then that would pretty much nix the idea of installing them on top of content mods.

 
2. Awesome.
 
3. Ok, I'm sorry...it's the HASH. Apologies. It's the hash that is what changes when scanning on modified files, which is why I've been asking this question if content mods affect them for weeks now (or if it's just texture mods that change the hash).
 
TPF files are made using a def/log file. That file contains one or more file names that consist of the hash of each texture to be replaced. I know this because I have made them myself for ThaneMOD. I also know that if the hash doesn't match, due to scanning on modded files, then the TPF will work for the creator but not for others. I know this because I've done that, as well.


 

When you first load a TPF with TPF Tools it looks like this:


You can see the hash in the file name. That hash has to be the same for the person who makes the TPF and the person who uses the TPF. Once you hit "Analyze with Texplorer" then it locates the texture with the identical hash in the game files, and the window then looks like this:



If it doesn't match, then you will get an error in this window with message telling you what the problem is. My understanding is the whole reason Kfreon has developed TPF Tools is that tpfs are better to use than .mod files since .mod file can break with REVs. A TPF file will always work, regardless of rev, and if it's made with Texmod, it is compatible with both Texmod and ME3Exp.
 
As long as scanning on content modded files doesn't change this hash, then we are good

[giftfish]

Hey JohnP, about the tlk files, have you tried introducing a new tlk for your new "DLC"? I assume you have to give the new DLC some name (just like the EC is called DLC_CON_END for example). Let's say you call your new one "DLC_MY_DLC". Have you tried adding a DLC_MY_DLC.tlk or e.g. a DLC_MY_DLC.bin or something like that?

 

Damn good idea, Fob.

[MrFob]

-snip-

As long as scanning on content modded files doesn't change this hash, then we are good

Aaahh, that makes sense. MEHEM for example doesn't change texture, so that is fine. I think Thane mod has a few, right?
So that would cause problems then. Maybe for future releases of any content mod, it might be a good idea t split the mod into the non-texture part and a tpf part which contains the textures and can/should be applied after a tree scan?

[giftfish]

Aaahh, that makes sense. MEHEM for example doesn't change texture, so that is fine. I think Thane mod has a few, right?
So that would cause problems then. Maybe for future releases of any content mod, it might be a good idea t split the mod into the non-texture part and a tpf part which contains the textures and can be applied after a tree scan?

 

Yep, ThaneMOD has a few. So, the content mods don't change the hashes? Only texture mods do?

 

For ThaneMOD I have the cabin photo and the hoodie as tpf files. Both of these are in all likelihood going to be used by those installing the romance versions of the mod. And both are textures users might want to swap out regularly.

 

BUT, for anyone who uses EWAC (included in 5/6 install packages), we do change four GUI textures in the codex and war assets. These are textures that are housed in pccs and not tfcs. They are also textures that most people would never need to replace. So, simply including the texture-modded pccs actually saves some of our users a good amount of work, because then they don't have to go through all the Texplorer hoopla (lots of people are scared of ME3Exp, as you probably know). They can just run the installer update TOC.bin and go.

 

So, what I'm currently trying to decide is if I should still do it that way. If it's only the hashes to those 4 files that would be messed up in the user's tree.bin, it doesn't really matter unless they want to apply a new texture to those files, which is very, very unlikely.

 

Otherwise, yeah, I'd have to make a TPF, which is easy enough.

[giftfish]

Yep, ThaneMOD has a few. So, the content mods don't change the hashes? Only texture mods do?

 
Quoting myself here, since nobody has actually answered my question yet :]
 
First, I want to re-phrase this to make sure what I'm asking is clear. My question about this has never been in reference to ThaneMOD--it's always been in reference to the idea of scanning on content-modified files. And, more importantly, whether telling people to scan on content-modified files will result in breaking all tpf files out there. This is b/c KFreon and Ottemis have drilled it into my head (and the heads of many others) that scanning on modified files is bad.
 
So, that said, here's the rephrased question:
 
Do content mods that do not contain textures change any of theses hashes?
 
This may seem like a silly question, but I really don't know. I don't know what the hash is, therefore I have no idea if editing a file _in any way_ could change it. And, I've always been told that "scanning on modified files is bad." That advice has never been in the context of content vs texture modifications. Just scanning + modification = bad.
 
So, if someone could answer this for me, I would be very grateful. I'm honestly not trying to be a pain in the ass. I'm simply trying to make sure we aren't going to break anything for folks by telling them to do this, and so far, nobody has assured me that we won't.

[JohnP]

Hey JohnP, about the tlk files, have you tried introducing a new tlk for your new "DLC"? I assume you have to give the new DLC some name (just like the EC is called DLC_CON_END for example). Let's say you call your new one "DLC_MY_DLC". Have you tried adding a DLC_MY_DLC.tlk or e.g. a DLC_MY_DLC.bin or something like that?

 

Well, maybe we shouldn't ask all this stuff one after the other and wait until you can compile some sort of tutorial. If you want to do that, just ignore my question. Looking forward to seeing this in action.

 

I'm put up an overview of what I had to do here soon, but yes I did put all my tlk changes from the EC into a tlk specifically for this mod now. Actually, as I'll show in another post, you HAVE to have at least one tlk file present.

 

So for content mods, if you want to avoid modding the main game file, you could put all your changes in a new tlk and change the string IDs in the modded pcc files.

[giftfish]

So for content mods, if you want to avoid modding the main game file, you could put all your changes in a new tlk and change the string IDs in the modded pcc files.

 

Holy ****** crap. That could maybe solve ThaneMOD's tlk issue. Maybe.

 

Can't wait to see this post. I have tons of questions, lol.

[MrFob]

Yea, that would be really nice. This would solve all tlk and coalesced compatibility issues between different mods. Looking forward to it.

[giftfish]

Ok, so I sent KFreon a PM to clear up the issue of when/how hashes get modified. I think some of you simply understood more about this than I did, but I needed a clear yes/no answer for my own peace of mind. Kfreon was also nice enough to provide an explanation that didn't get overly technical. I asked his permission to repost his PM, so here we go.
 
 
First, here's a section of my initial PM to him for context:
 

-- Giftfish says:
I have a question for you that pertains to a discussion you've been having with Fob about when to scan files and build tree.bin in relation to content mods (before/after install).

I am well aware that "scanning on modified files is bad." If you and Ottemis had a dollar for every time you had to remind people of this, you'd probably be a very rich man, lol. However, when using this mantra of not scanning on modified files you have never differentiated between content mods and texture mods. You've always simply stated "don't scan on modified files." And, the reason why users shouldn't scan on modified files is because it can result in indexing the wrong hashes, so the TPF Tool won't be able to locate the texture to replace. Ok, fine. No problem.

And so here comes my question. It might sound stupid, but I need to ask since I don't know the answer:

Will a content mod that does not contain textures alter hashes in a file it edits?

I don't understand what a hash is, so I have no idea if editing a file _in any way_ could change them. And, because I do know that sometimes when you edit a file in other ways (not with textures) it can have a "domino effect" in the file and can change other things that you don't intend it to (offsets, etc).

 

 

And his response:
 

-- KFreon says:
Yes, I think about this one all the time. I have to keep reminding myself why we can't scan on modified files.

The reason I haven't differentiated is because I don't really know what you guys do with the rest of the toolset, as in I don't know how you change the files.

So just off the top of my head (I can look in the code if you really want me to), as long as you don't edit the basegame tfcs (other than appending), or directly edit the texture area of pccs, you should be fine.

Lemme explain how/why and maybe shed some light on both sides here.

What Texplorer does during tree scan, is to look through a pcc's Export list for texture entries. It then extracts the image pointed to by that entry and runs a hashing algorithm on it, some maths that gives a unique ID for that image. A pixel change to that image will change the hash immensely.

That's why you can't scan on modified files with textures, cos when it pulls the image out, its completely different (hash-wise) and while the scan will probably finish, all your hashes look different from everyone elses.

Now as long as the content mods don't modify the Texture exports directly, doesn't matter if they're moved around, scans should finish just fine and hashes should also be fine.

tldr: If basegame tfcs are unedited, and pcc texture entries are unedited, all should be well.

 
Yay :]

[Deager]

This seems to be the live topic regarding installers so I'll post here. Just a few questions and, as always, no pressure attempting to be put on anyone. I'm just trying to figure out wise use of time.

 

IHJ graciously offered to make an installer for CEM v1 for me; like what he did for the Harby module. But, before he would spend time on that...

 

JohnP, your JAM method is awesome. And in that thread you have the entire thing pretty much laid out. Do you know if you'll want to/be able to offer a universal jury-rigged AutoTOC's source code to build the TOCs? It sounds like that's the last thing needed before others can utilize this way.

[giftfish]

@Deager --

 

I'm pretty certain that you don't modify the cnd, tlk, or coalesced for the main game, but I still have a lot of questions about that stuff. They are the 3 files that seems like you can't actually put the vanilla file name version in the DLC, you can only have the DLC version of the file. If that's the case, then I see these potential issues:

 

1. Coalesced.bin. DLC coalesceds lack most of the sections found in the main game coalesced, and it also seems that certain things might work differently. TM modifies a few different sections, and I would need to make sure that I can add all of them to the DLC coalesced and still have them work.

 

2. CND. TM adds new conditionals and alters vanilla ones. I would need to make sure that both types will still work if it's all moved into a DLC cnd.

 

3. TLK. This is the big problem. Right now, JohnP doesn't know if strings in the vanilla tlk can be used in the DLC tlk, resulting in the overwriting of the vanilla text. When I edited Huerta, there were strings the game wouldn't let me use when I was redoing the dialogue wheel. In addition, the FaceFX is what links to the audio, so the string IDs have to be the same for the tlk, the dlg file, the FaceFX, the VO_Play file, and the audio file. This is vital when it comes to adding new lines of dialogue, which is something I did, but I don't think JohnP or you did. I had to "steal" audio slots from elsewhere in the same conversation that wouldn't be used for those romancing Thane. Then, I had to clone face animations and link them with the relevant strings and make sure everything lined up. At this point I'm not sure I can get that all to happen using a DLC tlk. It would also mean I really have to redo the entire conversation from scratch...yikes.

 

Complications aside, wow, installing JAM yesterday was a piece of cake. It would be awesome if that could become that standard method of installing content mods. But the need to hex edit and ensure communication between all mod authors seems somewhat problematic. For large mods, I guess it's pretty likely we'd all be talking to each other on ME3Exp, but for small mods, authors would likely just stick to PCC overwriting the main game. It almost seems like we're getting to the point that someone needs to build a mod manager for ME3. A tool that will monitor conflicts and allow the user to select which file to use. Then it would be up to the authors to post on their Nexus page, or whatever, how to handle it. That tool could also have a built in TOC/AutoTOC/Check&Rebuild sfar feature. Installers then really wouldn't be necessary. The user would simply download the zipped mod, the mod manager would handle proper placement of the files. Meh, I'll stop rambling now, lol...

[MrFob]

Reading giftfish's post about modder communication got me thinking: we might want to get this kind of more general discussion over to the me3exp forums, so that we don't miss out on keeping everyone in the loop (after all, this group was mainly set up for beta testing specific projects like JAM or Harby).

 

Oh btw, I also invited AVPen. If anyone has more suggestions for members, let me know (not sure if only I can invite people or if all of you can).

 

On topic: I know that in regular DLCs, tlk strings in the DLC tlk overwrite the main game ones. Whether that also works for mod DLCs, I haven't had time to test yet.

Also, I had no problem adding new section to DLC.bin files (e.g. the DLC_CON_END.bin did not have an email section before I edited it to add the MEHEM emails). I don't really know of that works for all section types but I don't see why not.

 

Also, thanks for posting that pm with KFreon. It's great to finally have a definitive answer n how texture mods work with content mods.

[giftfish]

Reading giftfish's post about modder communication got me thinking: we might want to get this kind of more general discussion over to the me3exp forums, so that we don't miss out on keeping everyone in the loop (after all, this group was mainly set up for beta testing specific projects like JAM or Harby).

 

Oh btw, I also invited AVPen. If anyone has more suggestions for members, let me know (not sure if only I can invite people or if all of you can).

 

On topic: I know that in regular DLCs, tlk strings in the DLC tlk overwrite the main game ones. Whether that also works for mod DLCs, I haven't had time to test yet.

 

Also, I had no problem adding new section to DLC.bin files (e.g. the DLC_CON_END.bin did not have an email section before I edited it to add the MEHEM emails). I don't really know of that works for all section types but I don't see why not.

 

Also, thanks for posting that pm with KFreon. It's great to finally have a definitive answer n how texture mods work with content mods.

I was thinking that as well, Fob. We need to keep everyone in the loop, though I'm guessing JohnP is waiting to announce all this DLC mod stuff until he got a few more things worked out.

 

Ok, thanks for the info about the tlk strings and coalesced. I'll have to look into it, but it's making my brain hurt just to think about it. Huerta was complicated to get to work right, and I think trying to relocate it (or whatever) is just going to be even more complicated.

 

Kfreon is such a nice guy. We often get our wires a bit crossed when we communicate, but he's pretty patient with me. Sorry if I came off like a broken record with all that stuff, I kept asking the same questions, I just really wasn't sure what was going on with it. All sorted now though :]

 

Oh, and the TM codex bug is solved! Big discussion between TankMaster and I on Me3Exp. I just updated it with some nicely organized info, as well.

[Deager]

Waiting for JohnP seems like a good idea. I was messing around with his method a bit and while it may be my lack of understanding, I don't have the Pack003 version of CEM working yet.

 

Getting on the same page on me3explorer is a great idea. There's less than a dozen who actually make modifications right? Shouldn't be too hard to coordinate (famous last words.)

[giftfish]

Shouldn't be too hard to coordinate (famous last words.)

 

Dude, you just jinxed the **** out of us >.>