192 réponses à ce sujet

[Mephales]

Thanks. A clever solution in my book.

[Mephales]

0.5.2 out, no need to upgrade really, unless you want an easier way to export the known filenames and file extensions of ERF resources.

Edit: And then 0.5.2a came out, because I was a bit too aggressive in fixing one bug (em-dashes in ECStrings caused it to blow up) and made it worse (Essentially all binary data caused it to blow up). Fixed.

Modifié par Mephales, 16 mars 2011 - 02:12 .

[Thought Process]

Merged in your most recent fnvhashes (as of this post) with my lists, name completion now sits at 53% (was at 43% before I merged yours in).

http://svn.gib.me/pu...iles/status.txt

(my RebuildFileLists generates this status file

[Mephales]

Thought Process wrote...

Merged in your most recent fnvhashes (as of this post) with my lists, name completion now sits at 53% (was at 43% before I merged yours in).

http://svn.gib.me/pu...iles/status.txt

(my RebuildFileLists generates this status file

The credit goes to tazpn, not me.

0.5.2a uploaded, small change: Can now select multiple files to export from File > Export. And it asks if you want to overwrite existing files or not.

[Thought Process]

Started working on a CNV viewer/editor, not sure if I'll get anywhere with it, but it's useful to look at plot handling in conversations.



The main issue is that LINE structure has an unknown index into some table -- presumably of a cutscene/etc, but I can't find a reasonable place for this (CNV ("Conversation") are just lines / links, they branch out into CL ("Conversation Line") files for cutscene information).

Modifié par Thought Process, 19 mars 2011 - 03:29 .

[Thought Process]

Oh, it's also worth noting that I'm constantly updating my file lists with more file names, I've added like 30,000+ names since I imported your most recent one from tazpn.

[tazpn]

fnvhashes-20110318-tzupdate.7z

A merged version of the database with gibbed files hashes and a handful I've collected through playing.  Also included an updated unknowns db for my hash gatherer posted earlier.  Count is up to 137914 I think mostly to Thought Process this time.

[Thought Process]

Merged your new update into my lists, you added 11,876 names. Now sitting at 165526/220229 (75%).

Nice work on the DDS texture names.

[Thought Process]

Wrote a parser for sound bank files (*.bnk) to pull out ogg file names, this added 34093 file names, nearly all of ogg file names.

199619/220229 (90%)

[daywalker03]

Thought Process wrote...




The main issue is that LINE structure has an unknown index into some table -- presumably of a cutscene/etc, but I can't find a reasonable place for this (CNV ("Conversation") are just lines / links, they branch out into CL ("Conversation Line") files for cutscene information).

Look at the crf files (if I remember the extension correctly); I believe those are interconnected with the cnv files (one is the actual dialogs, the other is for the wheel, I believe).

[Mephales]

I've been thinking of a conversation viewer, but my first attempt was unwieldy. I'll probably just settle for making some sort of crude HTML export for myself. I'll stick with maintaining my editor. I haven't even finished Awakening yet, after all, much start the DA:O DLC or DA2.

[Thought Process]

That's not the extension, and I'm not sure what extension you could mean, so please look it up (and perhaps give an exact example).

[Mephales]

daywalker03 wrote...

Thought Process wrote...

The main issue is that LINE structure has an unknown index into some table -- presumably of a cutscene/etc, but I can't find a reasonable place for this (CNV ("Conversation") are just lines / links, they branch out into CL ("Conversation Line") files for cutscene information).

Look at the crf files (if I remember the extension correctly); I believe those are interconnected with the cnv files (one is the actual dialogs, the other is for the wheel, I believe).

Well, the cln_<cnv name>.crf files hold the .cl for that particular conversation, and the .cl filenames are just <cnv name><tlkid>.cl. I believe Thought Process is referring to the field 30200, which is appears to be a small number... I think it's not a cutscene id, but an actor/name...

Hmm, I just realised I can probably generate names for all .cl from the .cnv.

[Thought Process]

I did that already. The ones missing are due to lack of the cnv name. And huh, I never noticed those CRF archives before, oops. I guess there are more file names to be discovered!

Edit: that added 62843 files. All of which are .cl and .fxe files -- that explains the low number of CL/FXE files.

Modifié par Thought Process, 19 mars 2011 - 03:18 .

[Mephales]

Well, I checked and you seem to be missing a lot of .cl names, so here's what I generated: http://www.sendspace.com/file/zlrsbb

[Thought Process]

Yeah, I hadn't gotten around to doing .cl's yet, I did go and grab all FXE names though.

Thanks.

Edit: 27734 new files from your list.

269322/283072 (95%)

Modifié par Thought Process, 19 mars 2011 - 04:05 .

[tazpn]

Congratulations on the huge update.    I've tried to sync but missing quite of a few of the files you report.  Anyway, I've added some gpu dds,  gad and evt and others after fixing a bug in my code.  That bug also means I want to do a clean rebuild of the hashes to makes sure I haven't inserted some incorrect mappings in the past.

fnvhashes-20110319-tzupdate.7z

Modifié par tazpn, 19 mars 2011 - 05:41 .

[Thought Process]

That's what my RebuildFileLists tool does, and why I wrote it. To ensure consistency of my file lists.

Note that my status lists include duplicates of names across ERFs, which is probably what inflates the numbers. My file lists are stored on a ERF basis rather than globally (even though I load them this way).

Edit: merged your new list in, 4493 new file names.

273815/283072 (96%)

Edit 2: that reminds me, the fnvdump list that's in the pygff project now had merged in comments from my lists, if that's your doing you may want to skip lines from my lists that start with ';'.

Edit 3: also, given that file names are hashed lowercase, it's probably best to store all the names in the lists lowercase if you are not already doing that.

Modifié par Thought Process, 19 mars 2011 - 06:02 .

[Mephales]

Well, tables in the database are defined as case-insensitive, so while they're not being stored in lower case, they are being indexed correctly.
And yeah, after forgetting to remove the ; comments, I modified my program to skip them, though only if the first line starts with a semicolon.

Edit: Whelp, there goes to neighborhood. No more hosting fnvhashes here, it looks like.

Modifié par Mephales, 19 mars 2011 - 06:19 .

[Thought Process]

That's fine, I do have multiple lines of comments at the start but I'd never add them mid-file.

[Mephales]

Well, it's not as if it's difficult to purge them after all... Just a simple delete from files where string like ';%';

[daywalker03]

Thought Process wrote...

I did that already. The ones missing are due to lack of the cnv name. And huh, I never noticed those CRF archives before, oops. I guess there are more file names to be discovered!

Edit: that added 62843 files. All of which are .cl and .fxe files -- that explains the low number of CL/FXE files.

Everyone makes mistakes; there are so many new things in DA2 that missing the crf files is forgivable.

[tazpn]

My person preference is to not store lower case because that is not the case of the original filenames and I would rather be closer to the originals. I would expect the code using the database to do case insensitive compare. LIKE clauses should already be case-insensitive if you are doing the reverse lookup.

For reference, my bug was a specific case where I was searching for alternate extensions for a filename it could insert the name without the extensions as the hashed value. Not sure if it ever did though. The other one was because I used scanf at one point it was splitting paths at spaces and adding as additional hashes. Not harmful since the hash was correct just not necessary.

Anyway, I've cleaned up my local one and am just resyncing with the gibbed file list and will repost that soon. It is at 232069 with 8196 unknowns.

fnvhashes-20110319-1850-tzupdate.7z
fnvhashes-20110320-0630-tzupdate.7z : 6531 unknowns left

fnvhashes-20110320-1930-tzupdate.7z: 5413 unknowns left.  (Mystery extension 8dd70fe1 = assembly)

Modifié par tazpn, 20 mars 2011 - 07:39 .

[Thought Process]

Aha! Great to finally have that one down, how did you find it?

[tazpn]

I recently just ran my string scouring tool on all the unpacked files and it found a match somewhere (though didn't record where). The bug posted about above probably prevented me from seeing a lot of strings it could have matched like this earlier. Oh well.

Havent looked at your list recently but guessing most of the remaining files are the location specific texture files (hlac,llac,lla,hla) and then some related to the handful of models and animations. I wouldnt be surprised to find that some are just unreferenced files that we might never guess which I suppose is ok with me since they are unreferenced.