423 réponses à ce sujet

[OldTimeRadio]

Here is a simple scenario that illustrates why taking any model or batch of models from CEP (or anywhere else) and just running them through CleanModels is going to risk being a boondoggle.  This is not, in any way, meant to undermine the usefulness of CleanModels.  It can be a very useful tool, but it's not necessarily useful in all situations and, frankly, the results from CleanModels can be very misleading in evaluating the quality of a model. 

This misleading evaluation thing?  It's not CleanModels or OldMansBeard's fault.  This is a perceptual flaw on the part of many of its users.  It's a very easy mistake to make.

1. Make  simple teapot placeable.  Here's how I did that, and the rest on this line is only important if you want to reproduce my work: Make an aurorabase.  Make a teapot in GMax (Radius:250, Segments 4).  Move to X, Y 0.0, Z 250.0.  Drag and drop a texture on it.  Convert the teapot to editable mesh.  Add an Aurora Trimesh modifer with whatever settings you like.  I use Render and Override Material Values only.    Attach it to the model base and export.  I export geometry only, in this case. 
2. Run it through CleanModels.  Outside of the default settings, I'm doing a binary snap because this model is headed for the compiler when it's done.
Result:
plc_a01.mdl loaded.
  binary snap applied to node positions and mesh vertices
  vertices welded in [teapot01]
  null faces deleted from [teapot01]
  unused tverts deleted from [teapot01]
  tverts welded in [teapot01]
  renormalized tverts in [teapot01]
  Fixes made = 11073
plc_a01.mdl written.

Total Fixes = 11073

Okay, 11,000 fixes.  Are there really 11,000 errors that need fixing in that simple placeable?  No, of course not.  Are there 11,000 operations that CleanModels has carried out against that model which were at variance with the settings it was run with?  Oh yes, I have little doubt of that. 

However, 11,000 CM3 fixes ≠ 11,000 actual errors of signifigance.

Are some of those errors signifigant?  I'm sure, in some way, they are or could be argued to be.  Are those errors noticable in-game and/or do they have a negative impact on the Aurora engine?  That is more up for debate, IMO.  But that's far from the point I'm trying to make, which can only be evinced by compiling these models and having CleanModels analyze them again.

Before we do that, though, let's take the outputed model from CM3 and run it through CM3 again, with the same settings.  Looks like there are still some errors left, according to CM3:
plc_a01.mdl loaded.
  binary snap applied to node positions and mesh vertices
  renormalized tverts in [teapot01]
  Fixes made = 444
plc_a01.mdl written.

Total Fixes = 444

Again are there really 444 errors of signifigance in this model?  No, there aren't.  You can keep taking the outputted model and feeding it back into CM3 with the same settings and you're going to get a notification that there were 444 fixes performed each time.

Even all this so far, as I said, is not my main point.  Let's take that model, that CleanModels produced, and compile it with various compilers and then have CleanModels analyze those binarized models:

Internal BioWare compiler results: Total Fixes = 9271
DLA Model Compiler: Total Fixes = 9271
Acaos mdlcomp from Explorer Reborn 1.63: Total Fixes = 9271

After compilation, all three are reported by CleanModels to have gone up 20x the number of errors that needed to be fixed.  Interestingly, all by the same amount. 

What's actually going on here?  What does all this mean?

It means: If you load a model into CleanModels on the hope of looking for errors, you will likely not be disappointed.  You will likely see hundreds if not thousands of errors, just like you expected.  Using those numbers to support any kind of assessment of the quality of the model is often specious.  Only a relatively skilled modeler can actually make that assessment, especially with complex geometry.  The teapot placeable I made compiled just fine with the Bioware model compiler and worked fine in-game.

With the NWN2 models, and with basically anything else in the existing CEP package, I think the main questions should be:
* Is this model reported to have a specific error of some sort in the first place?

If so:
* Does the model compile well?
* Does it look good in game?
* Is there anything about it that taxes the engine, unnecessarily, and needs to be optimized?

CM3 is a powerful and useful tool, but feeding models into CM3 can lead to unintentional trips down the rabbit hole if one isn't careful.  Worse, its output can be misinterpreted to call into question the quality of models which are, at least in this case, more or less perfectly fine.

Modifié par OldTimeRadio, 27 janvier 2014 - 11:27 .

[The Amethyst Dragon]

Thanks, OldTimeRadio. Very insightful.

I was using it mainly because of reported ambient/diffuse settings being too dark on some of the models. It also welded some vertices for me.

[omen_shepperd]

 I read back on page 1 or 2 where you could use some
blueprint builders to help make ERF’s. That is something I can do and would be
very willing to help out with. I wanted to know if anyone else is making blueprints
as well so I did not make any that conflicted with ones already made. 


I will join the Wiki page and hopefully I can help out in other ways while I am
learning Hak work and scripting. 

[FunkySwerve]

omen_shepperd wrote...

 I read back on page 1 or 2 where you could use some
blueprint builders to help make ERF’s. That is something I can do and would be
very willing to help out with. I wanted to know if anyone else is making blueprints
as well so I did not make any that conflicted with ones already made. 


I will join the Wiki page and hopefully I can help out in other ways while I am
learning Hak work and scripting. 

Ideally, blueprints would be generated automatically by a perl script. That's what acaos did for all the places we made for 2.3. Much less room for pebkac.

Funky

[FunkySwerve]

And here's the script:

//
// --------------------------------------------------------------------------------
// This script generates static placeables from placeables.2da.
// --------------------------------------------------------------------------------
//
//   #!/usr/bin/perl -w --
//   #
//
//   use strict;
//
//   use NWN::Gff;
//   use NWN::GffRead;
//   use NWN::GffWrite;
//
//   while (defined(my $line = <>)) {
//     my ($app, $static, $name) = split(/\\s+/, $line, 3);
//
//     next if ($name =~ /\\*\\*\\*\\*/);
//     $name =~ s/"//g;
//     $name =~ s/\\s*$//s;
//     $name =~ s/\\s*\\([A-Za-z0-9]+\\)\\*$//;
//     $name =~ s/\\s*\\*$//;
//
//     my $res = sprintf("plc_static_%05d", $app);
//     my $utp = undef;
//
//     if (-f "temp0/$res.utp") {
//       $utp = NWN::GffRead::read('filename' => "temp0/$res.utp");
//     } else {
//       $utp = NWN::GffRead::read('filename' => "temp0/plc_static_00001.utp");
//
//       $utp->{'Tag'}            = $res;
//       $utp->{'TemplateResRef'} = $res;
//       $utp->{'PaletteID'}      = 4;
//     }
//
//     if ($name =~ /VFX: Flame/) {
//       $utp->{'PaletteID'} = 231;
//     } elsif ($name =~ /Lightshaft/ || $name =~ /Shaft of Light/) {
//       $utp->{'PaletteID'} = 230;
//     } elsif ($name =~ /Portal [0-9]/) {
//       $utp->{'PaletteID'} = 232;
//     }
//
//     $utp->{'Appearance'} = $app;
//     $utp->{'Static'}     = ($static == 1);
//     $utp->{'LocName'}    = { '0' => $name };
//
//     NWN::GffWrite::write($utp, 'filename' => "temp0/$res.utp");
//   }
//
// --------------------------------------------------------------------------------
//
//

Funky

[FunkySwerve]

Er, sorry for spam posting. Code is acaos'. It's also still in nwscript-commented format, as that's taken from our mod, and we didn't want it throwing compiler errors.

Funky

[Pstemarie]

OldTimeRadio wrote...

...CM3 is a powerful and useful tool, but feeding models into CM3 can lead to unintentional trips down the rabbit hole if one isn't careful.  Worse, its output can be misinterpreted to call into question the quality of models which are, at least in this case, more or less perfectly fine.

This is exactly why I use CM3 primarily for tiles and to fix specific errors or perform specific tasks. As OTR points out CM3 is VERY good at what it does, but it is not perfect (although its pretty close). In the example OTR used to prove his point, I have little doubt that on the second trip through CM3, those 444 errors were 444 operations CM3 performed becasue the settings FORCED it to, not because it HAD to apply those fixes.

All this being said, I would like to add that CM3 was designed for tiles and was not originally intended to work with other model classes. When it was discovered that it COULD work with other model classes, OMB added the parameters to allow it to do so. What this means to me: if you run a model through CM3 the first time and CM3 says it has 10,000 errors, that model had 10,000 errors. Simply put - CM3 doesn't lie, but it can get confused if you keep running a model through it, and, quite frankly, if CM3 does keep finding errors, you might want to look at the model and what CM3 might be trying to tell you.

Modifié par Pstemarie, 28 janvier 2014 - 01:06 .

[Zarathustra217]

Regarding CM3, the main cause of "errors" is simply a matter of how you decide to round all floats (the "snap" option). The best method here really is binary, at least for the final product.

I'd also recommend disabling the "Allow Splitting" option when doing mass operations. Most shadow issues (at least the significant ones) are already solved by welding and re-pivoting.

Anyway, I'd like to reiterate the concept of making overhauls of some of the current CEP content (as I suggested above) - any scepticism to that?

Modifié par Zarathustra217, 28 janvier 2014 - 12:29 .

[Pstemarie]

Zarathustra217 wrote...

Regarding CM3, the main cause of "errors" is simply a matter of how you decide to round all floats (the "snap" option). The best method here really is binary, at least for the final product.

I'd also recommend disabling the "Allow Splitting" option when doing mass operations. Most shadow issues (at least the significant ones) are already solved by welding and re-pivoting.

Anyway, I'd like to reiterate the concept of making overhauls of some of the current CEP content (as I suggested above) - any scepticism to that?

Grear points about CM3. Another thing that helps with the snap option is making sure that the texture size you're snapping to corresponds to the texture size you're using. Just a little tidbit OMB told me when I first started using CM3.

Overhauling models is definitely a good idea and an update - which is what TAD is working through now - offers the best medium through which to do so.  I'd say that, as TAD noted in an earlier post, if someone sees something that needs an overhaul, feel free to do it, and email the work to the address he provided.

The only problem at this point is tracking the stuff, but I see people are already exploring that as well. Tracking progress could be as simple as creating an excel sheet that lists all the CEP assets and authors can note what they are working on. However, I'm not sure if there is some utlilty out there that can port a ak file's content list to an excel sheet and how hard it would be to create one if TAD wanted to do the list idea.

[Tiberius_Morguhn]

One question - and it's something I've noticed on TAD's wiki - why are tilesets being proposed as additions to CEP? I may be in the minority opinion on this, but I stopped at CEP 2.1 for my own projects because of the addition of tilesets and other overreaching content that didn't really fit.

[Zarathustra217]

Tiberius_Morguhn wrote...

One question - and it's something I've noticed on TAD's wiki - why are tilesets being proposed as additions to CEP? I may be in the minority opinion on this, but I stopped at CEP 2.1 for my own projects because of the addition of tilesets and other overreaching content that didn't really fit.

I somewhat agree, and it does add to the amount of work involved with maintaining the CEP. And after all, the CTP package was made to fill out that role. Perhaps they - along with the other addons - should be made into seperate downloads, and possibly only as a legacy thing. Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even).

Anyway, so far, I've been tracking down models that have improper lighting settings and fixing them manually in the files, but if we use conservative settings for CM3, perhaps we could allow ourselves to pass all models through it? If processing time is a worry, I've got a reasonably fast CPU. It would still require some testing, but I imagine a beta-phase before release could catch most errors (given they arise) if we get enough testers.

As for overhauling - perhaps we should add a new page to the wiki for suggestions of overhauls? (like the current for suggestion new content).

[MerricksDad]

Pstemarie wrote...

However, I'm not sure if there is some utlilty out there that can port a ak file's content list to an excel sheet and how hard it would be to create one if TAD wanted to do the list idea.

I was just in the process of exploring HAK, ERF and MOD formats for use with my AuroraExplorer. I could easily create an exporter to a tab or space delimited file, such as CSV. Excel and OpenOffice should be able to open that just fine. This isn't top on my priority list, but it does need finished to have this explorer complete.

[Proleric]

Given that tilesets are already in scope for CEP 2.4, I've suggested some (hopefully non-controversial) fixes and additions on the wiki. However, for many reasons, it might be wise to defer consideration of the more recent tilesets for the time being.

[Proleric]

Zarathustra217 wrote...

...Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even)...

A game of two halves.

I've offered in another thread to share my version of the old CEP horses that works with the 1.69 phenos and horse scripts (subject to agreement on whether scripting is in scope for future releases). That would make the CEP mount phenos redundant, going forward.

However, those phenos would have to be retained for backward compatibility; at most, we could freeze further development. Likewise, there are other phenos which remain useful, such as swimming/flying, and of course the Bioware-compatible phenos for skeletons and custom races.

[The Amethyst Dragon]

Zarathustra217 wrote...

Tiberius_Morguhn wrote...

One question - and it's something I've noticed on TAD's wiki - why are tilesets being proposed as additions to CEP? I may be in the minority opinion on this, but I stopped at CEP 2.1 for my own projects because of the addition of tilesets and other overreaching content that didn't really fit.

I somewhat agree, and it does add to the amount of work involved with maintaining the CEP. And after all, the CTP package was made to fill out that role. Perhaps they - along with the other addons - should be made into seperate downloads, and possibly only as a legacy thing. Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even).

Anyway, so far, I've been tracking down models that have improper lighting settings and fixing them manually in the files, but if we use conservative settings for CM3, perhaps we could allow ourselves to pass all models through it? If processing time is a worry, I've got a reasonably fast CPU. It would still require some testing, but I imagine a beta-phase before release could catch most errors (given they arise) if we get enough testers.

As for overhauling - perhaps we should add a new page to the wiki for suggestions of overhauls? (like the current for suggestion new content).

Proleric1 wrote...

Given that tilesets are already in scope for CEP 2.4, I've suggested some (hopefully non-controversial) fixes and additions on the wiki. However, for many reasons, it might be wise to defer consideration of the more recent tilesets for the time being.

As far as tilesets go, I think the community has made a lot of better ones over the years.  Luckily, the ones already included in the CEP are optional for builders to use.  Fixes for issues some of them have would always be welcome.

Proleric1 wrote...

Zarathustra217 wrote...

...Maintaining all the various new phenotype is also a management hell the prior CEP team brought upon themselves that I think we could benefit from leaving behind (the 1.69 phenotype system made the method they use redundant even)...

I've offered in another thread to share my version of the old CEP horses that works with the 1.69 phenos and horse scripts (subject to agreement on whether scripting is in scope for future releases). That would make the CEP mount phenos redundant, going forward.

However, those phenos would have to be retained for backward compatibility; at most, we could freeze further development. Likewise, there are other phenos which remain useful, such as swimming/flying, and of course the Bioware-compatible phenos for skeletons and custom races.

I'm going to be sorting through the CEP's pheno1 to pheno5 haks to sort out which files belong to "normal" (phenotype 0-8) brownies and wemics, if any, and move them to the core haks.  I'd do the same looking for any robes with phenotypes 0-8 and pull them from the pheno haks, moving them into the core haks.

This should leave it so that the pheno1-pheno5 haks can be completely optional.

Proleric1: If you want to contribute a version of the old CEP horses that works with the new system, I think that would work out great, and would keep those horses from being completely outdated.

[Zwerkules]

The Amethyst Dragon wrote...

As far as tilesets go, I think the community has made a lot of better ones over the years.  Luckily, the ones already included in the CEP are optional for builders to use.  Fixes for issues some of them have would always be welcome.

I agree with Tiberius that tilesets shouldn't be included in the CEP. Those that are already in it of course have to stay in it and they need to be fixed because some of them have tons of tiny gaps and the like. I can't promise anything, but maybe I'll have time to at least fix the forest tileset.

[leo_x]

Pstemarie wrote...

The only problem at this point is tracking the stuff, but I see people are already exploring that as well. Tracking progress could be as simple as creating an excel sheet that lists all the CEP assets and authors can note what they are working on. However, I'm not sure if there is some utlilty out there that can port a ak file's content list to an excel sheet and how hard it would be to create one if TAD wanted to do the list idea.

Tools like that can be made pretty trivially with the libraries I mentioned above. Here is a small script that dumps filename, file size, sha1, originating hak to a csv file: hak_to_csv.py It would be just as simple with nwn-lib as well. I truely believe people having something to make adhoc tools/scripts that anyone can create would be so valuable to a project like this, Funky's post above is a great example.

I dumped the output sorted by file name on my dropbox in case it might be of use to TAD or anyone else looking to track down a file or whatever (I didn't do all the old top haks): csv file (it's 12mb so too big for Google Docs)

Modifié par pope_leo, 28 janvier 2014 - 08:49 .

[MerricksDad]

haha, I was just doing the same

I was doing a table of filename <tab> type of file <tab> file it came from, and then running it for all cep prefixed haks so you could sort it by what hak it belongs to

I also agree some tools that let ANYBODY make new content would be a wonderful addition to the community. These kinds of tools should have been around since day one.

[Proleric]

The Amethyst Dragon wrote...

Proleric1: If you want to contribute a version of the old CEP horses that works with the new system, I think that would work out great, and would keep those horses from being completely outdated.

Will do.

ROSALIND THE UNSEEN: Once, Proleric wrote a simple story about a girl and her pony, but the only pony in the magic kingdom was bound in a huge mushroom called a cep. So Proleric ventured into its depths, where, after many adventures, he rescued the pony. Then he thought, I could free all the other poor creatures; but all the while, he glanced over his shoulder, fearful of the creeping scope in the darkness.

[Carcerian]

Zwerkules wrote...

The Amethyst Dragon wrote...

As far as tilesets go, I think the community has made a lot of better ones over the years.  Luckily, the ones already included in the CEP are optional for builders to use.  Fixes for issues some of them have would always be welcome.

I agree with Tiberius that tilesets shouldn't be included in the CEP. Those that are already in it of course have to stay in it and they need to be fixed because some of them have tons of tiny gaps and the like. I can't promise anything, but maybe I'll have time to at least fix the forest tileset.

Adding walkable beds would be nice, and more clean rooms, fixes for minimaps, like the diagonal tile errors and missing maps, and walkmesh errors, such as on the city docks. More basic options like boats etc would be popular too. Or for example upgrading the CODI subset of CEP City Extrior to the new version

IF CEP does ever add any new tilesets they should be small and versitile, such as worm's fantasy interiors (a favorite example as a tiny tileset with lots of options)

I would think a lot of people would love to see the lower grade and buggy content fixed (lizard TGA flips), pulled/padded or better yet replaced with modern content... Examples = all of Hardpoints crashing dragons, a lot of the visually buggy NWN2 Walls, the infamous Stone Altar, Dolphin fountain, many animals with bad animations, and the like. IF the content is broken, pull it, it frees space.

To keep from breaking old modules placeholder content could be used for those upgrading to newer cep, such as badgers or penguins.  for bad plcs use the spinning CEP logo, a vfx, or an invisible object...

With the advent of tail based scaling in 1.69, all the size ranges of monsters, esp dragons, are basicly an obeselete file hog as well...

Personally, I'd love to see a new CEP 2.6 fully backwards compatible, and a later perhaps a new CEP 3.0 designed from the ground up, remaking the old content, in the spirit of CEP, but a "reboot" based off of modern content first, keeping the best of both worlds.

As a note, you can add **** before "[Todo] name" and the content will no longer show up in the toolset or client, but will still be there in the 2das to find and fix or weed out later... (and will crash proof both toolest and gameplay, as bugged content will no longer be loaded)

Modifié par Carcerian, 29 janvier 2014 - 08:26 .

[Tyndrel]

I would be pleased to see a greater choice of tilesets within an optional CEP section, but any additions or improvements to CEP will be welcomed by builders and players alike. Thanks TAD, thanks all.

[Zarathustra217]

I ran through all the files from cep_core0-7 and cep_crp through CM3, and used the additional feedback it gave to do a few hundred other minor fixes. Most of it is somewhat minor (roughly 7k models had improper lighting settings). I've forwarded it to TAD, but if anyone wants to check it out, you can grab it here:

(link removed)

/Update:

Even with the most minimal settings, it turns out I'm just making more new issues than I'm removing old by using CM3.

I'll keep toying with it, but had to pull back the above link for now.

Modifié par Zarathustra217, 29 janvier 2014 - 08:30 .

[The Amethyst Dragon]

The question of naming the top/base hak for versions 2.60 and forward came up on the wiki, so I added a poll to get the opinions of more community members.  Feel free to read the comments, add your own to the discussion, and/or vote.

http://nwncep.wikia....e_Plan#Hak_Poll

[Lazarus Magni]

3 Cheers! This has been a very long time and coming. Hope it can come to fruition.

[Tarot Redhand]

Of course for the lazy amongst us this is a mixed blessing. For a couple of years we haven't had to update our 2das in order to say that our cc offerings are CEP compatible:D. Now we might actually have to dust off the tools we used for doing that :wizard:and actually use them again! Why is it that even the laziest of us have to do some work?

Kidding aside good work so far.

TR