23 réponses à ce sujet

[Allan Smith]

Dec 10 Update:
A new version of the toolset has been posted for download.  This is not a patch, it is a brand new installer.  What this means is that YOU WILL LOSE YOUR EXISTING WORK IF YOU DO NOT FOLLOW THE STEPS LISTED AT: http://social.biowar...abase_migration

If you are upgrading from version 1.00 of this toolset and have database content that you wish to migrate into the new version, you will need to follow a special sequence of steps to preserve it.  Builder-to-builder files (extension DADBDATA) are not compatible between the first released version of the toolset and subsequent versions. See http://social.biowar...abase_migration for more details.


Please read the information found at http://social.biowar...ing_the_toolset for more details about how to install the toolset.

Oh, and by the way, this version contains the full single player database resources!

end of update


We are working on a both a new toolset installer and patch to work on the following major issues:

• Discourage/prevent the installation of the toolset on a Windows XP SP2 machine, in order to prevent the SQL Server reboot chaos that happens. (SP3 on Windows XP is the minspec)
• New core resources database with plot GUIDs that are consistant with the ones released with the game so that running the toolset does not mess with your save games.  This is the "Click Here if the Toolset Broke your DA:Origins Game" thread.
• The  "SQL and Steam" issue, which really has nothing to do directly with Steam, it is just an issue of installing SQL Server instance to a long pathname.  We are looking to find a new home for this data that will always be under the arbitrary 58 character limit imposed by SQL Server.
• Trying to work around the "Toolset Install Stuck at Execute:regsvr32.exe" part way through the install process
• Make ERF and GFF editors usable outside of the toolset (removing keyfile checks)
• Get FaceFx showing in the menu consistantly (update)

This is not an announcement of when we will have these ready, but that we are working on them.  Additionally, this list may grow or shrink depending on how quick things come together.  This patch will not be for new features, only fixes to deal with the most troubling issues that have come with the initial toolset install.

Please don't spam the thread with questions about when it is coming out.  I will edit this post to say more when I can give definitive information.  I can say  that it will not happen before November 16, and it will be day to day after that date until we have something that I am happy with.

Thank you for your patience and enthusiasm.

November 16 Update: 
Figured it was time for another update.  We are continuing work on both a toolset patch and a new toolset installer.
The patch would be for those with a working toolset that want to get the latest changes. It and would involve a  smaller download and quicker update.  The new installer would be for those installing for the first time, or who would rather discard their current installation and take another run at it from scratch.

We have added in the FaceFX fix
(it is a very simple one) and testing is underway on the rest of the items.
For those wondering about the "Can't connect to Database" issue.  This is just another symptom/manifestation of a failure somewhere in the installation of SQL Server 2005 and the populating of the database with data.  This is almost always an issue with the SQL Server 2005 installer, which is out of our control.  We have worked through many issues in the past with this, yet no matter how many tweaks we make to our process around it, it always seems to find another way to slap us around.  It has been the bane of my existance and I send virutal fireballs in its direction nearly every day.  We will continue to fight the fight, and eventually we will emerge victorious (I hope)

A complicating issue with the patch is that we need to provide a path for those of you who have created content and don't want to see it all destroyed through the patch process.   Without going into great detail, part of the change that we had to make for the Plot GUID affects how we can upgrade your existing data.  We can do it,  but it is not as straightforward as it should normally be.

I still cannot
provide a release date as all it takes is a failure on one platform/machine to send us back through another iteration.

Additionally, I am not entirely ignoring the forum, but I will not generally be responding to a lot of questions here, as time spent doing that is time not spent testing the toolset and the installer.

November 18 Update:
The new installer testing is going very  well so far.  We are now installing the Database instance to a different directory than we did in the past to avoid SQL Server's 58 character limit.  The  Plot GUIDs fix seems to be holding up well, so that the toolset  shouldn't be messing with your game anymore.

There will be a bit more delay so that we can co-ordinate with the next game patch.  There is one pending very soon, and it may affect the toolset.  Rather than sending out a new toolset installer followed by a game patch, followed by another toolset patch to be compatible with the game patch, we will follow their lead.  Once their patch is locked down, we will make sure things are still compatible on the toolset side before releasing our stuff.

As for Windows XP SP2:  We will not block installation, but give a scary warning so that people can proceed at their own informed risk.  I resurrected an old test computer here in order to get SP2 set up on it.  Lo and behold it was already running SP2.  It has had dozens of successful installs\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\unistalls done on it over the past year.  Lots of not so sucessfull ones either, but none that threw it into the endless reboot cycle.

For those fighting with the "can't connect to database" issue, Bryan is constructing a wiki page that walks you through setting up SQL Server manually outside of the toolset installer, restoring the database, and connecting it to the toolset.  This may help many of you get beyond the dreaded message in the meantime.

November 26 Update:
Sorry for the long absence here.  As you might well imagine, we have a lot of "irons in the fire" here.  The reality of things is that the game patch takes precedence over the toolset, and it needs to be locked down and stabilized before we can fully test and release the toolset patch.  If we get ahead of it with a release of the toolset, as the game patch could cause additional problems with the toolset, or the toolset could not be compatible with the newly patched game.

I am regularly  updating the following page on the wiki:
http://social.biowar...hp/Known_issues

I would recommend  that you keep an eye on it for regular developments, as it is much easier to keep it updated and organized with the latest information than the forums.  You can get a quick summary with the latest information there without trolling through multiple  threads.

Here are some highlights of what you can find there:

• many lightmapper issues have been sorted out, and a new seperate download for that is already available in advance of a new toolset.
• a simple workaround available to be able to see water from a custom created level in-game

We also have created a new database with the proper GUIDs and strings in it, that will not mess with your game if you use the toolset, and are considering releasing it prior to the release of the new installer for those that are conversant in SQL Server and would like to restore it manually.

We continue to find new and interesting ways to be thwarted by SQL Server.  We, and other community members continue to find new ways to solve it, and the wiki page for that is constantly being updated with new solutions as well.  Personally,  I got frustrated with it yesterday and decided to abandon 2005 on one machine and try to get 2008 running instead, hoping that it might have fewer issues.  After 2 hours fighting with trying to get SQL Server 2008 to even install on that computer, I abandoned that idea and went back to the enemy that I know...


December 4 Update:
New toolset is definitely getting closer to release.  I am doing currently doing regression testing on all of the editors while using the the latest version of the game patch that is wending its way through approvals. I expect that we will be able to release the new toolset some time next week.  With the new version, we also have the full single player database included, so you will be able to easily modify game content directly.  Along with that we will also be the source files for the levels (seperate download), so you will be able to modify levels directly and export them for use in the game or for your own content.

We have fixed up as much "cannot connect database" stuff as we can, but really the potential ways for it to fail seem to be infinite.  Bryan and I are just putting the finishing touches on a wiki page to try and help people encountering  this error troubleshoot it in an organized manner, rather than just randomly trying the potential solutions one at a time until something works.

Also, James has published another version of the lightmapper that is working very nicely, and fixes the problems with the sun.  See http://social.biowar.../index/219061/4 for more info.

We are actively working on improvements everyday, and are eager to give you the newer stuff as soon as we can
Again, thank you for your patience.
Stay tuned for more updates.

Modifié par Allan Smith, 10 décembre 2009 - 11:32 .

[BryanDerksen]

One problem is that "Unable to connect to the database" could be a symptom of all sorts of different possible underlying problems. We've found a few issues that might give rise to it and we'll be fixing those in the next toolset release, but that doesn't guarantee that we've found all of them.



One that we just uncovered serendipitously while fixing the "missing generate FaceFX" bug is that there's a registry key that can cause this message to come up if it's missing.



Location

HKEY_LOCAL_MACHINE\\SOFTWARE\\BioWare\\Dragon Age\\Toolset\\Environment



Name:

DefaultDatabaseConnection



Value

Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=bw_dragonage_content;Data Source=.\\\\BWDATOOLSET



If you're having the cannot connect error, check your registry to see if this key is there. Since the FaceFX bug is being caused by a registry key being missing it's possible that some of the connection failures are being caused by this too.

[BryanDerksen]

It looks in both locations, so that's not the root cause of this particular "unable to connect" error. Maybe one of our other fixes will catch it, I hope.



We haven't done anything with the lightmapper just yet, our priority has been to fix these various showstopper bugs that have cropped up and get a release out addressing them as soon as possible. Assuming nothing else unexpected blows up that'll probably be next on our agenda afterward.



I don't think a stand-alone gameless toolset is on the agenda. Some of the agreements we made that allowed us to distribute third-party components such as FaceFX, FMOD, Speed Tree, and so forth for free stipulate that we can only distribute them to owners of the game, so producing a stand-alone toolset that included those components could potentially get us into trouble. The disk check copyprotection doesn't kick in for the toolset, though, so you could perhaps install the game at work and then leave your disk at home?



Can't give an ETA on when we're releasing additional designer resources, but rest assured that I raise the issue at regular intervals in here.

[BryanDerksen]

Ranlas's description of the situation here in BioWare is pretty much on the mark. Internally, our developers' machines all run the same Windows XP version and IS pushes the same updates out to everyone in synch. We also have just one central database server, with dedicated database programmers who ensure that it always "just works" behind the scenes. So bringing the toolset out into the very diverse world of systems out there has uncovered all sorts of assumptions the code made that only work correctly on our homogenous environment. We caught a lot of those in beta testing but obviously there were a few biggies still lurking.



The plot GUID issue is a nasty one that we missed in testing because it doesn't jump out and grab you by the face until you actually try playing the main campaign for a bit. Internally we were mainly focusing on checking whether we were able to create and run _new_ modules correctly, which the GUID bug doesn't interfere with. In hindsight a mistake, of course. You can be sure we'll be testing the effects of the toolset on the main campaign much more thoroughly this time around.

[Allan Smith]

Figured it was time to throw you some more crumbs. I have posted an update on the initial post in this thread.

[Allan Smith]

I have edited the the top of this thread with another update. Things are proceeding well, but you will have to wait a bit longer still.

[Allan Smith]

New update posted at the top of the thread.

[BryanDerksen]

Internally, we use one big SQL server that everyone working on the game connects to. The advantage of this is that we can't inadvertently step on each others' toes thanks to the checkout/checkin functionality, and that we can always see the latest versions of the game's resources when working on it. It's really really handy... when you've got dozens of people all working on the same module simultaneously. For an individual user working on his own perhaps not so much, but doing a rewrite of the toolset's resource storage system was a bit beyond the budget we had for getting the end user toolset available.



The other reason why it works well for us internally is that we've got people whose main job is to do the database administration for us. We're not able to package that up to ship with the end user toolset, alas.



Nikki> the missing party member problem sounds like the plot GUID bug biting you. See http://social.biowar...p/Plot_GUID_bug for details.

[Allan Smith]

XiNAVRO wrote...

Any plans to fix the locale issues? Systems that use Asian languages for their unicode display have encountered problems compiling scripts; switching back-and-forth between English and the language I use is quite bothersome (since I need to reboot every time), and I don't really feel like installing AppLocale just for the toolset; especially so when some programs and installers are known to show oddities with AppLocale installed to the system.

Nothing on the immediate horizon.   Not discounting it completely, but it is certainly not the top of the list.

[Allan Smith]

Update posted on first post

[Allan Smith]

Update Posted: Version 1.01 of the toolset installer has been released

[Allan Smith]

You can install overtop of the old installation. However, in case you missed it on the announcement: YOU WILL LOSE YOUR EXISTING WORK IF YOU DO NOT FOLLOW THE STEPS LISTED AT: http://social.biowar...base_migration.



The full single player database is also included in this installer, the previous one only had core resources (bad ones at that). The .lvl files are not included in the installer, but will be posted on a project on the social site as a separate download in the very near future. In the meantime there is a lot more you can do, now that you have the single player resources.

[BryanDerksen]

Odd, the stages all export correctly for me on my test machines. Anyone else having export problems?



Oh, for those experimenting with single player resources, there will definitely be some cutscenes in the database that are broken for known reasons; they were only used internally to generate bink files that were included with the game, and then some of the animations that were used only there were stripped out of the final build's resources to save space. The cutscenes showing the Darkspawn charge at Ostagar were binked, for example.

[BryanDerksen]

Rodro Lliv wrote...

So, installing this new toolset should have repaired old saves that suffered the Plot GUID bug? Cause there are no party members on the camp in my last save yet.

No, this won't fix old saves that have been affected by the plot GUID problem. This update only means that if you export core resources from it in the future it won't cause any more savegames to become corrupted.

The problem is that when you play the game with broken plot GUIDs in core override, the game records changes to the plot flags under those GUIDs and not the old ones. Saving the game records those new broken GUIDs in the savegame file, and so even when you switch the GUIDs back the flags you set during that time are "lost" (they're still in the savegame, but they're saved under incorrect GUIDs).

[BryanDerksen]

giskard44 wrote...
I hate to say this Bioware but I read the wiki about the builder to builder back it and its so vague i found my self having to guess which parts are my mod needs backing up. Your example of a builder to builder create lists 1 amulet, my mod has 2000 entries and i have no idea which parts belong to my mod and which parts are the old database I should avoid loading.

For example is Genip_shrub something I should be backing up, it says it exists already in the backup of my mod. So why is it in the backup in builder to builder ?

The builder-to-builder process automatically brings along copies of all of the dependancies of the resources you selected. The reason this is done is because the database checks whether a resource's dependancies exist before it allows you to add a resource to it, so if you were to try loading a resource with missing dependancies it would fail to do so. We'd wind up with builders whose B2B backups were useless because they'd forgotten to include some little thing somewhere.

When you do a B2B load, the B2B load screen will have a "hide/show dependancies" button that should allow you to exclude all the auto-included dependancies from your view. Other things you can do to sort through these extra resources and keep your work organized:

* If you're importing something into your database for the first time, sort by the "exists" column to find which resources will be created anew by the B2B load.
* If you're importing an entire modules' contents, sort by the "core" column and exclude the core resources. Provided, of course, that you didn't edit core resources. You should probably avoid editing core resources until you're familiar with resource organization in any event or producing a tidy builder-to-player package with core overrides in it will be difficult.
* Place all of your module's resources into consistently-named folders within your toolset. You can then sort by folder name in the B2B create/load screen to get specific groups of resources easily.

And what about 1999 other entries that also say EXISTS ?

"Exists" simply means that there's already a resource in your database with that resref. Whether you want to import it or not will depend on the details of what you're doing - if you're importing a module into your database that you didn't have in it previously, you probably don't want to import those. Importing a resource that "exists" will replace the version in your database with the version from the B2B package.

Why is zz_upgrade listed with a ResRefID of 6924 ?

Is that part of my mod ?

ResRefID is only used internally by the database to keep track of individual resources. If you were to load the B2B package into two different database, I believe wind up with different ResRefIDs associated with the imported resources. Under most circumstances you probably have no need to pay attention to ResRefIDs, they're exposed for troubleshooting purposes. For example if you somehow wind up with a resource that's missing its dependancy, despite all the safeguards in the system that should prevent such a situation from happening, those dependancies would be identified by ResRefID in the error message.

And the sound sets for zz_ss_legionaire, i am pretty sure i never touched those but they are in my back up anyway.

It's in there because something you selected to include in the B2B package depended on it, somewhere down the chain of dependancies. If it _wasn't_ included in the B2B package, you would be unable to import your resources into a database that didn't already have zz_ss_legionaire in it. Imagine the complaints we'd get then.

Now try and tick only the new entries and you will be there all day, its quicker to remake the mod than to use your backup system. Your own example has 1 listed, mine has over 2000 and 15 minutes in to this I had hardly made a dint in the list.

Sort your resources by some appropriate column or click the "hide dependants" button, and then use shift-click to select the group of resources you want to include. Click "uncheck all" and then "check selected".

If there's no column to sort by that will group your resources appropriately, you may be able to alter the directory structure of your mod to organize it better.

You have created a system that requires us players to be system administrators and the result is a complete and utter desaster that will result in thousands of complaints appearing on this forum in the coming weeks.

Given that we don't have the resources to completely rewrite the foundations the toolset is built on, there are unfortunately limited options for how we can approach managing it. We can of course do better - there's always room for improvement - but some of the issues you raise are the result of things we've had to do to avoid far _worse_ issues. The automatic inclusion of dependancies in B2B packages is I think one of those things that's likely unavoidable because the alternative is worse.

There could be improvements to the B2B interface to more clearly explain what's going on in it, I suppose. Something we could look at.

Also currently we modders face losing our work every time you update your toolkit because of the above system being so complicated and the solutions you offer are no solutions at all. Many will in fact result in errors that are unpredictable, for example how would you know if a player forgot to untick 1 box out of several thousand in his mod, he might be using a single old file after accidently catching it whilst trying to import just his mods new entries.

This particular update to the toolset was much trickier than future updates will be because we had to change both the default core resources installed in the database _and_ the B2B file format at the same time (the plot GUID bug was intertwined into both of these things). I don't expect we'll have to do that again. In future, it should be a simple matter of just unchecking the "install database" options when you run the installer.

Simple SQL Backup option.

I know your commited to the current system now having developed and used it so I am not expecting you to change everything. But a decent program to take the pain out of backup and restoring mods database options is pretty much required to bridge the gap between players and your system administrators expections of us.

All the program needs to do is allow us to select the mods we wish to back up and have a backup button and the program should then automatically back up that mods database entries. Theres no reason why you cannot impliment backups cannot be decided in to groups so team support remains if you wish to go that far.

Just keep it nice, simple and no SQL experience required.

In a nutshell, this is what the B2B process is there for. We could perhaps put in an interface to allow you to more easily auto-select all resources in a particular module to include in a B2B export, but it would still wind up including core dependancies in the B2B package.

A "backup entire database" menu command would be nice to have, to avoid messing around with those batch files, but that falls under your other objection because restoring it would overwrite any mods that weren't in it.

If you intend to leave everything as it is, eg continue using the system presented with 1.1 can you please say so, so those of us who see major problems ahead can move on and leave you to it. I would like to have the choice between risking hundreds of hours of work or spending that time else where basically. I am sure you understand.

PS I am following my own advice here and have given up trying to get the old work to important safely in to 1.1. I consider it far too risky even if it does work and ticking thousands of boxes over and over to find the right combination to make a safe import is more than I am willing to do, if I am honest.

Well, I guess I have to say that we're unlikely to change the fundamental structure of how the toolset's database works - we just don't have the resources for it right now. And that the basic B2B system is also unlikely to change. But I think that it should be quite possible to manage it in a way that doesn't require picking through thousands of entries one by one, and there are some tweaks I've mentioned in this post that we may be able to include in the future to simplify matters. Now that we've got Single Player out and have stomped the plot GUID bug we can start looking at improvements like that.

[BryanDerksen]

Challseus wrote...
EDIT - Curse you Bryan. I should have known you were writing a more in depth response

Heh. No worries, I'd actually forgotten that there was an "owner module" column you could sort by too. Makes things even easier, thanks for mentioning it.

[BryanDerksen]

Ah! This could be a cautionary example where BioWare didn't follow the proper best practices that we recommend for other builders to follow. The stage "3p_a_pattern" is a core resource that has been set to use a single player resource as a dependency (in this case the area layout that's being used as a visualization source). That means that whenever you're not in the single player module, its dependency doesn't exist. Try switching to Single Player and it should work again.



There's a handful of these in the core stage resources. They shouldn't actually cause a problem in the game, since a stage's visualization is used only in the toolset for ease of editing and isn't part of the exported resource, but it's sloppy nonetheless and you may wish to edit it so that it isn't using that area for visualization purposes. Our bad.

[Allan Smith]

you do not have to update to game patch 1.02 first, but it would probably be a good idea to patch your game anyway.

[BryanDerksen]

giskard44 wrote...
I find 95% of my problems disappear instantly the moment I give up on my old v1 work and choose to start again. All the risks associated with the accidental import of old data disappears from my own mods too. I'll just have to avoid mods that where updated or made prior to 1.01 incase they bring with them some nasty surprises that affect my saved games.

It should be possible to confirm whether any database's core plots are "good" or not by opening a few core plots in the toolset and checking their GUIDs in the object inspector. It'd be too tedious to check _all_ of them, but a quick sampling of the most heavily referenced ones (such as gen00pt_generic_actions, which should have a GUID of D5301CC1-DCC1-4D9D-9E55-5AD56C05C7A3) should be enough to be reasonably confident that you're clean. As you've noticed these things tend to get swept up in B2B creation in large numbers.

If you're wanting to double-check your own builder to builder load there's an easy way to ensure that your core plots weren't changed. When you B2B load all of the resources that get changed in the process get automatically checked out. So restore the new database (wherein everything starts out checked in by default), do your B2B load, and then filter your palette window to only show checked-out resources. If any core plots show up in the list of checked out resources that would indicate that they were changed by the B2B load. If that happens you can just restore your database to the default and try again.

[BryanDerksen]

BioSpirit wrote...

Is there a list of known bugs and issues regarding to the Toolset ?

We're maintaining a list here: http://social.biowar...hp/Known_issues

However, since there's just been a new version put out most of the issues it's probably going to have are not known yet. I believe that all of the issues currently listed here have been resolved or have had workarounds developed (aside from the ever-popular "cannot connect to database" error message, which is unlikely to ever entirely go away since it's a catch-all that basically translates to "something went wrong with installing and setting up MSSQL server" - there's a huge number of reasons that can potentially happen).

[BryanDerksen]

We bundled 2005 simply because it's the version we use internally so the toolset's had more testing with it. Mind you, we haven't tested _installing_ it much internally.



We've never heard a negative report from people manually installing 2008, so by all means give it a try if 2005 is misbehaving or otherwise not to your tastes. In future we're thinking of separating out the database installation step from the main installer to make it easier to try alternate versions (and hopefully to make troubleshooting the installation step much easier).

[BryanDerksen]

Kaldir II wrote...
Allan wrote somewhere in the first post of this thread:
"Personally,  I got frustrated with it yesterday and decided to abandon 2005 on one machine and try to get 2008 running instead, hoping that it might have fewer issues.  After 2 hours fighting with trying to get SQL Server 2008 to even install on that computer, I abandoned that idea and went back to the enemy that I know..."

Okay, I've heard a negative report from a user regarding 2008 now.

[BryanDerksen]

The core plot GUIDs have been fixed in both the 1.01 full single player database and the pre-release core-only database, so B2Bing between the two should be safe.



Note that the B2B format changed between version 1.00 and 1.01 (it was a flaw in the B2B process that was the root cause of the plot GUID corruption in the first place), so you'll still want to follow the procedure outlined in the database migration page if you're upgrading your toolset and want to keep your existing work.

[Allan Smith]

@Quatayba -Thanks.



I am un-stickying this thread now. Will let it work its way down through the pile.