70 réponses à ce sujet

[Fast Jimmy]

This thread is an amazing read. Seems like we've been getting a lot of these recently on the BSN!

Having PMP certification and dealing with software development and bug identifications and case testing for my job (although having all the technical experience of a soggy loaf of bread), it's pretty funny and interesting to me to see aspects of my job in what you all do in the development of games like Dragon Age. It makes me see how you all would need want to play the final game, as the (small) tasks I do involved with such software releases are the very definition of TEDIUM. I hope making a game would be more exciting, but I'm sure most days it's not THAT much more exciting, so you guys have a tip o' the hat from me.

That being said, I had previously asked this question of Allen, but I'll ask again: the new Unreal engine is being designed to be not just a graphic and physics powerhouse, but also to use significantly less direct programming in order to implement features. The team's stated goal was to be able to have non-programmer developers be able to develop lots of real, active content and be able to test it as soon as they design it, without having to send it to a programmer to make into a reality, relying on programmers to only do the last 10%, of tying the code to the overall end product.

Would either Stan or Allen be able to say that such a design philosophy would be a benefit to game creation? Does the Frostbite engine allow this type of development, that results in developers creating content, instead of writing a requirements document and hoping the programmer has the same type of vision before the feature is drafted?

Thanks again for giving this information guys, it's always a real treat to see how the 'magic' happens.

[EricHVela]

It seems to me that it falls to QA being responsible for keeping a record of the weirdest BTS events for the blooper/gag reels. (Give it an ironic title such as "Working as Intended".)

[Dhiro]

I... I ship Epler x Schumacher now.

(I appreciate the post and think it was a very good read, also! Now, to the fanfiction...)

[Beerfish]

From Stans post...

"You misunderstand a little bit. You don't "move on" from anything. At any given time, something you've already worked on might break due to something someone else has done, or new content placed into the game, or new features added, or, you know,"

This is one of the most over looked things by fans asking for features, new features, changes and thinking it is always a simple thing that can be done in isolation. Any time you change ANYTHING in a piece of software, there is a chance something else will break or not function as intended as a result.

[Allan Schumacher]

That being said, I had previously asked this question of Allen, but I'll ask again: the new Unreal engine is being designed to be not just a graphic and physics powerhouse, but also to use significantly less direct programming in order to implement features. The team's stated goal was to be able to have non-programmer developers be able to develop lots of real, active content and be able to test it as soon as they design it, without having to send it to a programmer to make into a reality, relying on programmers to only do the last 10%, of tying the code to the overall end product.

I don't know if I entirely understand the question, because what you describe has been my impression of the way engines and games have been for quite some time now. Do you have any type of example?

[Vandicus]

I think his question is in the last paragraph rather than in that paragraph.

[Fast Jimmy]

Allan Schumacher wrote...

That being said, I had previously asked this question of Allen, but I'll ask again: the new Unreal engine is being designed to be not just a graphic and physics powerhouse, but also to use significantly less direct programming in order to implement features. The team's stated goal was to be able to have non-programmer developers be able to develop lots of real, active content and be able to test it as soon as they design it, without having to send it to a programmer to make into a reality, relying on programmers to only do the last 10%, of tying the code to the overall end product.

I don't know if I entirely understand the question, because what you describe has been my impression of the way engines and games have been for quite some time now. Do you have any type of example?

http://www.gamesindu...-next-gen-at-e3">[url=http://www.gamesindustry.biz/articles/2012-06-08-unreal-engine-4-illuminates-next-gen-at-e3]http://www.gamesindustry.biz/articles/2012-06-08-unreal-engine-4-illuminates-next-gen-at-e3

Specifically:

Of course, the key for Epic and developers looking to create next-gen games is the ease of use and speed at which changes can be made in Unreal Engine 4. With improvements to the toolset, event graph, construction script and more, Willard emphasized that iteration time on a project is "incredibly fast."
Epic showed how everything is running in real-time in the editor and designers can literally play the game and make changes to it in the editor at the same time. You can make changes to the source code, script changes and more all while actually running the game. And then when the code is done compiling, you'll see the changes you made take place in-game right away.

Epic has been touting this for a while now in relation to the Unreal 4 Engine, that it is designed to have everything able to be created with very little actual coding experience required. So an artist can adjust the lighting mechanics to best reflect their vision of a dungeon, or a writer can create multiple variables and outcomes to a situation that can be tested immediately afterwards to see if the experience is what they anticipated, etc., etc. The design goal for the new engine is to stem the tide of resource bloat with video game development, resulting in less content needing to be designed, created by a programmer, tested by a QA analyst and then returned back to the designer to see if the finished product is what they had in mind. 

I'm curious to see if they can live up to such claims, of course, but they said one of their stated goals was to allow AAA studios to develop high quality games for significantly less resources and for small developers to create games with a level of polish and fluidity you would usually see with much larger teams.

Modifié par Fast Jimmy, 03 octobre 2012 - 04:40 .

[Kidd]

Beerfish wrote...

This is one of the most over looked things by fans asking for features, new features, changes and thinking it is always a simple thing that can be done in isolation. Any time you change ANYTHING in a piece of software, there is a chance something else will break or not function as intended as a result.

Tell me about it. I may never have worked on anything as hefty as a BioWare engine, but I remember coding an RPG engine as a kid. After giving the inventory system a complete rewrite to enable more awesome features (and less work for mundanely featured items, booya!), without ever touching anything outside of item management code, my items would actually depower the hero after the hero had gained a level with them equipped. Oopsies.

It's so easy to think you're working in a vacuum. When in reality, you almost never do.

[Allan Schumacher]

Epic has been touting this for a while now in relation to the Unreal 4 Engine, that it is designed to have everything able to be created with very little actual coding experience required. So an artist can adjust the lighting mechanics to best reflect their vision of a dungeon, or a writer can create multiple variables and outcomes to a situation that can be tested immediately afterwards to see if the experience is what they anticipated, etc., etc. The design goal for the new engine is to stem the tide of resource bloat with video game development, resulting in less content needing to be designed, created by a programmer, tested by a QA analyst and then returned back to the designer to see if the finished product is what they had in mind.

IMO what Epic is describing is nothing new. The quote you're mentioning is just continued improvements to the tools to allow faster turnaround for designers. It's still a significant programming investment to get things set up like that. But the goal of "faster iteration" is always paramount. If you always have to spend 30 minutes building assets to make a small change, you lose a lot of time, and in some ways Frostbite does indeed have large improvements to verifying this sort of stuff compared to Eclipse.

In other ways Eclipse is still superior, in that Eclipse had better conversation previewing and whatnot because stuff like that didn't really exist in Frostbite. But it's the type of thing that my team is actively working on so that people like Mr. Epler can better preview their changes with as little time wasted as possible. On the plus side, these are problems that were already solved with Eclipse, so it's just a matter of seeing what it would take to work with Frostbite.

But with the aspect of live changes (making changes to the level/scripts/whatever while the game is running), yes Frostbite does allow for stuff like that. It's also super easy to fire off relevant events and whatnot without having to design specific console commands or anything like that. I can set up a conversation, quickly fire off some events in the editor to set various plot states, and then start the conversation up in game and see if it's behaving the way I expect based on the plot state.


Tools have long been developed with the idea of an ease of use in mind, because the reality is that there's a lot of creative story teller types that just don't know how to program. I actually worked on a project while at University, called ScriptEase, which worked with the NWN Aurora Toolset and was designed explicitly to allow people that don't know how to program to create scripts. We ran experiments with high school English courses where they would create stories in Aurora. It was pretty amazing actually. Programmers don't really create much content unless they kind of have a dual role. Epler isn't a programmer and never looks at code, but he's the one that creates the in game content. None of the programmers on the digital acting team are ever directly responsible for any of the explicit content you see in a game. They don't frame the shots for cinematics, they don't place the triggers that start up a conversation and move the guys around. The programmers create the tools for the designers to be able to do all this stuff on their own.

1. Is bioware working on an agile type of development process?(that is if you are in the position to answer)

Yes. We do employ agile development. Our scrum work is usually split into 2 or 3 weeks of work with short term deliverables.

2. Problem with software today is how not a lot of firms are able to release the software on time and on budget. This causes qa to have the mentality of not trying to catch all the bugs but rather to let in as few bugs as possible(I prefer this method as you could have a ship but you need it to sail at some time). When it comes to making games is it a similar mentality?

I still try to "catch them all" in that I try to find as many as I can. The reality is that not everything can be caught though. Just for reference sake, if 100,000 people pick up the game day one and play it for one hour, they just spent about 34 years worth of time into the game assuming that I am working 8 hours a day for 365 days a year. Or, if we have 34 people QA guys going crazy, a year's worth of time. And there's no way that all that time was all spent on JUST the first hour of the game! So yes, towards the end of the project the focus definitely shifts to game breaking type of issues in terms of what gets fixed simple due to "zot management."

[hoorayforicecream]

FemaleMageFan wrote...

Hello Allan, thank you for the information as always. I would just love to ask some questions if you are in the position to answer them.
1. Is bioware working on an agile type of development process?(that is if you are in the position to answer)
2. Problem with software today is how not a lot of firms are able to release the software on time and on budget. This causes qa to have the mentality of not trying to catch all the bugs but rather to let in as few bugs as possible(I prefer this method as you could have a ship but you need it to sail at some time). When it comes to making games is it a similar mentality?
Thanks

I can't really answer #1, but I can answer #2. It's always, always, always about triage. Every bug that is discovered is evaluated. QA and production decides how serious the bug is. The most serious are the ones that will crash the game or cause the game to fail certification. Then there are those that make (critical) sections of the game unplayable. Below that, there are other types of bugs like weird animations, missing VO, UI problems, dialogue not triggering, etc.

Every bug is then evaluated by the appropriate departments (engineering, art, design) to figure out how long it would take to fix the bug, and the estimated fix time (and potential risk of the fix) is calculated into it. Then the producers need to consider things like "How far away are we from certification?" and "Is rewriting large chunks of the whole animation system worth fixing this one animation bug when we have to ship in a month?"

When a project is pressed for time, more of the lower-end bugs get let go simply because there isn't time to address them all safely. The more severe bugs are always addressed. Always. If you don't address them, you can't sell the game. Sometimes you get a dispensation from Sony and/or MSFT to do a title update (day 1 patch)  to fix any bugs that they find in the certification process, rather than having to fix them all and resubmit.

It's because of these evaluations that certain bugs live for a long time (DA2 Zevran Import bug), while others are immediately fixed (world of warcraft hotfixes). Unfixed but identified bugs are often deemed to be either high risk, require too much time to properly verify the fix, or just aren't high enough on the priority list to be addressed. It's almost certainly logged somewhere, but it probably wasn't addressed because a producer somewhere decided it was better to spend that development time elsewhere.

[hoorayforicecream]

Avejajed wrote...

So I can imagine now why the process of creating a new game takes as long as it does- it seems like it can be a rather slow, tedious process for everyone involved.  When there are games that are churned out every year- like Madden for instance- is it done so quickly because they are really only tweaking what was already there?

For annual SKUs, it just means a smaller scope. This year, you build these big new features, and adjust those small features. Next year, you'll touch a different set of big new features, and another set of small features.

Creating a new AAA game from nothing is very rare nowadays. Most of the time you have to leverage some form of old tech, usually the game engine. But what it's actually like, if you can imagine, is like building a house while your tools are still being built. Imagine trying to hammer nails down with a hammer that's still being developed. The hammer might not have a full-sized handle, or the claw in the back to pull the nails out. But it kind of works and you do what you can. Later on, the hammer might be improved with the addition of the claw, which lets you go back and pull out the nails you screwed up on before. You can't just sit and wait for the hammer to be completely finished, because you can work in the meantime. And hammering these nails down now will be used by somebody else later to construct a wall, which will eventually become part of the living room, which will be part of the house.

Game development tends to be a moving target. Things change over time as everyone continues to work, and we developers tend to do what we can with what we have while we can as time passes.

[Allan Schumacher]

What do the digital acting guys do? This may sound stupid so forgive me, but I always sort of imagined that after the "coding" is done by programmers, that the process was a little like animating movies in a way. Where on a computer screen there are guys who take an arm and move it to where it needs to go next frame by frame. But I'm thinking that's not what happens at all?

So the Digital Acting team is made up of a bunch of programmers and Cinematic Designers and similar type of support staff like animators and stuff like that.

Frostbite has a toolset to put content in the game, and the work the programmers typically do is create new features/editors in this toolset as well as the underlining game code to make sure the systems work in game.

So to use your example regarding animating characters in a scene, programmers would work on the tools to allow animations (which will get created in something like 3dsmax or Maya or something) to be used in the Frostbite toolset, and properly recognized and animated in the game. The animations themselves are created by an animator that's familiar with whatever animation tools are made, and then a Cinematic Designer will take said animation work and place it in the scene and manipulate the cameras and whatnot to make it look good in game. The programmers only really worked on the underlying infrastructure to let the content guys work their magic with as little programmer involvement as possible.

[Arokel]

Great thread. Always great to have more insight.

[hoorayforicecream]

Avejajed wrote...

That description makes a lot of sense.  I take it you work in the industry- can I ask what you do? I don't need a resume or anything but I might have a whole other set of questions for you.

Again thanks to everyone for induldging me and others in the thread. After reading in-depth about how certain things work, I don't know if I'll ever be able to play a video game again without imagining the work that's gone into it.

I've had a lot of roles over the years, though typically I do hybrid designer/gameplay programmer work. I typically spend a lot of time working with designers and animators about making stuff in the games I work on fun. I've been a level designer, camera designer, UI programmer, gameplay programmer, scripter, and all sorts of wacky stuff in between. I'm typically the kind of person Allen will bring up concerns and corner cases to, and who has to make the system robust enough to handle them (and all of the reasonable situations that may arise in between).

[obZen DF]

See, this is why I love the Dragon Age boards and team.

[Ninja Stan]

Avejajed wrote...
Awesome, so I was kind of right in a way. I wondered how that was all layered together.  And QA's- they get that after that is all done- maybe not you since you're a bit more specialized- but after a QA gets ahold of it and logs bugs to be fixed, it goes back through the line again, yes? Programmers fix a bug, then the other teams that have had the content after the programmers- they have to go back over it as well to make sure their "layer" still works after said bug is fixed?

It's an ongoing process. Something is always happenig and content is always going into the game, so QA is always working on something new. QA has cycles and schedules like everyone else. It goes back to all the coordinating of schedules that happen.

I really appreciate you all- horrayforicecream included- for taking the time to explain what you do in such layman's terms for us. I imagine there are a lot of people who would like to have a job in the industry and this sort of insight is really helpful.

With that in mind, what kind of advice would you give someone who was interested in becoming a QA tester?

Work on analysis. Take a critical look at the games you enjoy (or don't enjoy) and try to articulate just what it is about them your like or don't like. Google "filing a bug report" to check out some guidelines for filing a proper bug report. Work on your objectivity and try to view games from the perspective of a gamer who is very unlike you. If you dislike something in a game, see if you can come up with a type of gamer who would enjoy that thing and why.

Written and oral communication skills, attention to detail, the ability to do monotonous work for weeks or months at a time, offering and accepting constructive criticism, working well in a tight-knit team--these are all skills and abilities that QA makes use of.

I've also heard of at-home testers- people who get early copies of games to play at home, who just go through the game over and over again looking for specific things and then logging them online. Is this a thing?

It is a thing, but it's usually not a job, per se, and it's not really QA. It's more focus testing, since the games you receive are usually already complete. Those kinds of "jobs" also rarely pay in money, and you don't become an employee of the developer or publisher. There is no real "QA from home" job that is recognized as a "QA job." You need to apply at either a developer or a publisher in order to get a "Real" QA job, and you'll be competing with dozens of other game development hopefuls, as QA is often seen as a point of entry into the game industry.

But once you're in, it's a tech job like any other. You'll work in an office, deal with various levels of management, have lunch breaks and coffee breaks, sit and stare at a screen all day, go to meetings, and chat with co-workers around the water cooler. On the other hand, you're helping to make videogames for a living, and that can be pretty fulfilling and fun all on its own!

[hoorayforicecream]

Ninja Stan wrote...

Work on analysis. Take a critical look at the games you enjoy (or don't enjoy) and try to articulate just what it is about them your like or don't like. Google "filing a bug report" to check out some guidelines for filing a proper bug report. Work on your objectivity and try to view games from the perspective of a gamer who is very unlike you. If you dislike something in a game, see if you can come up with a type of gamer who would enjoy that thing and why.

Written and oral communication skills, attention to detail, the ability to do monotonous work for weeks or months at a time, offering and accepting constructive criticism, working well in a tight-knit team--these are all skills and abilities that QA makes use of.

A friend of mine who interviewed for a QA job actually told me that they asked her to QA an orange. What qualities of it are you looking at? What sort of standards would you need to apply to an orange before it goes to market? If you were to write a report about this particular orange, what would you note down? How easy to read is your report?

[Allan Schumacher]

Haha, the context of a question like that can be tricky, though your answers aren't particularly poor haha.

I often find myself with a pretty diverse gaming experience, so when I ask people what their favourite games are, they'll usually mention one (or several) that I have played. So I'll ask them questions about those games. What they like, what they would improve. Get them talking about games and looking at them from a critical perspective.

Although QA need not really a "mathy" type of position at all.


In terms of a valuable job quality, the idea of having a good eye for detail is great, specifically with respect to how to reproduce issues. Even if you struggle to describe precisely what you're seeing, if you can make it easy for someone to reproduce the issue you're seeing, that's probably 90% of the battle. A lot of time can be wasted, and bugs missed and not fixed, if other people aren't able to understand what they need to do to reproduce the bugs.


Having a desire to want to release something that is kickass, and having pride in being associated with said product really helps stay motivated in keeping to these details throughout the project.