210 réponses à ce sujet

[Ziggeh]

Zine2 wrote...

Moreover, the Bioware poll noted in the article happened one day before the release. Hence it was almost certainly a controlled test audience study. The fact that it mirrors the results of the other polls so closely (over 90% unhappy with the ending) demonstrates that the sample size has in fact mitigated the margin of error to a sufficient degree for the data to be valid.

Conclusion: 90% of Mass Effect 3 players think the ending sucked. This is fact.

Oof, you were doing pretty well there.

I'll grant you it is more likely representative than "other polls" conducted through the internet, but the fact that it mirrors inherently bias polling data does not disprove it's bias. If anything it suggests that it exists.

[Ziggeh]

Zine2 wrote...

Again, pure speculation on your part. Show evidence that the samples are actually tainted by selection bias (the BIOWARE poll

In the absence of methodology, assume bias. Always. The burden of proof in on showing it's absence, not existence.

[Cazlee]

Where is the poll the author talks about in the article? Why would Bioware intentionally leak the endings ONE day before release?

The day before the release of Mass Effect 3, BioWare leaked the endings to its fans and polled them to gauge their reaction. Ninety percent of users said "the endings suck." That's a pretty unambiguous statement. After about a month of similar comments from consumers, BioWare co-founder, Dr. Ray Muzyka, published a blog post explaining the studio's intention to make some changes to the offered endings.

[Zine2]

incinerator950 wrote...

Dunno, I took a statistics course and so far none of these polls have a random population.  Infact, its like surveying a group of people who have been waiting for something for over an hour, unconfortable and asking how they feel.  As opposed to randomly surveying people every fifteen minutes, somewhere not congested.  

That's actually harder to determine than you'd think.

The BSN polls may seem non-random (it's made up of BSN people after all - a subset) but if the overall distribution of different major demographics remains the same between the BSN base and the total player base then it actually doesn't matter. For all intents and purposes, BSN Player Base == Entire Player Base. And certainly, the fact that everyone now has to get an Origin account is making that distinction largely moot.

Also, I'm assuming that the Bioware test audience poll had random sampling; at least within the confines of the "ME3 player" population. Leaking it a day before the game's release implies a deliberate test.

The alternative is that I'm giving Bioware too much credit and they can't even do their own polls right.

[GiarcYekrub]

LadyWench wrote...

GiarcYekrub wrote...
I like the ending so do my friends that play it, only seen people on the internet hate it. i suspect half of those just watched it on youtube out of context. also i havent seen a consistent complaint about it just alot of individuals with there own ending

Obviously haven't watched THIS. This is pretty much the definitive video on why people who dislike the ending do so.

That guy likes to hear himself talk, 8.32minutes in and he still hasn't hit a point, just a lot of waffle about Star Trek...

Modifié par GiarcYekrub, 18 avril 2012 - 09:24 .

[Zine2]

Ziggeh wrote...

Zine2 wrote...

Again, pure speculation on your part. Show evidence that the samples are actually tainted by selection bias (the BIOWARE poll

In the absence of methodology, assume bias.

LOL, fine, maybe I gave Bioware too much credit.

[Torrible]

@Zine2
Why is the onus on me to prove the results are skewed? Isn't the burden of proof on the researcher who wants to show that the sample isn't biased?

Yeah, I'm going to conduct a research and tell others to prove to me that my not-so-randomly collected data is not biased. "Lalala I only collected the data from one place...prove to me it's not biased." The data is collected from a forum that has been known as the place where unhappy consumers of Bioware games congregate and complain (constructively or otherwise). These people (passionate about all things Bioware, interested in lore and RP) are not a good representation of the entire consumer population which also consists of shooter fans, casual gamers, indifferent lurkers and forum avoiders.

That's BIASED. It's not as bad as a online poll on internet literacy but it's still biased. That's logic. No proof needed. 

Modifié par Torrible, 18 avril 2012 - 10:49 .

[count_4]

Ziggeh wrote...
I'll grant you it is more likely representative than "other polls" conducted through the internet

It's a poll held by the developer with the sole purpose to produce reliable results to predict the response of a greater audience. 
It's pretty safe to say that it is representative. Otherwise it would just be a huge waste of time.

Modifié par count_4, 18 avril 2012 - 09:28 .

[Zine2]

Torrible wrote...

@Zine2
Why is the onus on me to prove the results are skewed?

Well, I was making the mistake of assuming Bioware was competent. Cooler heads have reminded me that their incompetence may extend to more than just screwing up an ending.

Modifié par Zine2, 18 avril 2012 - 09:28 .

[Cazlee]

Zine2 wrote...

Ziggeh wrote...

Zine2 wrote...

Again, pure speculation on your part. Show evidence that the samples are actually tainted by selection bias (the BIOWARE poll

In the absence of methodology, assume bias.

LOL, fine, maybe I gave Bioware too much credit.

You give the author of the article too much credit.
What reason do you have to believe everything the author says is fact?

[Ziggeh]

Zine2 wrote...
LOL, fine, maybe I gave Bioware too much credit.

You'll find that many polling companies don't understand bias. One internally conducted by someone with no track record of quality polling? Yeah, assuming there bang on is too much credit. It's not impossible, but it's improbable.

[incinerator950]

Zine2 wrote...

That's actually harder to determine than you'd think.

The BSN polls may seem non-random (it's made up of BSN people after all - a subset) but if the overall distribution of different major demographics remains the same between the BSN base and the total player base then it actually doesn't matter. For all intents and purposes, BSN Player Base == Entire Player Base. And certainly, the fact that everyone now has to get an Origin account is making that distinction largely moot.

Also, I'm assuming that the Bioware test audience poll had random sampling; at least within the confines of the "ME3 player" population. Leaking it a day before the game's release implies a deliberate test.

The alternative is that I'm giving Bioware too much credit and they can't even do their own polls right.

I never give anyone too much credit unless they really did something over the top. 

It's like saying ME seems like its full of choices, but the majority of it is smoke and mirrors, and you're actually playing a Linear story with a few extra paths.

However, I wasn't there for those polls.  When the endings were leaked weeks in advance to what this poll was talking about, the outrage was pretty much 60/40 for dislike/like or don't care. 

I'm referring to these polls now as opposed to his supposed control group poll.  It's generally a proven fact with gaming boards that more people post there to voice their negative opinions or seek help then to generally discuss something positive or make guides and help newbies.  There are more people griping who haven't been on here for years then people who have established for a few years that they're there to generally discuss something and move on.  You know, busy playing the game.

[Ziggeh]

count_4 wrote...

Ziggeh wrote...
I'll grant you it is more likely representative than "other polls" conducted through the internet

It's a poll held by the developer with the sole purpose to produce reliable results to predict the response of a greater audience. 
It's pretty safe to say that it is representative. Otherwise it would just be a huge waste of time.

As I say, polling bias is a deeply misunderstood area. You can't assume it's absence.

[evisneffo]

...Did that poll really happen? Wow. But I'm curious: what would be the use of conducting a test-screening-type poll the day before release? It would obviously have been too late to change anything should the poll have indicated that that was necessary.

Modifié par evisneffo, 18 avril 2012 - 09:35 .

[tomcplotts]

Johcande XX wrote...

You guys don't get it. 90% is a vocal minority using BioWare math.

At least the same math that brought you the impossible SP EMS math.

[78stonewobble]

evisneffo wrote...

...Did that poll really happen? Wow. But I'm curious: what would be the use of conducting a test-screening-type poll the day before release? It would obviously have been too late to change anything should the poll have indicated that that was necessary.

If they were suddenly having second thoughts and wanted to prepare... uhm ... spin...

[Zine2]

Ziggeh wrote...
You'll find that many polling companies don't understand bias.

I think it would be truer to claim that many polling companies deliberately introduce bias to support certain aims.
 

One internally conducted by someone with no track record of quality polling? Yeah, assuming there bang on is too much credit. It's not impossible, but it's improbable.

I think we can't really tell either way unless we see the actual study data.

[Cazlee]

evisneffo wrote...

...Did that poll really happen? Wow. But I'm curious: what would be the use of conducting a test-screening-type poll the day before release? It would obviously have been too late to change anything should the poll have indicated that that was necessary.

There isn't any valid reason to do the test screening a day before release. That's why anyone that believes the author is seriously lacking in common sense.

[Ziggeh]

Zine2 wrote...
For all intents and purposes, BSN Player Base == Entire Player Base.

Dear lord. We're about as bias a sample as it gets. We are the very definition of self selection.

Added to that there are a number of issues with internet polling beyond random. For one, all the broohaha amounts to advertising for negative reinforcement. People with negative opinions will seek out negative opinions, so if people are aware BSN contains largely negative feedback, it will create a feedback loop, leading to higher "negative" traffic for any given poll.

Zine2 wrote...
Also, I'm assuming that the Bioware test audience poll had random sampling; at least within the confines of the "ME3 player" population. Leaking it a day before the game's release implies a deliberate test.

How do you image this was achieved among "its fans"?

[count_4]

Ziggeh wrote...

count_4 wrote...

Ziggeh wrote...
I'll grant you it is more likely representative than "other polls" conducted through the internet

It's a poll held by the developer with the sole purpose to produce reliable results to predict the response of a greater audience. 
It's pretty safe to say that it is representative. Otherwise it would just be a huge waste of time.

As I say, polling bias is a deeply misunderstood area. You can't assume it's absence.

Just so we're on the same page here: Could you explain what kind of bias you are referring to? 
I'm in no way proficient in this area so I'm just comparing the bias in internet polls with the dev poll here. For me the bias in internet polls comes from the fact that unhappy players are more likely to vote than happy ones a.k.a silent majority.
As the dev poll requires everyone in the audience to vote after experiencing the ending without any prior knowledge, where would any bias come from that makes the poll non-representative (given the audience is diverse enough and not just a hundred 14yo)?

[Zine2]

incinerator950 wrote...
I'm referring to these polls now as opposed to his supposed control group poll.  It's generally a proven fact with gaming boards that more people post there to voice their negative opinions or seek help then to generally discuss something positive or make guides and help newbies. 

"Generally" is the key word however, and like I said the problem with simply assuming is that you may end up throwing out good data. It's one thing to assume that BSN is not representative of the populace at large, but if it turns out the proportions for the major demographics are the same anyway then it's still an actual valid sample.

We won't know either way though unless we do see the methodology of the Bioware polls.

[Ziggeh]

Zine2 wrote...
I think it would be truer to claim that many polling companies deliberately introduce bias to support certain aims.

Oh they definitely do that. But those two aren't mutually exclusive.
 

Zine2 wrote...
I think we can't really tell either way unless we see the actual study data.

Agreed, but that uncertainty itself means you can't use it.

[Cazlee]

@count_4 : It's called selection bias but there's no proof that the poll the author is talking about actually exists in the first place so it's a moot point.

Modifié par Cazlee, 18 avril 2012 - 09:44 .

[MikeC99]

Torrible wrote...

Zine2 wrote...

Torrible wrote...
Edit: Only referring to the BSN poll

Which again makes your statement a complete and total waste of time.

Again: We already HAVE data other than the BSN poll, which has NONE of the problems you outlined. Survey says? 90% hated the ending.

Now, backtracking to the BSN poll, we find very similar results. Why? Because you completely and totally ignored the concept known as "sample size". We can, in fact, use the poll results from 1,000 people to project the opinions of millions of others within a specific margin of error.

The only way to impeach a huge sample size is to demonstrate - using other data - that it is in fact significantly skewed due to selection bias. But you didn't prove that - you only made unsubstantiated accusations while at the same time completely ignoring a different poll that showed very similar results and yet had none of the problems you outlined. Similarly, claiming "hardcore" or "casual" fans skew the polls is nothing more than more ubsubstantiated accusations on your part. A proper test screening would, in fact, include casual fans as much as hardcore ones. That's the point of a test screen.

In short, this is nothing more than you rejecting good data because it proves you wrong.  The Bioware Poll is assuredly clean and returned a 90% "the ending sucked!" result. The BSN poll's selection metholology has been called into question, but has a huge sample size to mitigate it, and having very similar results to the Bioware results merely further confirms its validity.

So again: 90% of folks who've seend the ending think it sucks. The ending objectively sucks. End of story.

The sample is collected in a way that a certain type of people are more likely to respond than others. Therefore it is biased. Extrapolation is only viable if the sampling is not biased. Therefore you can't extrapolate that data to the entire population. End of story.

And you have no idea what objectivity is.

Sample is biased? yes. By definition, all samples are - just some are statistically random enough to deal with statistically. No statistical analysis 'proves' anything. They are tools that provide - usually to 95% confidence - that the analysis likely indicates that the patterns detected fit or characterise the full population.

90% of survey respondents hate ending = 90% of ME players hate the ending. Agreed. No, it doesn't.

Statement that those 90% who responded to survey are 'minority'. Well obviously numerically they are a minority of all users. But you seem to infer that 'minority are end haters is wrong assumption' = 'therefore majority of users are not end haters'? Are they? There is an onus of proof both ways and simply pointing to the pitfalls on one side provides no substantive argument for any alternative interpretation or view.

One thing you could do - trawl through and record every article, survey etc concerning the ME3 ending. Collate and compile and tell us what results you come up with re pro and con the ending.

I totally agree that often what surveys say is not what people think they say.

"Therefore you can't extrapolate that data to the entire population. End of story." Well, you can extrapolate - there are just risks. But equally, in the absence of figures, data, collation of info etc that indicates to the contrary, using the information that is available is not flawed, but it is limited and yes most people don't acknowledge the limitations.

Re BSN poll - what basis do you have to assume that because the poll is based in BSN that it is a biased sample in terms of good or bad ending? I don't see that as proved at all. It's a 'biased' sample only in so far as it would be reasonable to assume that only people who have an interest in Bioware games would bother to register. I just don't see any logic in assuming that BSN = end haters per se. End lovers had the same opportunity it seems to me.

Oh - and was this a deliberate juxtaposition : "entire population. End of story."?  Pithy summary of ME3, yes?
cheers
MikeC

[Zine2]

Ziggeh wrote...

Zine2 wrote...
For all intents and purposes, BSN Player Base == Entire Player Base.

Dear lord. We're about as bias a sample as it gets. We are the very definition of self selection.

Added to that there are a number of issues with internet polling beyond random. For one, all the broohaha amounts to advertising for negative reinforcement. People with negative opinions will seek out negative opinions, so if people are aware BSN contains largely negative feedback, it will create a feedback loop, leading to higher "negative" traffic for any given poll.

Dude, check the statement before that. I did not say BSN Player Base == Entire Player Base. I said "If X, Y, and Z hold true, THEN that is true".

You're the one making the assumption that internet polling is always beyond random. It often is, I'm pointing out it's not always.

You can say that data is questionable because it may have selection bias; therefore you should not extrapolate and merely use it as a dipstick.

But you cannot prove that it actually has selection bias simply because it's an Internet poll. You have to show that it does in fact significantly diverge from the results of a poll that DOES conform to random sampling.

Zine2 wrote...
Also, I'm assuming that the Bioware test audience poll had random sampling; at least within the confines of the "ME3 player" population. Leaking it a day before the game's release implies a deliberate test.

How do you image this was achieved among "its fans"?

Competently, like a major company should?

Modifié par Zine2, 18 avril 2012 - 09:48 .