362 réponses à ce sujet

[zarnk567]

nice job, OP.

[MikeC99]

DamonD7 wrote...

I contributed to the poll, and that certainly looks all official and stuff. It has pie charts, people, pie charts!

Good job putting this all together. All data is good data, in my line of work anyway.

... and many thanks for doing that! Please keep tuned in over next couple of days - we're hoping even more folks can come on board for the follow up survey. From that we need to distil real answers - just makes my head hurt thinking about it!
cheers
MikeC

Modifié par MikeC99, 19 avril 2012 - 10:37 .

[Gwtheyrn]

Well done. Very well done. My hat is doffed to you, sir.

[Aurica]

Wow... brillaint stuff OP!

[Kreid]

Thank you so much for doing this.

[HannibalSW]

Bravo, great job on this. I'm impressed with the outcome. I had no idea when I took the poll it would end in such a detailed report.

[lil_lene]

This was a fantastic read OP. Thanks. The detail is amazing. Reminds me of my university days..

[Swordfishtrombone]

A lot of work went into that one, impressive! Somehow missed that survey myself, but had I responded, my answers would have pretty much gone along with the majority on each category.

[MikeC99]

lil_lene wrote...

This was a fantastic read OP. Thanks. The detail is amazing. Reminds me of my university days..

Oh man - I hear you!!!!!  Thanks!

Creid-X, Aurica, HannibalSW and Gwtheyrn - Many thanks for the support. Much appreciated - on behalf of ofish and myself.
cheers
MikeC

[MikeC99]

Swordfishtrombone wrote...

A lot of work went into that one, impressive! Somehow missed that survey myself, but had I responded, my answers would have pretty much gone along with the majority on each category.

Many thanks - there will be a follow up poll to try to get to some nitty-gritties to get to Bioware, so please stay tuned and participate in that one!
cheers
MikeC

[paul165]

An excellent analysis and I look forward to contributing to follow up surveys as required.

[Unfallen_Satan]

This is a lot of hard work, so kudos! However, I have two concerns: one regarding the validity of the results and one regarding the relevance of the study.

I only took a couple of statistics courses in college and grad school, so I am not in a position to review your work. However, I think it essential that a paper with this level of technical details be either peer-reviewed, if you are a practicing statistician, or reviewed by your instructor, if you are a student of statistics. Specifically, your case will be much stronger if other statisticians agree that your results were derived correctly from available data using the appropriate methods.

One example that troubled me almost immediately is your second conclusion:
"There is a demonstrable level of understanding of the issues, and a commitment to pursue this understanding further."

First of all, how did your survey allow respondents to demonstrate any understanding of the issues in question? Granted, the various reasons for dissatisfaction with the ending are self-evident for many. However, what tools did you use to ensure that understanding is assessed both accurate and consistently among all respondents? Secondly, on what ground did you discover the "commitment to pursue this understanding further"? No question regarding intentions was include in your survey. Furthermore, what is your definition of "commitment to pursue this understanding"?

Technical review aside, the validity of your survey as it pertains to the greater ME3 gamer community is also in question. You study had 2,076 participants, and you mention another poll on BSN with "over 71,591 responses" where some 92% of people expressed dissatisfaction with the ending. Currently, Mass Effect 3 has a documented non-digital sale of over 1.3 million. While statistical sampling is a well-established practice, what measures have you taken to ensure that the respondents to your survey make up a representative sample of the general ME3 player population? The nature of your poll has high affinity for people who are dissatisfied with ME3's ending. Although I cannot prove this next point, it is my experience that people's participation in online polls and discussions varies positively with their interest in the issue. Do you think it's fair for me to conclude that your poll disproportionally sampled, by a large margin, people who are dissatisfied with ME3's ending and do not accurately represent the ME3 player population at large?

The other survey you mentioned, having more diverse choices, would seem to be more indicative. Without a close examination of that poll, it is difficult to judge its merit. However, even conceding this point, I still find that poll or yours insufficient at justifying further action based on the results. Of the 92% respondents of that poll who were unhappy with ME3's ending, what was the distribution of the degree of unhappiness? What proportion of those unhappy with ME3's ending feel a new ending is warranted? What proportion of those unhappy have reasons to believe that there is an ending that will satisfy the majority of ME3 players? These and no doubt questions by others need addressing to more accurate interpret the result of your poll and the other one, especially if you conclude that the results justify further action.
EDIT: This point came to me a little late. In light of BioWare's recent announcement to offer an extended ending DLC. What proportion of players still believe it is necessary to take additional pre-emptive action? What about those that now prefer to wait and see how the DLC will turn out?

Please forgive my forwardness with my concerns. I have read hundreds of papers from scientific journals and critiqued dozens of them over my college and grad school education. I make it a habit to critically view all papers of a technical nature. I think we will all be better for it.

Modifié par Unfallen_Satan, 19 avril 2012 - 11:21 .

[GreyLycanTrope]

Bioware did say they still wanted feedback, well done.

Modifié par Greylycantrope, 19 avril 2012 - 11:19 .

[GreyLycanTrope]

AwesomeName wrote...

People who hated the ending were probably more likely to even be aware of the survey and have the most motivation to take part in it. I wonder if that was taken into consideration.

This survey was ment as a comprehensive view of why people disliked the ending not of how many

[Rothgar49]

AwesomeName wrote...

Greylycantrope wrote...

AwesomeName wrote...

People who hated the ending were probably more likely to even be aware of the survey and have the most motivation to take part in it. I wonder if that was taken into consideration.

This survey was ment as a comprehensive view of why people disliked the ending not of how many

If it's meant to be "comprehensive", then it's something they need to take into account.

Perhaps, I think the OP has done very well considering there was no requirement on them to produce this. It's an excellent piece of work and certainly worth Biowares attention

[MikeC99]

Unfallen_Satan wrote...

This is a lot of hard work, so kudos! However, I have two concerns: one regarding the validity of the results and one regarding the relevance of the study.

I only took a couple of statistics courses in college and grad school, so I am not in a position to review your work. However, I think it essential that a paper with this level of technical details be either peer-reviewed, if you are a practicing statistician, or reviewed by your instructor, if you are a student of statistics. Specifically, your case will be much stronger if other statisticians agree that your results were derived correctly from available data using the appropriate methods.

One example that troubled me almost immediately is your second conclusion:
"There is a demonstrable level of understanding of the issues, and a commitment to pursue this understanding further."

First of all, how did your survey allow respondents to demonstrate any understanding of the issues in question? Granted, the various reasons for dissatisfaction with the ending are self-evident for many. However, what tools did you use to ensure that understanding is assessed both accurate and consistently among all respondents? Secondly, on what ground did you discover the "commitment to pursue this understanding further"? No question regarding intentions was include in your survey. Furthermore, what is your definition of "commitment to pursue this understanding"?

Technical review aside, the validity of your survey as it pertains to the greater ME3 gamer community is also in question. You study had 2,076 participants, and you mention another poll on BSN with "over 71,591 responses" where some 92% of people expressed dissatisfaction with the ending. Currently, Mass Effect 3 has a documented non-digital sale of over 1.3 million. While statistical sampling is a well-established practice, what measures have you taken to ensure that the respondents to your survey make up a representative sample of the general ME3 player population? The nature of your poll has high affinity for people who are dissatisfied with ME3's ending. Although I cannot prove this next point, it is my experience that people's participation in online polls and discussions varies positively with their interest in the issue. Do you think it's fair for me to conclude that your poll disproportionally sampled, by a large margin, people who are dissatisfied with ME3's ending and do not accurately represent the ME3 player population at large?

The other survey you mentioned, having more diverse choices, would seem to be more indicative. Without a close examination of that poll, it is difficult to judge its merit. However, even conceding this point, I still find that poll or yours insufficient at justifying further action based on the results. Of the 92% respondents of that poll who were unhappy with ME3's ending, what was the distribution of the degree of unhappiness? What proportion of those unhappy with ME3's ending feel a new ending is warranted? What proportion of those unhappy have reasons to believe that there is an ending that will satisfy the majority of ME3 players? These and no doubt questions by others need addressing to more accurate interpret the result of your poll and the other one, especially if you conclude that the results justify further action.

Please forgive my forwardness with my concerns. I have read hundreds of papers from scientific journals and critiqued dozens of them over my college and grad school education. I make it a habit to critically view all papers of a technical nature. I think we will all be better for it.

I was going to cut out just the most relevant points but decided to leave the whole quote so as not to risk putting things out of context.

First off - sincere thanks for reading and taking the time to seriously consider the report.

Quick background on me - not a professional statistician or a student. Just done a lot of stats for research over many years. So my first acknowledgement is: believe me I don't know far more than I do know! So  I try to stick to the simpler analyses.

OK - your points as I understand them:

Peer review. Yes, and no. Peer review is required for scientific publication, and should be undertaken for published work. As this is neither, there is no requirement for peer review. Also, the purpose of the statistics is mainly to inform understanding - this is not a technical paper or scientific endeavour, but an attempt to make sense of a broad ranging debate about what poeple don't like about the ME3 ending.

Demonstration of committment and understanding. Fair comment - IF the report was solely relying on the actual statistics to deliver all the information. Statistics are tools - that's all they can ever be. Tools don't build things - people do. The stats are in the report to try to provide some transparency, rigour and objectivity. But they are tools to do this. The demonstration of committment and understanding comes from the fact that folks took time out to actually so the survey. Understanding is from the fact that the vast majority of them chose to select all options for each question. Your point is valid - these are not things that are derived from the statistics - I totally agree. But as I said this is a report based on a survey that uses statistics to inform the understanding of the results.

Validity of the survey as regards the wider ME community. Hmmm - this is a misconception as to the purpose of the survey and report. To quote: "The purpose of this poll was to examine the issues people had with the ending of Mass Effect 3...".  The statistical population the sample relates to is not the 1,3 million ME users, but specifically those who don't like the ending. As also stated in the report "The objective is not to balance dissatisfied with satisfied ... the purpose is to clarify why people are dissatisfied".  Also we tried to be clear that the purpose was to at least attempt a general consensus of the reasons why people didn't like the ending - not to poll yet again if they liked the ending.  The notion of random sampling gets over used and abused I'm afraid. Sampling does not have to be random - but if it's not there are severe constraints and caveats on how you can deal with the data. In the end, statistics are used to try and model the reality you are trying to define for the whole population. Any statistical analysis is wrong. Can I repeat that - wrong. Stats can inform you - and inform you very well - and assist greatly in making decisions about things, in determining the nature of things, but they are NOT THE THINGS themselves. Atsome point you extrapolate from the incorrect model to reality. The smaller the sample inrelation to the total population the more risk. It's not more wrong or right - just greater risj that it might be more wrong than you want it to be. The report does not purport to be science. It's what I term a 'soft' report in that it does not have the signifcant constraints required for a fully random, normal distribution based etc etc analysis. Apologies if this wasn't made clear - I did try to insert caveats, and warnings and statements about limitations at every opportunity!

The issue of BSN participants being a biased sample has been raised in these forums before. People misunderstand 'bias'in this sense. What information is there that says BSN users are negative per se? They areinterested parties - absolutely. But if you are undertaking an analysis to determine patterns within this population of ineterested parties, then them being ínterested parties'is not a bias at all - it's just a condition of the population you aresampling.  We did not set out to sample BSN - it just happens that most of the respondents came from BSN.

In summary - in theory all your points are valid, and I sincerely thank you for raising them. In practice, the report is not a technical statistical report. As stated in the report: the 'poll was designed to primarily to help focus and simplify the range of issues'. I'm happy it has succeeded at that.

Hope this answers your questions? It's a bit hard to keep track of things while writing in this confined text box!!!!!!
cheers
MikeC

[Unfallen_Satan]

I think we may even have other statisticians or student of statistics on thes forums. Before you send your study to BioWare, perhaps you should wait and let them have a chance to review yoru paper.

To everyone else, technical papers require much greater scrutiny than common opinion. I've seen some major mistakes in papers submitted for peer-review. I shudder to think what would happen if they all got published and accepted as valid. This paper isn't wrong, it just needs more examination.

[Cucobr]

congratulations OP.

This work is awesome to use as data for the BioWare.

I hope they have seen

[MikeC99]

Unfallen_Satan wrote...

I think we may even have other statisticians or student of statistics on thes forums. Before you send your study to BioWare, perhaps you should wait and let them have a chance to review yoru paper.

To everyone else, technical papers require much greater scrutiny than common opinion. I've seen some major mistakes in papers submitted for peer-review. I shudder to think what would happen if they all got published and accepted as valid. This paper isn't wrong, it just needs more examination.

Very happy for review - but I don't intend to specifcially organise it. As I said - this is not a technical paper. Just because a report uses statistical - or any other type - of analyses doesn't make it a 'technical paper'. I understand what you are saying, but I'm not sure you understand the very different basis for what is 'opinion' supported by analyses, and 'technical reporting'. Would I consider this as publishable as a scientific or technical paper - no. As I said - I've inserted a number of caveats, limitations etc trying to point out that there are flaw, what we've tried to do to address those in some palces, but basically the report also presents just the raw figures for any one to interpret. Again apologies if it all doesn't seem to be as obvious as you think it should be. But I'm comfortable that the report does what it says it was trying to do, and that the limitations are articulated.
cheers
MikeC

[Flextt]

Omg this is beyond my expectations when I filled out the survey! Seriously good job and looking forward to a reaction. Do you consider sending a copy to supporting media?

[MikeC99]

AwesomeName wrote...

Greylycantrope wrote...

AwesomeName wrote...

People who hated the ending were probably more likely to even be aware of the survey and have the most motivation to take part in it. I wonder if that was taken into consideration.

This survey was ment as a comprehensive view of why people disliked the ending not of how many

If it's meant to be "comprehensive", then it's something they need to take into account.

Thanks for the comments - sincerely I appreciate anyone looking at this critically - I mean objectively critical, not negatively per se.

Re the first point you made - as we said in the report we were trying to get to some kind of consensus as to why those who didn't like the ending didn't in fact like it. So the fact that people who 'hated the ending were probably more likely to even be aware of the survey...' is perfectly valid as an observation but has no bearing on the context of the survey.

Comprehensive? I'm not aware we even used the word? We were always aware of the limitations - but in the end we had to take what was offered and make the most of that.
cheers
MikeC

Modifié par MikeC99, 19 avril 2012 - 12:13 .

[SgtGlory]

Stunningly good work there, I'm happy to have participated now seeing these nicely polished results.

[Flextt]

Doesn't the focus on BSN allow for Bioware to ignore us even more easily?