89 réponses à ce sujet

[MerricksDad]

WhiZard wrote...
When I am looking for a good module to play, I am not particularly looking for a catchy sound track or an impressive tileset.  My concerns are based on how engaging the plot is, how developed the theme is, how much does it cater to a variety of playstyles, and similar fundamental properties of a module.  The supportative stuff, yes can be of quality, but I want the module itself to be quality not just popular.

Maybe such as system could be created for modules like you might see on video game ad sites, where the user has a few subtopics to vote on. The system could create from that an overall score. You'd probably need to save both for quick page load, but the math shouldn't be too strenuous on the system either if you just wanted to save the categorized subscore and calculate on the fly every view.

Something like this comes to mind:


And since modules are really new games, why should they not be rated as one?

Edit:
I also completely support PHoD's lengthy and entertaining take on this topic.

Modifié par MerricksDad, 22 avril 2013 - 12:18 .

[Rolo Kipp]

<posting...>

There's no way to do justice to my reply to PHoD on this little thing, so I'm going to ignore him completely (for now) and just peek at him now and then to see how he handles the sting of my neglect...

@ AD: Technically, this is the third Vault. I actually posted regularly on the *original* original Vault, though only (some of) my Fan Fics were transferred over. Oh, and one interview.

I also like the Hall of Fame as a concept. With more than 6,000 mods to sort through, and limited playing time, that was the first place I looked for a playlist when I returned to NwN.

But, actually, there's an organization that specializes in reviewing and rating mods, the AME.

I'm all about metrics and providing as many possible ways to search and fine-tune choice as possible, but I think the AME has always done a better job at this aspect of it.
Caveat: The reviewers Guild was AME quality, IMO.

So (peeking at PHoD), maybe we integrate the AME reviews to produce that particular search view? Why reinvent a pretty damn nice wheel?

@ MM: that's something like what I was thinking, but not focused on mods. And as Funky & Whizard suggested, weighted. So lightweight users would only be able to flag posts as "useful" or not flag them. Posters who gained weight (or were grandfathered in with weight), could rate a project.
And Mods, the end product, we leave the reviews in the hands of the specialists.

*throwing a bone* We, none of us, do this for money. We do it because we love to create. But, for me and most others, we *do* like it when we see some form of appreciation for our massive effor and cunning ingenuity.
That's where the comments, ratings, use counts and reviews come in. They are all the payback we're likely to get and I'm personally not willing to exterminate anything that encourages Creation.

My purpose in starting *this* discussion is to try to make such things more meaningful and rewarding.

<...from his tiniest scrying device>

Modifié par Rolo Kipp, 22 avril 2013 - 02:33 .

[Tarot Redhand]

I tend to think that having a scoring system of whatever sort actually helps to generate more comments. It's the "I really liked/hated that so I will rate it with a score" effect that tends to give an incentive to make a comment.

@MerricksDad I'm just curious, what does a score of 1 look like?

On a side (but related note), earlier today I sent a message to an organisation via their web page and instead of the barely readable list of letters and/or numbers they had something I thought was rather novel. They simply had the question "What is 2+2" and a little box to type your answer. Is this viable for the project?

TR

Modifié par Tarot Redhand, 22 avril 2013 - 03:48 .

[MerricksDad]

@TR, do you mean a score of 1 on a suggested mod rating system? I have no idea at all. Just a branch in thought. I wouldn't of course be suggesting to use the exact rating system in the picture either. Its just to show potential sub-mechanics. I do realize the actual picture has at least two topic we probably have no interest in because they are engine related more than custom content related: Graphics and Sound in the image. There are some things in that image that I think all people could use to describe mods though, especially fun factor and challenge.

Or do you mean, if a person only had a rating of one on their mod via some other rating? And do you mean how it looks to other viewers? If that's the question, In a point system, weighted or not, I would say a score of one looks horrible from a distance, or if the viewer is actually looking at the entry data, it might just simply say its very new, or simply not yet voted on by many. That should be very apparent on the VPP at all costs. New content should never appear as junk just because its new.

Given the suggested mod rating system, a score of one would have no doubt about it, it would be bad. The reason being that somebody would have actually taken the time to key in each of the X number of sub mechanics. We'd have to discuss sub mechanics and what not to determine which aspects would be used to rate mods. And then again on how to divide the whole to give a rating, and also what the scale of rating would be. I assume the image has a rank of 0-5 for each category, but not sure.

I've been eyeballing ratings systems out there on game magazine pages, and most of them just use sliders and user input. Others require a written review to be able to post a score, but really its not much different than the comment+vote system on the OV. But check out stuff like the links below, which I suspect Rolo has already been doing.

GameSpot
MetaCritic

But I think Rolo is onto something here if he can instead link to external reviews that are already of professional grade. And of course, that would require whoever the raters are to continue the good work and also to fetch data from here rather than the dying OV. I don't suspect that will be a problem, especially if we can save the vault and improve upon it before its blows away like dust.

[PJ156]

Rolo Kipp wrote...

*throwing a bone* We, none of us, do this for money. We do it because we love to create. But, for me and most others, we *do* like it when we see some form of appreciation for our massive effor and cunning ingenuity.
That's where the comments, ratings, use counts and reviews come in. They are all the payback we're likely to get and I'm personally not willing to exterminate anything that encourages Creation.

My purpose in starting *this* discussion is to try to make such things more meaningful and rewarding.

Hearty agreement from me in this respect. I don't create for vote nor comments but I do appreciate them for two reasons:

One - A vote tells me that one person, just one person per vote has actually played my work and thought about it. A download does not tell me that. My module took five years. I don't mind that but I want to confirm that someone has played it. A comment does that too but a vote is more informative to me, you can sort modules by votes you can't sort them by comments.

Two - A vote and a comment show that the person who downloaded my work has the decency to give up fifteen minutes of thier time, to pay back my many hours, by telling me what they think of it. I don't mind if they think it sucks or they love it, if they tell me why it can all go into the melting pot.

For me, a voting system needs to be explicit. I am not a fan of a simple endorsement systems as I like HoF type tables. They make mods, that the community as a whole think are good, accessible to new players. Those players are more likely to stay if they are introduced to the best first.

I also prefer a vote/comment system because it allows a browser to see what they might be getting. A mod may be high on the list but turns out to be a guitar simulator. Technically great but not very representative of other content. It also allows the voter to give all impotant feedback to the writer. I like the earlier post that suggested the mod could be broken down into key areas and voted on in sections, for roleplay, for technical content etc, but I would be wary of over complicating things. It's hard enough getting people to use simple systems leat alone more complicated ones.

My twopenneth in a nutshell,

PJ

Modifié par PJ156, 22 avril 2013 - 05:05 .

[The Amethyst Dragon]

Rolo Kipp wrote...
...exactly zero donations (not that I'm really expecting them, understand)...

Creating and hosting this sort of stuff takes a lot folks, so let's push over a few dollars/euros/etc.

[henesua]

I agree with the need for Votes and Comments but in different measures than the "Vote" proponents. For me the comments are the most important thing. As someone who primarily works on scripts, I want feedback on my work. I want to know what didn't work for someone so that I can improve it. Criticism is more valuable than encouragement to me, but I'll take either. And its difficult to get that without comments.

Also helpful is having a discussion area associated with a project. I really like that "forum" link on the vault which we can use to point back here. I am disappointed that no one ever says much, but the potential is there and sometimes threads get lively. But the Neverwinter Nexus and the BSN projects enable you to create threads specifically tied to a project. I like this even more.

Perhaps one of the things about the voting on the vault that doesn't appeal to me is that the wide scale from 0 to 10 is ignored. I think most people are overwhlemed by the wide scale and simply vote at the ends. It is difficult for them to construct criteria or at least more effort than they are willing to apply when all they really want to do is let you know if they liked or disliked something. I think the Nexus gets around this entirely with a "like" feature - their "endorse" button. I think a 3 point scale could also work. Negative, Positive, In the Middle. But given the lack of thought most people put into voting (most do not vote), I think the simpler the better in the hopes that more people will participate rather than just downloading.

[edit]
In other words I am (more or less) with PHoD on this one.
For finding content, better means to organize them is the a more important thing in my view than user ranking. I think a TAG system as well as a CATEGORY system is essential. And as viewers gain status in the community their ability to create TAGs and CATEGORIES should come on line. I think that is where KARMA comes in.

I've seen KARMA work best on question and answer web sites. I'm still not sure how it would work at the vault, but I think its a good idea and like what I have heard with regards to implementing it so far.

Modifié par henesua, 22 avril 2013 - 06:35 .

[NWN_baba yaga]

I dont remember the last time where i not gave a 10 to be honest. But a 10 in this time (for me) means much more then just pure quality... as i almost only vote for modeled/ textured content.

It has more to do with how i judge someones work in terms of his enthusiasm and love for the game or the content he creates. Someone who just started to mod for nwn and you can clearly see he/ she spent some time (even hours) and thoughts behind his content shouldnt be judged like CODIs work. No one could just get a 10 for pure gfx or technical aspects of modeling except someone like BatintheHat who is the unbeatable champion of modeling/ texturing up to this day! his stuff is so good it´s timeless!

and a 0 is for guys like barry... LOL!

Well i was a bit offtopic again right? hmm. i like the vault system and i miss the days of the many comments and huge activities down there... yeah i really miss those days:crying:

Modifié par NWN_baba yaga, 22 avril 2013 - 06:43 .

[painofdungeoneternal]

Treat them as separate problems

1. First problem is content being there to begin with. This is simply preserving the Vault and what it is, and is of course the only current priority.

2. Voting is a completely different issue, and frankly there are merits to multiple systems. The criteria for any system should be based on some metric ( end user input, downloads, playtime, number of times playing, number of users at any one time ) which once understood becomes metrics ripe for abuse. But by having enough metrics, the abuser cannot manipulate every input well enough to not be noticed. The basis of any system should be usage metrics, which is the second problem.

3. The goal and purpose though is presenting the end user with the best content first of all. Higher quality stuff should be easier to find, a user should not have to dig thru ten pages to find project Q, CEP, or the other must have content. ( and this should not be subject to favoritism, SEO trick keyword spamming, or the like ) I'd call this scoring.

4. Secondary is feedback and support to the content authors, which is largely thru pat on the backs, and thru user feedback, not to mention discussion. I'd call this feedback. It should be judged on how many end users decide to make a comment of some sort, if there are 20,000 downloads and 5 comments, this is not working very well.

I personally don't think there is one solution for the last 2 problems, and any solution should support multiple solutions for 3 and 4. Problem 2, should support a lot of inputs, some of which hidden, and from multiple sources ( the VPP, the Nexus, etc ), it's more about tracking data and correlating it in a central location, and providing it to system who does scoring. Regardless we need to start at problem 1, then figure out how to get valid numbers on the usage metrics.

Problem 3 I'd like to allow it to be open enough to allow multiple scores. This seems more complex, but it's actual laziness on my part, i'd prefer to deal with part of the problem and make tools which enable others to deal with the much harder problem of how to rank content. I see manually tagging ranks on things, and also doing it based on metrics ( downloads, votes, etc ) as both being important ( and there are examples of each in the various search engines ).

Modifié par painofdungeoneternal, 22 avril 2013 - 07:19 .

[MagicalMaster]

Rolo Kipp wrote...

@ MM: that's something like what I was thinking

Er, I haven't posted in this thread until right now

I don't like the idea of a thumbs up/down either, for various reasons mentioned.

However, as Henusa somewhat just beat me to the punch on, I think part of the problem is that there are 41 possible rankings on the vault currently.  What deserves a 9.5 vesus a 9.25 or 9.75, for example?  And if we're trying to make distinctions between those, it's only natural to expect to see most people voting 9+ (because even at the 9+ mark there are five possible rankings).

I think either a four or five star ranking would well, something like...

1: Avoid
2: Average
3: Good
4: Excellent

and possibly

5: Amazing

I think people would feel better about voting a 3 or 4 if they're only 1 or 2 rankings "down" from the highest and are still portrayed as being good or better.  And even if you still see a lot of 5s, the main goal is to be able to indicate what modules are extremely well done, not try to arrange them in a precise order.

Hell, I'd probably argue that at most you should list modules alphabetically for each category of score.  So assuming we have 1-5 and module a has a 4.1, b has a 4.9, c has a 5.0, d has a 4.3, and e is a 4.0, it would still be...

a
b
c
d
e

Because the categories would be 4.0+, 3.0+, 2.0+, and 1.0+.  Make people go to the individual pages to see the actual average so they can read the description and comments.  Don't have the average available outside of the page itself.

Modifié par MagicalMaster, 22 avril 2013 - 07:20 .

[Claudius33]

PJ156 wrote...

Hearty agreement from me in this respect. I don't create for vote nor comments but I do appreciate them for two reasons:

One - A vote tells me that one person, just one person per vote has actually played my work and thought about it. A download does not tell me that. My module took five years. I don't mind that but I want to confirm that someone has played it. A comment does that too but a vote is more informative to me, you can sort modules by votes you can't sort them by comments.

Two - A vote and a comment show that the person who downloaded my work has the decency to give up fifteen minutes of thier time, to pay back my many hours, by telling me what they think of it. I don't mind if they think it sucks or they love it, if they tell me why it can all go into the melting pot.

I second PJ for the exact same reasons. I am very happy when a player writes a mini review of my mod or writes he just registered to comment it. As a moder you are eager that other moders use your creations and that players play your modules. Above you can’t wait for getting feedbacks, of course positive  , but also constructive criticisms.

No matters the type of vote, as the Vault today, A to E, stars split in graphics, fun, ... I probably would prefer A to E or stars, just a personal opinion. I am not a big fan of classification, 9.xx vs 9.yy (damn! I did it in my news ).

So I would prefer simple lists of modules/creations with a rate (4 stars, 3 stars, or whatever) rather then a ranked list. Of course moderation would be welcomed because I totally understand why Tsongo advises to scrap votes. A very few (immature or irrespectful :innocent:?) people can kill any willingness of submitting something. I could live with no voting all, however I still feel that's a way to reward quality submissions.

As other have written, categories/tags are interesting for modules eg. length (gameplay hours), setting (D&D, historical, SciFi, ...), story driven / sand box, hack & slash, riddles, thievery, ...

However, IMO, what drastically matters is the visibility. Therefore, whatever the voting/rating system, I am a huge fan of rotating lists  on the front page, so every module/creation gets a chance :

• New modules or creations (current year + last year)
• Highly rated modules or creations (curent year + last year)
• Highly rated modules or creations (old goodies)
• Any modules/creations whatever the rating and the submission date.
• List by categories.

In addition, a topic open to mention  ‘underrated’ modules or creations should be opened.

Hope it helps.

Modifié par Claudius33, 22 avril 2013 - 08:03 .

[FunkySwerve]

MagicalMaster wrote...
Hell, I'd probably argue that at most you should list modules alphabetically for each category of score.

Allow me to introduce the latest iteration of my PW,  ________AAAA Aaron's Higher Ground PoA Legends!

Phonebook syndrome, no thanks. Either a raw list by score, or ORDER BY RAND() the display order each display.

Funky

Modifié par FunkySwerve, 22 avril 2013 - 08:31 .

[Gruftlord]

a simple like/endorse system would be the hardest to abuse. you can only like a contribution and never dislike it.
then add search functions based on number of likes, number of downloads and lumber of likes per download.
also the less you activelly support/spotlight mods that gather high ratings (irrespective of the rating system) the less likely the system will be abused.

at this time and age i cast my vote for the system, that is least likely to be abused (or that is the least abusable) by random internet trolls and spam bots.

my two cents as a mod user

[Tarot Redhand]

I have seen a number of different proposals for scoring systems on here and they all have the same small but significant problem. They are all based on an odd-number. This is a something that many courses both academic and vocational teach you to avoid because people will tend to pick a middle number if you give them a choice. Better to use an even number be it 4/6/8 or whatever.

TR

Modifié par Tarot Redhand, 22 avril 2013 - 08:48 .

[MagicalMaster]

FunkySwerve wrote...

Allow me to introduce the latest iteration of my PW,  ________AAAA Aaron's Higher Ground PoA Legends!

Point taken, I didn't consider people trying to get to the "top" of the list (since the "top" would have no inherent meaning).

I was trying to see if we could get away from the idea of a "top 10" list or something, come up with a system where people can generally see what modules are considered to be roughly on par with each other and then they look at the actual modules.  Which also reduces the impact of a module being rated 9.9 versus 9.8.

FunkySwerve wrote...

ORDER BY RAND() the display order each display.

Would be fine as long as the order changes each visit or something and it's made clear that it is random.

Tarot Redhand wrote...

I have seen a number of different proposals for scoring systems on here and they all have the same small but significant problem. They are all based on an odd-number. This is a something that many courses both academic and vocational teach you to avoid because people will tend to pick a middle number if you give them a choice. Better to use an even number be it 4/6/8 or whatever.

Hey now, I gave a system for four!*

*with the possibilty of a fifth if desired, so I'm still semi-guilty.

That said, I don't think what you're saying is entirely accurate - if they played a module and really liked it, they aren't going to vote for a 3 out of 5 because 3 is the middle.  If you asked them if they thought it was a 3, 4, or 5 they might pick 4 because it's the middle of those, though.  Framing is important.

Modifié par MagicalMaster, 22 avril 2013 - 09:16 .

[Tarot Redhand]

@MagicalMaster, I didn't put names to it and I didn't say all. All I am saying is that it is something to be aware of. One thing I think definitely needs to be given the order of the boot is fractional scoring, So no more 9.5, 9.6 etc.

TR

[MagicalMaster]

Tarot Redhand wrote...

@MagicalMaster, I didn't put names to it and I didn't say all.

Tarot Redhand wrote...

I have seen a number of different proposals for scoring systems on here and they all  have the same small but significant problem.

>.>

Tarot Redhand wrote...

All I am saying is that it is something to be aware of. One thing I think definitely needs to be given the order of the boot is fractional scoring, So no more 9.5, 9.6 etc.

Indeed.

Modifié par MagicalMaster, 22 avril 2013 - 10:09 .

[ColorsFade]

PLUSH HYENA of DOOM wrote...

IMPORTANT:- A whopping, dirty great Archive of Community Content (hopefully continuing to grow).

UNIMPORTANT:- Pretty much everything else.

I just want to second this before I go on to write about voting... Preserving the data seems paramount. 

Now, about voting... 

I can only tell my side of the story, as a user. 

When I decided to "reintroduce" myself to the NWN2 world, I played through the OC and about half of MotB before I started to get the itch to take a look at the Toolset and do my own thing. But I decided that before I embarked on learning the Toolset, I wanted to see what kinds of content other people had created with it. But I have limited time, so I didn't want to play just anything, or to play the modules with the most downloads. I wanted to play the stuff that the community, as a whole, generally considered the cream of the crop. 

To that end, the voting system was invaluable to me.

Seeing the Hall of Fame tags was great; It gave me a really concise idea of what was considered good, quality content. Seeing the high aggregate ratings also helped. Seeing the Top lists also helped. 

It was through all of those votes, awards and whatnot that I was able to narrow down my first downloads to a few select modules that seemed to best represent the "best quality" work of the community. And from there, I was able to learn, play, have fun, and generate my own ideas for campaign. 

A site like Vault for NWN2 content is a lot like shopping on Amazon. There is a reason rating systems exist - so users can make efficient use of their time in trying to find what they want. Yes, comments are ultimately the most valuable in this regard, but that's data that requires a serious time investment to parse. A simple five-star rating system allows a user to make efficient use of their time by consolidating the data. 

If it were me, I'd have a rating system in place. And if possible, I'd add additional tags to the content, for things like Hall of Fame, etc. I think it's still worthwhile and relevant to let someone know, "Hey, this content was moved to Vault 2.0, but back in the day, this module won a Hall of Fame award in 2007". To me, as a consumer, that's useful information. It lets me know that at one point in time, that content was considered the highest quality. 

[Tarot Redhand]

Oooops, Sorry. Well I thought I hadn't. Getting senile. Bad Kitty.

TR

[Rolo Kipp]

<scribbling...>

Just a little update from the little screen. My boss is back from Bali with a boat load of exotic materials and jewelry, so I won't be near a computer for a few days.

Great discussion here, thank you! :-)

@ Pain: #1 is the migration, yes :-) but I'd like to keep in mind incorporating metrics generation in the API.

@ MM: I was thinking MD, but you were on my mind :-P Glad you added more, though :-)

Current set up uses heirarchal categories and free form tags. Every project (and post, for that matter) has a Talk tab for comments and discussion. Each project has an arbitrary number of Related projects" links that may be added, to link to other projects on V1, V2, self-hosted files or wherever the author want to link (like these forums).
I'm keeping the current ten point vote in place until the migration is complete.
Awards & badges from V1 will be retained, though I haven't set them up yet.

Future plans are in flux, but are already heavily influenced by this discussion.
I'm going to at least try out a multi-level karma system in parallel with usage metrics and try to work some sort of API out with AME (when I get a chance to email Andarian ;-).

Let me ask one very focused Q next: what info should be blasted on the front page?
V1 floods the front page, IMO. I'd really like to simplify it and just make it the "news" page. Seriously thinking of setting up dashboards for users that they can customize, but for now: What should be on the front page?

<...nonsense, near enough>

[Tarot Redhand]

One thing if at all possible, can you insist that any content mentioned on v2 is hosted on v2. You find it all over the web, broken links to great sounding stuff that is no longer where the link says it is. I for one would hope we can avoid that by having all content that has its own page hosted locally.

Will think about front page when I get chance.

TR

[FunkySwerve]

Tarot Redhand wrote...

One thing if at all possible, can you insist that any content mentioned on v2 is hosted on v2. You find it all over the web, broken links to great sounding stuff that is no longer where the link says it is. I for one would hope we can avoid that by having all content that has its own page hosted locally.

Will think about front page when I get chance.

TR

That may be asking a lot of someone who is paying for said hosting out-of-pocket. I empathize with your frustration over broken links, though.

Funky

[painofdungeoneternal]

To do what the vault did, requires either dedicated servers at colo facilities which we own, or using AWS ( amazon web services ) or the like. This is easily over $70 a month, and probably will end up being closer to $200, since you pay for both storage and how much bandwidth is used. Unless we have big contributors, who finance this sort of thing in perpetuity, it's not going to happen.

I think we can set up a system which prevents any broken links. But at the same time having everything on the VPP at once, there is no way a hosting provider would accept that without either capping the band width, kicking us off, or just having the server not be accessible due to people downloading multiple times.

There has to be a final authority copy which has everything, but at the same time there needs to be a wide distribution of where they are located, just to ensure the main copy does not get over loaded, with some files hosted on dropbox, box.com, nexus, and the like, yet also on the VPP systems just in case those go down. Basically we will need both people willing to set up mirrors, and also just willing to buy a large 2TB hard drive and keep a back up copy on their local system ( which we already have happening ). The system has to be such that only when the last community member gives up will it stop working.

If a link is broken, and all the links are redundant and in more than one place, you'd just have to download it from another place. ( and this should be something I can automate )

[FunkySwerve]

painofdungeoneternal wrote...

To do what the vault did, requires either dedicated servers at colo facilities which we own, or using AWS ( amazon web services ) or the like. This is easily over $70 a month, and probably will end up being closer to $200, since you pay for both storage and how much bandwidth is used. Unless we have big contributors, who finance this sort of thing in perpetuity, it's not going to happen.

If you have to resort to Amazon, it'll probably be more than that. HG ran on Amazon for a good while, and was coming in at $300/mo. We probably had more bandwidth than you would need, but a lot less storage.

Funky