Updates to Point System

Update (Sep 20): There is now a new topic on the revised point system. You can read it here:

Original Topic:
Today we have made few changes to our Point system, with following goals in mind:

  1. Ensure that contestants with consistently high rating get rewarded even more
  2. Higher penalty for excessive entry submissions which receive low ratings.
  3. Provide a level playing field for new users and existing users when it comes to earning additional points.

Here is a highlight of the changes.

  1. Contest Participation: Now contestants will earn flat 40 points for participating in a contest. Earlier, users could earn additional points by submitting more entries to the contest, however we believe that model was a bit flawed and created wrong incentive for users to submit more entries, even if they were not high quality entries.

  2. High Rated Entries: We have increased the points for 5 star and 4 star rated entries to 50 and 40 points respectively.

  3. Low Rated Entries: We have also increased the penalty for low rated entries. Two star ratings will deduct 10 points, and 1 star ratings will deduct 15 points. For example, if you submit 20 entries to a contest, and 10 of them receive 1 star rating, you will lost 150 points in that case.

  4. Winning a contest: You will earn 100 points for winning a contest

  5. Earning a bonus: If you receive a bonus from CH for any of your entries, you will earn an additional 75 points.

30 Day Leaderboard
We will also be launching a 30 day leaderboard on the site. This will show the top 25 contestants who have earned most number of points during the last calendar month. These users will receive a substantial amount of additional bonus points. This provides a level playing field for New and Existing users on the site because the Leaderboard bonus will only be based on their last 30 days of activity.

On the other hand, the bottom 25 contestants (who lost most points in a month) will lose additional points, and a participation warning. They will be given an opportunity over next 30 days to improve their rankings.

We recognize that the CH ratings may not always be consistent, however we feel any other mechanism to gauge quality would lead to more subjective outcomes. We will also implement additional guides for contest holders to ensure they use a consistent criteria for rating of entries.

2 Likes

Thanks for working so hard to make the system better for all of us! Should be interesting to see how this all plays out.

I want to repeat my request for a few sample contests for CHs to check out before they undertake ratings. As I said on another thread: “In the Door Selling Company Needs Professional Name” contest, a new post today: ‘Sorry for the delay, rated almost all of them. Based on the entries we have seen yet, we like the shorter 1 or 2 word names, with original words. So not sentences like : Complete door suppliers.’

“Is it fair that names that met the original brief (The Open Door, Door-to-Door Doors) were rewarded with 1s?”

Thanks, as always, for the chance to weigh in, and the respect you show creatives.

3 Likes

I’m going to echo Stalias on this: Briefs change, and there are CHs who use ones (either initially, or later) as a weeding process.
Does this start immediately? Or retroactively to entries that were already submitted? For instance, I received a 4 just now. I also submitted a few others without reading this.

I wanted to add something else: With these changes to the points system, I think SH is going to get a lot of “safe” entering. Some of the most prolific winners on this site, as well as ones with the most points, have achieved this because they are willing to submit entries in a wide variety of directions, take 5 point losses for the ones that don’t work, and do a lot of editing. SH will probably get fewer “bad” entries, but it also might not realize the editing process that sometimes accompanies the “ideal” names. Is this what we want? I foresee contestants giving up early because they’re afraid of reprimands, etc. I also submit on Crowdspring where I’m nowhere close to being able to compete in restricted contests because my naming score isn’t high enough. Yet, when I was invitedto these by a CH, sometimes finished in the finalists. I see SH as more of an “everyman’s” site than CS, and sometimes entries that sound nothing like the brief win. Please consider all this, and I don’t know if anyone wants to add their own experiences. Thanks.

4 Likes

I was half way posting to the unawarded contests thread, then i saw this. Anyway, here are some of my thoughts.

Maybe it’s too easy to click on the 1-Star, or CHs use it as a catch-all for something that’s missing from the rating system. What if the way the rating looks is different.

I think Stars should all give positive points, well 1-Star would maybe get a -2 which ends up being 0 points for the contestant. Shiny stars that end up punishing is a little misrepresented I think.

I think -10 and -15 is too drastic, maybe ratings are being used to solve a different set of problems.

So, we’ll add two more non-star clickable icons, one called “Don’t Get It” which is (-5), another one called “Horrible!” (-10). Non-star icons should be styled differently to stars, and should be visually separated, not sticking together. Emoticons anyone?

I think initially CHs would probably have three things in mind 1) Nope 2) Don’t know 3) Feeling it (can change with time and with new entries added). So maybe the rating UI should look like it stands for these three things.

If after that an overwhelmed overworked CH ends up giving swaths of 1-Stars nobody gets hurt. Maybe we will see less of unintended ratings.

Here’s something different. Just maybe, SH could have a First Wave where points aren’t affected, since it’s hardly fair to require the CH to have to make a decision the first time they see a name, much less hundreds of names in one go (busy CHs, no time!), it taxes the mind. We all know a fresh mind is more rational.

As a bonus contestants get to know the CH had seen their entries.

How about the First Wave be just a simple Keep or Hide, and Second Wave will be the actual rating. This route could end all the negative points though. In fact, I’ve seen some CH who use the Hide feature, it’s good because nobody loses points and CHs can get away from a mountain of entries, as well as eliminate bad entries repeated by other contestants, maybe SH should actively encourage that use.

EDIT:
If SH should implement a system that do not ‘often’ cause Contestants to lose points, then +40 and +50 might have inflationary consequences and points could become worthless. I’ve seen a few contests that CH give out many 4-5 Stars. In that case, the points should be adjusted.

4 Likes

Good ideas, Front. I agree that CHs can easily become overwhelmed.

1 Like

I agree with @auntshommy. Many creatives work empirically. Restriction is the enemy of creativity. Creativity needs flow and lots of lateral thinking space. It’s better to come up with a way that won’t punish either Contestants or CHs.

3 Likes

Exactly, @Front.

Another great example: In one contest right now I have two 2s and one 5. Am a slacker namer or high-value contestant?

3 Likes

@Stalias don’t think you need to worry the cream will rise to the top!, JUST MY OPINION if everyone thinks about the new system it really makes sense! For those that want to try a variety to gauge the CH can surely do it in 5 submits or less, and keep the other 35 in the bank until the ratings hit and even at 5 1stars that’s only a net lose of 25 points 2 pts for submit + 40 for entering the contest alone leaving -25 points. ( with a good detailed prebrief for the CH including SH rating guidelines ) if you get a one then you clearly didnt followed the brief! So in most cases if the CH doesn’t like them 2 stars all you would end up with is a net zero no harm no foul,

Lastly I’m sure a provision could be made on a contest basis where the CH is a 1 star happy CH , or we would KINDLY ( lol ) let him know rating all 1’s when we are following the brief isn’t cool

I just find it hindering the entire creative process we are here to do. It’s like telling Michelangelo to only paint 8% of a chapel ceiling and wait to see if enough “likes” are there. It’s like telling Country artists to put half a heart into a new country song and wait to see if JayZ gives it his thumbs up or not.

And I don’t see how punishing people when they are already down does anybody any good…(i.e. On the other hand, the bottom 25 contestants (who lost most points in a month) will lose additional points)). Is this really necessary? Seems like hitting a dog for pooping in the house 30 days later. I can understand “awarding” people for a job well done, but kicking them when they are down, I disagree.

6 Likes

Agreed.

@Dan So now, we will get no points for just submitting entries? And, every contestant from new to veteran will get a flat 40 points to join? I guess this could offset the -10, -15 a bit.

EDIT:
Some time ago, I wondered how some new contestants could submit as many entries, if not more, than a veteran contestant. Could they have gamed the system by submitting and immediately withdrawing repeatedly to jack up their points. This was an opened contest, so I could do a count.

I agree with @jackieheraty the bottom 25 getting hit harder like kicking a dog that’s already down ((((((

2 Likes

We appreciate everyone’s feedback. Clearly there are different point of views, and one system will not address all scenarios. We have worked on revising the point system based upon several months of data and taking into account different perspectives of all constituents. We have seen several contestants who have misused the entry submission by submitting random, unthoughtful entries, without having any implications. In fact, the previous point system rewarded them for submitting even more entries.

We actually think it is unfair to those contestants who are putting in their hard work, to be put in the same bucket with other contestants who are not actually making any effort before submitting their entries. It not only results in good quality entries getting lost in the crowd, it also overwhelms the contest holder who sometimes give up the contest because they have too many low quality entries to sort through, before they find good entries. We have also heard this feedback directly from several contest holders.

There are some really creative people in our community and our platform must reward them by giving them privileged status and additional points. We have increased the points for 4 star and 5 star entries for that very reason.

On the other hand, if someone is submitting lots of entries that are consistently receiving 1 star or 2 star ratings, we must flag those people to take corrective action or limit their benefits. We are only considering bottom 25 (out of thousands of contestants) who have lost the most number of points. If we let this play out for few months, we really think that this list will consist of those people who are actually not playing by the rules. and if despite our warning, they continue to lose the most points again the next month, we really think the implications should be more severe in that case. There’s a trade off between quality and quantity, and there is no doubt that we want to stand for a quality platform which recognizes the hard work of our contestants.

While we certainly appreciate that certain contest holders do not use a consistent mechanism to rate entries (we are working on other suggestions to fix this). However, on an aggregated basis, we think this issue will correct itself. In the end, those who receive consistently high ratings across multiple contests would be the ones who show up in our leaderboard. Think of point system as a relative system, rather than an absolute system.

I suggest we let this play out for next few months. Our promise is that we will continue to monitor this and make adjustments to the system, if we believe it is not resulting in the outcomes that we expect it to deliver.

Thanks again for all your feedback, It just shows that you all care about collectively improving this platform, and we welcome any constructive feedback, even if it is contrary to our point of view.

1 Like

@Dan Good to know that this tweak is based on collected data and will be continually monitored.

1 Like

Ok, I’m not a big forum participator but I can see that this will affect me and I am definitely putting my 2 cents in, or maybe a buck.
First, I could care less about the point system. I find it hinders the creative process and takes the focus off of finding a good name to worrying about if you are going to score. I say do away with the points period.
Second, this business of punishing the lowest scored members is just flat wrong.
I don’t participate in every contest, I do have other life issues going on so I am here when I can . I take my names seriously when I enter them. I know I will have a low score at the end of each month because I will not be entering as much as others and to be punished for that is insane.
So, again, I say stop with the points, focus on the names and forget about punishing anyone.
Ty :v:

2 Likes

Silly slacker (the) superb Stalias! (sorry Squadhelp!)

1 Like

@LorinRyle I think it’s highly unlikely you will show up among the 25 lowest on the Leaderboard. It’s meant for those who actually ‘lost’ a lot of points, the troublemakers. SH said they do keep an eye on what’s going on, I think they know what they are doing, we should trust their judgement. I’m all for the Hall of Fame, i’m not so much for the Hall of Shame, but if it can work out deter the troublemakers then I’d like to wait and see what happens.

1 Like

I really appreciate how hard SH is working to improve the site. For those of us who are habitual “editors,” it will be interesting to see how this plays out. I have several wins which occurred after I was completely offtrack in a contest and had all low scores initially. “LifeWiseLegal” and even “LEDgendari” are 2 of them. To be honest, I don’t know if I would have hung in there during either of those contests under the revised points system. I’d probably bow out like I do on CS if things aren’t going well. Maybe not. We’ll see.

2 Likes

@LorinRyle, thanks for sharing your feedback. Just to be clear, if you don’t participate often, doesn’t mean that you will be among the lowest 25 members. What we are referring to is those users who receive hundreds of 1 or 2 star ratings in a month and no 4-5 star ratings at all, resulting in a significantly negative point balance ! If you participate less or don’t participate at all, you will not be able to achieve such a negative point balance.

Also just to be clear, we do not intend to publish the list of bottom 25 users. Leaderboard is only intended for top 25 users who achieved the most points in a month. For the bottom 25 users who have the highest negative point balance, we are simply going to setup a warning system that results in corrective action from their side. We have even backtested the algorithm of new point system with last several months of data to validate that only those users who have truly misused the system end up in that “warning” list.

4 Likes