Percentile Score Changes?


#1

Hey Everyone. I’m still pretty new to SH, and am trying to figure things out. One question for all of you is how often does your percentile score change? A few times a day? Once a day? I’ve been having issues with my score since I joined on Oct. 31. I didn’t even have a score at all until early in January (very frustrating). Then when I finally got a score (83%) it dropped like a rock over 2 weeks from 83% to 52% (Even after winning 5 contests)…Dropping me from Tier A totally. I was told it’s because of a few blocks from CH, but I think there’s more to it because my score has been stuck at EXACTLY 52.1% for 3 days!!! And this was after 2-3 love its and many like its. I’ve been having a percentile issue from the beginning and insist that my account is “broken” somehow. But because I’ve been having all these technical issues I feel like I’m annoying the SH staff. HELP!!! I feel so stuck and frustrated and I don’t want to be a pain for them, I just truly think all of this is NOT NORMAL. So, again, the question is how often does you percentile score change? Daily? Multiple times a day? Feeling stuck, Marvon


#2

Mine changes once every 24 hrs. To my understanding the ratings are based on the last 6 months. A few things to keep in mind are how your ratings stack up against other ratings in a given contest. Go back to those contests that you got the love its in and see how many love its were given in respect to all ratings given in that contest. If there were a ton of love its given then it’s not going to change your percentage much, if any at all. But it works both ways as well. If you get a NTY when the majority of contestants are getting better ratings than that, that’s going to hit your percentage hard. A love in a sea of NTYs is great for your percentage!! Another thing is the 6 month history. Keep in mind that everyday that 6 month history loses and gains a day of good or bad ratings. So maybe the day it loses was a love in a sea of NTYs and the day added is a NTY in a sea of loves. That’s prob gonna hurt! Also, the percentage is based on CH ratings, I don’t think it includes wins.

This may not account for all the trouble you’ve been having but hopefully it helps a little. Good luck with everything!! :slightly_smiling_face:


#3

Thanks for sharing Chasity! Your message only confirms that I clearly DO have an issue with my account. Here’s why: 1. I’ve only had a percentile score for 3 weeks (not even close to 6 months). 2. I was “loved” at least 4 times in contests where I was one of 12 or less “Loved” and in one contest I was 1 of 2 loved. I was “liked” by at least 8 more. 3. The reason I mostly feel something is wrong with my account is that my score has been EXACTLY 52.1 for 4 days straight (again, after being shortlisted and liked, etc). I could even understand if it went from 52.1% to 52.3% or 51.9% - but to have NO movement at all?? 4. I’ve won 5 contests since Jan. 1 and my percentile has not gone up at all…instead it dropped over 30%.


#4

As a reminder, your percentile score works based upon the percentage of high ratings you receive compared to the percentage of high ratings your peers receive.


#5

Hi @Marvon7,

The team is working with you directly regarding your concerns. In these conversations, we’ve looked into your score and provided a few specifics related to your account.

In general, it is important to note that newer accounts can see larger shifts in percentile rating. Additionally, please remember that your score is based on factors beyond ratings and wins.

An important ranking factor is how many times you are blocked from a contest in relation to other Creatives. If you have a high number of blocks over a short period of time, a large drop in percentile ranking can occur. This policy is put in place to protect the overall integrity of the platform.

As long as you continue to follow the best practices outlined in the below article, you should expect to see an improvement in your percentile score over time:


#6

Thanks Grant! I want to THANK YOU publicly for looking into my account anomalies more. I have read the “best practices” many times and am following it to the letter. However, my percentile score has been stuck at EXACTLY 52.1% for 5 days now. Not 51.9% or 52.3%. If, as you say, newer accounts see “larger shift” mine should not be stuck at the same place for 5 straight days. Wouldn’t you agree? I sent you an email earlier today and hope we can chat via phone soon. Again, THANK YOU Grant and team with your patience. As you know my account has been “flukey” (is that a word?) from the beginning and I really don’t want to frustrate you guys, but something is definitely abnormal here. Let’s chat soon may friend. Thanks again!! - Marvon


#7

Hi LisaMac. Thanks for the message. How often does you score move? Here’s the issue, Mine has been EXACTLY 52.1% for 5 days now. Not 51.9% or 52.3%. It’s not moving AT ALL. My account has been having issues with percentile score for day one. It didn’t work at all for my account till after 2 months. I trust Grant and the team will look into it (I hope). - Marvon


#8

As discussed previously, the algorithm is working correctly. Once a score drops due to consistently being blocked from contests, it can take a bit of time for it to move again.


#9

Hello!

Sorry to bring this thread up again but I feel like although the algorithm may be working correctly, it seems to me that the algorithm itself needs tweaking/improving. As I understand it, the percentile takes a lot of factors into consideration, but a big part of it must be the comparison with other creatives.
I have been on the top 25 of “LEADERBOARD (MOST HIGH RATINGS)” consistently. I have also won 25 contests in the last year and have been winning a couple of contests every month, in the last few months. I frequently receive those bell messages saying I “You Rock” or stuff like that, because of all the high ratings (daily and weekly) and yet, my percentile is 55%… It doesn’t seem coherent to me to be in the top 25 of ratings (!) and at the same time my percentile only rates better than 55% of the other creatives…


#10

This kind of drop is only possible if you are receiving a significant number of blocks from Contest Holders. Blocks play an important role in our algorithm. Even consistent high ratings will not keep blocks from reducing a Creative’s overall score. The score will go up over time, as you continue to get good feedback from Contest Holder, while reducing the percentage of blocks received. We will continue to evaluate our algorithm.


#11

Hello @grant!

Thanks for the feedback!
If contest blocks are the “only possible” explanation for this, then my theory that the algorithm needs tweaking/improving must be correct. In fact, only today (AFTER the percentile was already at 55%) I got my first Contest block (that I know of - this would be a good improvement also: to be notified if you get blocked from a contest)!
So I really appreciate your willingness to further evaluate and improve the percentile algorithm. It’s much needed, in my oppinion.


#12

Hey @DNFront - I don’t know how often you participate in contests but I did want to tell you, in case it applies to you, that working on contests every day, or at least every other day, improves your score. Maybe that doesn’t apply to you but your case does sound highly unusual to me… so I thought I would pipe in. Hope it helps you.


#13

Hello @Commulinks!

Thanks for chipping in! :smiley:
As a matter of a fact I participate everyday, in all contests I’m able to (taglines and company names). Usually I enter 5 suggestions for the company names contests and 3 for the taglines and, if I get a like, I suggest 5/3 more at each type of contest.


#14

Then I totally believe there is something wrong with your score situation! Hang in there! I hope SH figures it out because I am totally convinced there is something wrong with your score calc.


#15

We have looked into the situation. There are quite a few blocks tracked on your account. You can contact the customer support team directly if you’d like to learn more about the specific numbers.


#16

Hello @grant!

Thank you for this feedback!
I had no idea about having several blocks in past contests. But if this is so, then in fact it must make sense to have such a low percentile.
I make the sggestion that creatives should receive a bell notification everytime they receive a block.

Best regards!


#17

I was wondering if creatives are aware of receiving a block? If not is there a place to look? If you are blocked and submit another name is there a notification then? Thanks


#18

Shanna, the only time I’ve ever known I was blocked is when I tried to put another entry into a contest that blocked me. If there have been other blocks I wouldn’t know it. I didn’t get a notification that I was blocked on that one until I tried.


#19

We will consider adding some alerts or tracking for blocks in the future.


#20

Sorry to bring it back to life again. :smile:
It seems to me the negative weight a block has over the precentile is disproportional to the positive weight a contest win has.
When I win a contest I usually get a 2% to 3% increase over the next few days. However, when I get a block it looks like the percentile drops around 8% to 10% (these are all rough numbers).
I think I have had 16 blocks over the last year, and in the same period I’ve won 26 contests. My current percentile is around 53%, which, considering what I shared earlier (about consistently being on the top 25 of ratings and frequently getting bell messages about how great my ratings are), seems a bit harsh. :wink: