1. Due to the increased amount of spam bots on the forum, we are strengthening our defenses. You may experience a CAPTCHA challenge from time to time.
    Dismiss Notice
  2. Notification emails are working properly again. Please check your email spam folder and if you see any emails from the Cantina there, make sure to mark them as "Not Spam". This will help a lot to whitelist the emails and to stop them going to spam.
    Dismiss Notice
  3. IMPORTANT! To be able to create new threads and rate posts, you need to have at least 30 posts in The Cantina.
    Dismiss Notice
  4. Before posting a new thread, check the list with similar threads that will appear when you start typing the thread's title.
    Dismiss Notice

RT Audience Score Rigged Again

Discussion in 'Solo' started by DailyPlunge, May 20, 2018.

  1. RoyleRancor

    RoyleRancor Car'a'Carn

    Joined:
    Dec 9, 2016
    Posts:
    5,793
    Likes Received:
    34,671
    Trophy Points:
    159,917
    Credits:
    25,780
    Ratings:
    +43,325 / 185 / -97

    Oh please.
    The "canon" of the EU was a joke. It's almost an insult to the Fox X-Men universe to call what the EU was a "canon"
    It was a mishmash construct of madness linked together by similar terms and names only.
     
    • Like Like x 6
    • Disagree Disagree x 2
    • Wise Wise x 1
    • Funny Funny x 1
  2. Jayson

    Jayson Resident Lucasian

    Joined:
    Dec 24, 2015
    Posts:
    2,160
    Likes Received:
    6,601
    Trophy Points:
    16,467
    Credits:
    8,697
    Ratings:
    +9,540 / 39 / -14
    I find this rather interesting:
    https://medium.freecodecamp.org/who...-tomatoes-metacritic-or-fandango-7d1010c6cf19

    I strongly encourage folks to read this really well done data analysis study done by Alexandru Olteanu, but I'll run a summary of it here.

    Alexandru studied 214 film ratings across multiple rating services, and he also ran data checks on that against a much larger volume for validity in the data modeling.

    Basically, the microwave dinner version of the result of the study is the following

    The possible ways data can be shaped looks like this:
    [​IMG]


    The ideal result of a bunch of ratings spread out over a bunch of movies is this:
    [​IMG]


    And the results of the study found that the ratings data for the studied ratings sites looks like this:

    [​IMG]

    Which results in a conclusion of:

    From the site:
    ---------------------------------------------------------------------------------------------------------------------------------------

    The verdict: use Metacritic’s metascore
    All in all, I recommend checking the metascore whenever you are looking for a movie rating. Here’s how it works, and its downsides.

    In a nutshell, the metascore is a weighted average of many reviews coming from reputed critics. The Metacritic team reads the reviews and assigns each a 0–100 score, which is then given a weight, mainly based on the review’s quality and source. You can find more about their rating system here.

    Now, I just want to point out a few downsides of the metascore:

    • The weighting coefficients are confidential, so you won’t get to see the extent to which each review counted in the metascore.
    • You’ll have a rough time finding metascores for less-known movies that appeared before 1999, the year Metacritic was created.
    • Some recent movies whose main language is not English aren’t even listed on Metacritic. For example, the Romanian movies Two Lottery Tickets (2016) and Eastern Business (2016) are not listed on Metacritic, while they are on IMDB, with ratings.
    ---------------------------------------------------------------------------------------------------------------------------------------

    This is similarly echoed, with a few differences, by fivethirtyeight.com when they did their similar study...basically Metacritic still has the most bell-like curve of the set.

    https://fivethirtyeight.com/features/fandango-movies-ratings/

    [​IMG]

    Cheers,
    Jayson
     
    • Informative Informative x 5
    • Like Like x 2
  3. ZDTemplar

    ZDTemplar Rebel Trooper

    Joined:
    May 5, 2018
    Posts:
    123
    Likes Received:
    140
    Trophy Points:
    197
    Credits:
    501
    Ratings:
    +223 / 13 / -0
    Oh my, those were the good old days. As a kid, my friends and I simply decided on a movie that looked cool based on the trailer. We had absolutely no inkling what others thought of it.

    Looking on RT, quite a few of the old movies my friends and I loved have middling scores, or even rotten. Still very fun movies.

    I have to ask myself, would I have seen the original "Saw" movie if I looked at what critics thought beforehand, giving it a 49%? I honestly stopped caring what critics think about thing years ago. they don't represent me, they are people paid to watch all the movies, and that can often leave them jaded or overly-critical about things, hating something just because it's something they've seen too many times before, or reacting too positively to things just because it defied their expectations.

    Why are people these days so obsessed with trying to validate their opinions on things by appealing to critics, or popularity?
     
    • Like Like x 4
    • Wise Wise x 1
    • Cool Cool x 1
  4. Sargon

    Sargon Rebelscum

    Joined:
    Apr 8, 2018
    Posts:
    131
    Likes Received:
    238
    Trophy Points:
    607
    Credits:
    601
    Ratings:
    +364 / 12 / -0
    @Jayson that's a terrific article! Thanks for sharing it.

    My only problem with it is as it relates to RT. Again, RT is being misunderstood what the Tomatometer actually is. 82% Fresh doesn't mean 8.2/10 . Its not an average, so it is completely invalid to have it in this comparison, he mistook what it was a measurement of, that's why the distribution is all over the place. Tomatometer is already itself a measurement of binary yes/no recommendations without regard for average ratings. RT does release averages, but they are not the tomatometer--and they are also almost identical to Metacritic, which is also a measurement of averages.

    I hate to have criticism because it's a terrific piece otherwise. But time and time again I feel like I have to correct people with regards to what RT is actually measuring. Since the site became so popular in the mainstream the last few years people don't stop to actually think about how it functions, they just see that big, bold percentage number and think that's an average.
     
    #84 Sargon, May 23, 2018
    Last edited: May 23, 2018
    • Informative Informative x 3
    • Like Like x 1
    • Friendly Friendly x 1
  5. Jack_Forest

    Jack_Forest Force Attuned

    Joined:
    Feb 12, 2018
    Posts:
    907
    Likes Received:
    5,678
    Trophy Points:
    15,617
    Credits:
    4,543
    Ratings:
    +6,600 / 30 / -16
    Because now that everyone can have their loud opinions heard many people feel insecure? Or rather, people who are already insecure now have to somehow deal with that every time they look through a comment section?
     
    • Like Like x 2
  6. Jayson

    Jayson Resident Lucasian

    Joined:
    Dec 24, 2015
    Posts:
    2,160
    Likes Received:
    6,601
    Trophy Points:
    16,467
    Credits:
    8,697
    Ratings:
    +9,540 / 39 / -14
    Man, there's so much to say on this, but I don't have a lot of time to at the moment (literally pulled over in my car, whipped out my laptop, flipped on my mobile hotspot to just chime in because I have so many thoughts here). But I'm going to go for the bang for the buck and just cut to the chase.

    Don't get hung up on his labels, as that doesn't really matter.
    What matters is the plotting of the score distributions.
    It doesn't really matter what the Tomatometer is representing; it's fine that it represents a binary proposition embedded into and behind the percent. Actually, most consumer service surveys are made this way.

    What matters is that the scores are not well distributed, but instead are polarized to both low and high percent volume ratings.

    That means that your representation of the data you are providing is poorly formed. This kind of representation of a binary survey proposition is exactly what gets bonked for as manipulation in the political survey circles, because it essentially plops the results into more of a "two camp at odds" narrative that isn't actually accurate.

    Regardless of what the Tomatometer represents, you shouldn't see a pile up at the low end of the percent of critics and a pile up at the high end of the percent of critics.
    That means you either have a problem with your data input method, your data calculation, or your data representation (or all of these).

    He doesn't use these same words, but he lightly does address this in the comments section in his own way, and I generally agree with him.
    The onus is on RT to ensure that their data is well distributed in every representation of the KPI; not just on the raw values behind them.
    That is pretty useless since no one is going to see that information.

    It's actually more of a problem that it is not well distributed for RT than it is for some of the others because of this property of their data representation, as when you do this KPI modeling, the KPI model distribution should inherently and properly reflect the total distribution pattern with relatively good correlation; not split off and deviate from the averages distribution. That means your KPI has some issues because it is incapable of representing a broad range of values and biases towards one range of bins or another.

    So ignore his labeling, because his language of labels doesn't change the hard data itself (as you can see, the same kind of behavior exists in fivethirtyeight's version as well - it's just how their data presents, and it's not actually good).

    The idea is sound, but the management of weights in the distribution is not.

    Cheers,
    Jayson
     
    • Like Like x 2
  7. Pomojema

    Pomojema Ayatollah Of Rock-&-Rolla
    Staff Member

    Joined:
    Jul 23, 2014
    Posts:
    10,177
    Likes Received:
    42,740
    Trophy Points:
    171,329
    Credits:
    44,734
    Ratings:
    +53,035 / 127 / -63
    Reviews like this are why I'm almost tempted to start an official "Star Wars Cringe Thread" right here on The Cantina.

    It has everything. "Rian Johnson ruined muh childhood" drivel. Pointing fingers at Kathleen Kennedy and Disney. The insane conspiracy that critics are somehow afraid and/or paid off to say that they don't like a movie... Even though critics that aren't huge fans of Solo have made it clear that they didn't like the movie despite the fact that some of them liked TLJ. Along with hating a movie that they didn't even watch that Rian Johnson had nothing to do with.

    But most of all, it has the notion that not liking a sci-fi movie for twelve-year-olds is a cause for TRUTH and JUSTICE and OTHER FEEL-GOOD ABSTRACT CONSTRUCTS. And that's the cringiest thing I've seen out of the troll groups to come out of this age of Star Wars content.
    I think they said something to the effect of being unsure that they were going to have Thanos be the villain behind Loki in The Avengers until they had already started filming the movie. Marvel Studios has done a lot more improvisation than people actually realize.
     
    #87 Pomojema, May 23, 2018
    Last edited: May 23, 2018
    • Like Like x 2
    • Wise Wise x 2
    • Great Post Great Post x 1
  8. Jayson

    Jayson Resident Lucasian

    Joined:
    Dec 24, 2015
    Posts:
    2,160
    Likes Received:
    6,601
    Trophy Points:
    16,467
    Credits:
    8,697
    Ratings:
    +9,540 / 39 / -14
    @Sargon

    So just to make it more acceptable, I downloaded his raw data (because he provides it), and ran the values straight raw without lumping any bins or anything.
    Just raw data, and each value axis is only representative of the rating system used by the service and a count of how many of each value there are.

    Now I also provided a radio chart version embedded for each of the data sets. The purpose here is that it's an easy asthetical way to see what is well distributed and what isn't, because a well distributed data set should be polarized to the 12 o'clock position and fanned out from there mostly evenly on both sides.

    I also added the RT Audience ratings (which he had in his raw data, but didn't add to his article).
    [​IMG]

    The RT Audience ratings are the third most well distributed, but are lop-sided, and even with being lop-sided, they have an acute spike in the lower values that is a bit odd in character.

    The Tomatometer remains a near plateau distribution, which isn't ideal.
    Fandago is pretty much useless.

    IMDB does well, but has a bias to higher values, so you'd have to keep that in mind.

    Still, Metacritic has the most well distributed data set of rating values. You have relatively the same probability of running into low values as you do high values, and a higher probability of seeing many values in the middle ranges.

    I hope this helps. :)

    Cheers,
    Jayson
     
    #88 Jayson, May 23, 2018
    Last edited: May 23, 2018
    • Informative Informative x 5
    • Like Like x 1
  9. Beauke

    Beauke Rebel Commander

    Joined:
    Feb 16, 2018
    Posts:
    185
    Likes Received:
    463
    Trophy Points:
    2,757
    Credits:
    830
    Ratings:
    +652 / 3 / -4
    RT is a great tool, but only for the critics part of the site.

    The Audience score on RT, has always been garbage, for the fact that you can use it to your means. Matt Atchity, the former editor in chief of RT, said in one of the "Young Turks" shows on youtube that he never looked at the audience score on RT, because he had seen how directors of bad reviewed movies asked their fans on social media to upvote the audience score, ...
     
    • Like Like x 3
  10. cawatrooper

    cawatrooper Dungeon Master

    Joined:
    Nov 14, 2016
    Posts:
    4,818
    Likes Received:
    21,980
    Trophy Points:
    149,167
    Credits:
    19,951
    Ratings:
    +26,705 / 65 / -37
    Sure, I'm not saying that the subjectivity of a reviewer makes their reviews worthless- because that pretty much eliminates the entire industry, at that point.

    I'm just saying that the inherent mean-ness that the anonymity of the internet causes some real issues with this. You see it all the time on YouTube videos. For instance, Justin Bieber's song "Baby" was the most disliked video on YouTube for a really long time.

    Is it the worst song in the world? I mean, it's bad, but I wouldn't say the worst, not by the leagues this stat suggests.

    Is it the worst video in the world? Hardly.

    So honestly, what value can we denote to YouTube's "like" system other than "This is how this video's viewers who also own accounts have reacted to it thus far- taking into account that almost certainly many viewers went in specifically to dislike it".

    Another example, this one on Rotten Tomatoes. A few years ago, Kirk Cameron made a film about Christmas. He urged his followers on social media to give it a positive rating to boost his rating on RT. When social media got wind of this, many went to give it as low of a rating as possible.

    Remember, Cameron wasn't saying "Watch my movie and rate it", and the internet mob wasn't saying "Watch Cameron's film and evaluate it". This was simply a game of tug of war, between two teams with no inherent prerequisite to actually watch the film they were all squabbling about.

    As of now, the score has evened out a little more, as you said. But at this point, I think Saving Christmas's run is over. For better or for worse for the film, Rotten Tomatoes had done its damage, and almost four years later the more tempered effect that it had on the film won't be changing nearly as many minds as it did during its frenzy in 2014.

    So yeah, even when the "dust settles" and the review is a little bit more reflective of a realistic view, we can't pretend as if Rotten Tomatoes is worth the trouble it causes.



    I guess in the end, accountability is important. That's why I put a lot more stake in the critics portion of the reviews. Not because they're snobby film folk, but because they're proudly putting attaching their reputation to their words.
     
    • Like Like x 4
  11. RoyleRancor

    RoyleRancor Car'a'Carn

    Joined:
    Dec 9, 2016
    Posts:
    5,793
    Likes Received:
    34,671
    Trophy Points:
    159,917
    Credits:
    25,780
    Ratings:
    +43,325 / 185 / -97
    Anonymity is wildly important but simultaneously the worst thing about the internet.
     
    • Wise Wise x 4
  12. cawatrooper

    cawatrooper Dungeon Master

    Joined:
    Nov 14, 2016
    Posts:
    4,818
    Likes Received:
    21,980
    Trophy Points:
    149,167
    Credits:
    19,951
    Ratings:
    +26,705 / 65 / -37
    It's important in some cases.

    I don't mind when, for instance, news websites require you to sign in with your FaceBook account to make comments. Comments can be the low point of humanity, so I think this really does help mitigate that. When you're talking times when massive amounts of money and influence are at stake (like a film in theaters) I think this type of thing would be especially useful.

    But now I guess I'm starting to sound a little authoritarian, like this guy...
    upload_2018-5-23_8-56-38.png
     
    • Funny Funny x 3
    • Like Like x 2
    • Great Post Great Post x 1
  13. RoyleRancor

    RoyleRancor Car'a'Carn

    Joined:
    Dec 9, 2016
    Posts:
    5,793
    Likes Received:
    34,671
    Trophy Points:
    159,917
    Credits:
    25,780
    Ratings:
    +43,325 / 185 / -97
    Comment sections should be abolished tbh.
    Take it to a forum.

    If YouTube forced Facebook linking....it'd be such a better place.

    Twitter is where the area gets the most grey.
    Because anonymity on there for people who are in sensitive places it's incredibly important. Journalists or activists especially.

    I enjoy my quasi-anonymity on here.
     
    • Like Like x 3
    • Great Post Great Post x 1
    • Clouded Clouded x 1
  14. BobaFettNY21

    BobaFettNY21 Force Attuned

    Joined:
    Jun 30, 2015
    Posts:
    795
    Likes Received:
    6,010
    Trophy Points:
    15,667
    Credits:
    8,146
    Ratings:
    +6,932 / 36 / -12

    Wait....you're not Jeff Bridges????

    [​IMG]
     
    • Funny Funny x 6
  15. Fernus

    Fernus Rebelscum

    Joined:
    Dec 21, 2017
    Posts:
    91
    Likes Received:
    253
    Trophy Points:
    662
    Credits:
    658
    Ratings:
    +306 / 17 / -10
    That's impossible. But he definitely made the future a lot worse for lots of passionate fans. Many of us speak from the heart, with genuine emotion. It's not right to dismiss that.

    Anonymity is the best thing about the internet. At least, when it was a thing. I still stand by it.
     
    • Like Like x 2
    • Old News Old News x 1
    • Unoriginal Unoriginal x 1
  16. cawatrooper

    cawatrooper Dungeon Master

    Joined:
    Nov 14, 2016
    Posts:
    4,818
    Likes Received:
    21,980
    Trophy Points:
    149,167
    Credits:
    19,951
    Ratings:
    +26,705 / 65 / -37

    That's fair. But unlike YouTube and RT, there's at least not a built in system for rating created content for people to abuse with their anonymity.
    Basically, you support accounts that you like and hopefully just ignore those you don't. It's not perfect, but it's better than the alternative on other platforms, I guess.

    The idea of a "dislike" button is so widely supported, but I hate the idea of promoting negativity. To be honest, I don't even love it on these forums.
     
    • Like Like x 4
    • Great Post Great Post x 1
  17. RoyleRancor

    RoyleRancor Car'a'Carn

    Joined:
    Dec 9, 2016
    Posts:
    5,793
    Likes Received:
    34,671
    Trophy Points:
    159,917
    Credits:
    25,780
    Ratings:
    +43,325 / 185 / -97
    I definitely agree about the dislike function, though I use it.

    But when I say anonymity on Twitter, I mean for people in places that have very strict social media laws or are embedded in much more risky situations than me, who tweets about football and movies.
    --- Double Post Merged, May 23, 2018, Original Post Date: May 23, 2018 ---
    It's not about dismissing people's emotions it's about dismissing the notion that a movie can ruin a childhood. It doesn't (or at least shouldn't) work like that. I've never understood it and never will be able to fully grasp the concept of it.
    You know what is more likely to ruin a childhood? Finding out that the guy who created something you loved was a raging rapist (Cosby), not a story of a fake character being different than what 12 year old you wanted. That is what makes it difficult to discuss the film in earnest IMO. If you don't like it, fine. You don't have to like it. But hyperbolic nonsense of it ruining childhoods or destroying EVERYTHING that came before it is ridiculous and not worthy of serious debate.

    The best thing about the internet is free discussion and trade of thoughts.
    Anonymity both helps and hurts that. If these weirdos threatening Rian Johnson or Kathleen Kennedy had to put their name on it and do it to the faces of them, chances are they'd ask for a Maclunkey autograph and picture.

    The best place to hide would be right under your nose...so maybe I am. Maybe I'm not.
     
    • Like Like x 3
    • Funny Funny x 1
  18. FN-3263827

    FN-3263827 First Order CPS
    1030th General **** (Mod)

    Joined:
    Jan 19, 2016
    Posts:
    10,000
    Likes Received:
    141,413
    Trophy Points:
    173,077
    Credits:
    68,954
    Ratings:
    +157,742 / 65 / -7
    as someone who mighta blown a wee fuse with the PT, i do understand the emotional tidal wave of this kind of disappointment.
    but eventually it recedes and you regain perspective.

    no of course nothing can ruin what was good about your childhood ~ unless you let it.

    it does make discourse a bit difficult, but ultimately it's not a wall to break through, it's a storm to weather.

    we could all be more patient and remember that we only disagree so vehemently because we do love Star Wars.
    or at least aspects of it. XD
     
    • Like Like x 5
    • Wise Wise x 2
  19. cawatrooper

    cawatrooper Dungeon Master

    Joined:
    Nov 14, 2016
    Posts:
    4,818
    Likes Received:
    21,980
    Trophy Points:
    149,167
    Credits:
    19,951
    Ratings:
    +26,705 / 65 / -37
    That's fair. I have a sister who lived in China for a while, and if she hadn't been smarter about her VPN usage she could've easily ended up in a sticky situation multiple times

    Yeah, I think it's really all about choice.

    I love Jackson's Lord of the Rings movies, for instance, same as Lucas's original Star Wars trilogy.

    Not as big of a fan of the prequel trilogies for each, though. Nevertheless, I find it's not hard at all to view the original trilogies as their own thing, and taking whatever good I can from the prequels.

    For example, An Unexpected Journey wasn't great... but in The Fellowship of the Ring we know that Bilbo had the ring at one time, so I can still enjoy the Riddles in the Dark scene as something that is representative of how Fellowship's Bilbo first encountered Gollum and found the ring.

    Same with Star Wars. It's easy enough to view Order 66 as representative of how the Jedi fell, while essentially seeing Jar Jar as rather inconsequential to the OT (Even if he was somewhat involved in the rise of the Empire).

    So, I can see how people want better movies, and that's always something to strive for. Again, I don't love the prequels, and I would've truly appreciated a less flawed trilogy of them. Heck, I have a laundry list of ways that the Sequel trilogy could be better, and I actually enjoy those films for the most part so far. But if something ruins an existing media for you, you're trying to hard too let it do so.
     
    #99 cawatrooper, May 23, 2018
    Last edited: May 23, 2018
    • Like Like x 5
  20. RoyleRancor

    RoyleRancor Car'a'Carn

    Joined:
    Dec 9, 2016
    Posts:
    5,793
    Likes Received:
    34,671
    Trophy Points:
    159,917
    Credits:
    25,780
    Ratings:
    +43,325 / 185 / -97
    My dad visits China for business often. He's so not tech savvy in regards to social media and I'm never more relieved than when he travels abroad lol.

    I think both you and @FN-3263827 are right but I still think what you are describing isn't the "ruin my childhood" level outrage we are seeing now.

    I know I didn't (and still don't) like the PT but it never ruined Star Wars because I still had the OT and the books and games etc.
    Just like now. There are still books and games. And the old EU books didn't evaporate because the nefarious Kathleen Kennedy snapped her fingers once she inherited the gauntlet from George. They're still there too.

    We should always want better movies or else you get the Transformers movies when you just accept big dumb spectacle with 0 effort in story.
    Which is different than say, Rampage. Which the focus is clearly on ridiculous action but they clearly were like "Okay we need some semblance of a story here. Just enough for the Rock to team with a giant gorilla but still..."
     
    • Like Like x 3
Loading...

Share This Page