VCDQuality Forums (http://www.vcdhq.com/forum/index.php)
- Site News and Discussion (http://www.vcdhq.com/forum/forumdisplay.php?forumid=3)
-- About ratings (http://www.vcdhq.com/forum/showthread.php?threadid=27233)

About ratings

I sugest to enhance the rating system in some way.

Problem
In the current system it is not considered what type of codec is used and what is the frame rate and resolution (just for example). So very often ratings do not show the real quality of the video and audio. Esspecially when movie is not very popular.

There could be a maximum rating limit defined for the movie according to the mentioned criterias. Alternatively there could be defined some quality ratios depending on the mention criterias (in my opinion ratios are better).

How it works?
For example, if the movie encoded as XVID or DIVX codec in 2 CDs. Release has 640x352 resolution and 25 fs frame rate plus >200 kbps data rate. Probably the best quality for usual movie release. So for such release maximum rate limit could be ten (for video rating points) or ratio = 1.
Using the maximum limits voter could choose from 1 to 10.
Using the ratios voter choice is multiplied with 1.

If release is on one CD encoded in WMV, has 352x288 resolution, has 23 fs frame rate and <200 kbps data rate. At most average quality. So for such release maximum rate limit could be 8 (for video rating points) or ratio = 0.8.
Using the maximum limits voter could choose from 1 to 8.
Using the ratios voter choice is multiplied with 0.8.

What is the real problem in implementing such system?
Is how to get data about currently collected releases in your database.

How to define ratios automatically if you have all the data?
If you have all the data then it is very easy.
Let's consider the video framerate.
The question that should be answered is: How to define what framerate could have the ratio = 1 (or the maximum rating limit = 10).
You cannot simply assign 10 to the highest framerate. Because it's not true. You can have only one movie that have very high framerate (28 fps) and the rest 99% of releases with a 25fps (for example) framerate. If you assign 28fps - ratio=1 then all other movies will never get ratio=1 or max rating = 10. Criteria's highest ratios should be calculated according to popularity. The maximum percentage of movies with the same framerate during some period of time (last 3 months) should have the ratio = 1.
Criteria's lowest ratios should be equal to the lowest framerate. in such way the ratio scale should be defined.
From lowest registered ----------------- To most popular
Lowest and highest ratios should be taken from defined period of time.

For example if during last 3 months maximum percentage of movies had 25fps framerate then ratio of 25fps = 1. Movies with higher fps will have the same ratio = 1. During the 3 months one registered movie had the lowest 20 fps. This movie will have ratio=0.5 (which is the lowest). Movie with 23fps will have
FPS ratio percentage = (moviefps-lowestfps)/(maxfps-lowestfps) = 23-20/25-20=3/5=0.6
FPS ratio = FPS ratio percentage*(highest ratio-lowest ratio)+lowest ratio=0.6*(1-0.5)+0.5=0.3+0.5=0.8
So the FPS ratio for this movie is 0.8. When voter gives 10 points to for release with 23 fps the vote is corrected and multiplied by 0.8 so maximum rate that voter can give to this movies is 8 (in case of one criteria).

We just mapped fps scale (from 20 to 25 fps) to ratios scale (from 0.5 to 1) and then mapped the movie fps to ratio.
We calculated the ration only for on criteria FPS. There can be other criterias. All of the ratios for all of the criterias have to be multiplied.

In the end movie can have such
video quality ratio table (sample):
resolution: 0.85
data rate: 0.97
frame rate: 0.9
number of CDs: 1
overall video ratio = 0.74

audio quality ratio table (sample):
codec: 1
data rate: 0.9
overall audio ratio = 0.9

I should notice that with such system ratios for most popular qualities will be 1.

That's probably the craziest suggestion that I wrote. And probably there are to much technical details in it. Forgive me, i'm mathematician, but anyway...
what do you think?

Overkill

Yeah, I think everyone around here would love some measure of automation in the rating system, but this does seem like overkill. Mostly because "real" release groups tend to follow the scene rules pretty closely -- nobody releases in WMV format, for example. And most releases that satisfy those rules look pretty much okay. Framerate often matters less than, for example, the number of encoding passes. I've certainly seen (A) a 2-pass 2300kbps release that looked pretty crappy compared to (B) a 4-pass @ 1800kbps.

You suggested that we could use multiple criteria in the same way and multiply the results, but then the question becomes, will (B) always be better than (A)? Maybe not for certain types of movies. In the end, any "fair" system would probably (as you admitted) end up giving just about every rules-adherent release a pretty similar "ratio". Or, a worse outcome might be that people would lose interest in fine-tuning the system to make it fair and end up consistently underrating certain formats that don't deserve it ...

__________________


Scrobble all you want, we'll make more!

Impressive stuff. Alas, as already stated, a little too complex.

Still, wow. You really put alot of effort into making sure a rating from 1 to 10 is accurate.

ummmm how about just grabbing the sample, and after reviewing the ratings a release got here, watch it and judge for yourself what you consider the "real" rating to be. or am i too much of a simpleton?

To horizon:

"will (B) always be better than (A)?"
No, of course not. The technical aspects of encoding do not show the quality of movie source. Although encoding can be done perfectly, the source of the movie could be very bad (theoretically). If criterias are considered carefully, probability of such thing is not so big. Anyway, movie viewers can always give the lowest possible rating.
With described system it's always possible to give lowest possible rating, but the highest is not always allowed.
Proposed system is just a theory. It can be developed further or can be simplified. However, you made a good point.

TO sp0ng3b0b:
"how about just grabbing the sample"
Yes, that's true. But, if i (or we, vcdquality users) could simply rely on samples, then ratings are not needed at all. Rating system is not needed.
I was talking about improvement of rating system, not about it's usage.

Thanks for comments.

you forgot to carry the one

__________________
563-773-1880

<Dwaggy> i caught you a delicious Bass
<The404> This is because you are an idiot