2013 Hostelworld Conference and HOSCARS (Dublin)

According to my Buddy, the algorithm they use for the awards also take into account the ratio of reviews to beds that you have. If a smaller hostel has a good rating with tons of reviews, and a larger hostel has an even better rating but with less reviews, then the smaller hostel will be given higher preference for the award. But that's just one aspect of the algorithm.

She also assured me that HW does not run the calculations and then make adjustments at the end if they don't particularly like the hostel that comes out as the winner. I'm not entirely convinced of this. As someone commented to me at the conference, "Everyone plays the game all year, but we don't know what the rules are until the very end." And after the winners are announced and the awards are handed out, we still don't know the rules. This gives HW the freedom to make all the adjustments they like.

As Gordo said, it would be wonderful if HW would make the method for determining the winners clear at the beginning of every year. So call your Buddy and ask for this!

High review scores that average over 90% indicate happy customers. Happy customers that are inclined to stay longer... The 98% hostel listed above is always full, it is doing an amazing job in a backpacker mecca. This hostel gives the vast majority of their allocation to HW. Obviously these happy customers are not extending their stay by re-booking through HW. This reduces their chance of being recognized by HW for being the amazing hostel that they are. They still had hundreds of reviews for the year but obviously didn't make HW enough money. The hostel won almost every monthly award given in 2012 for their area. The hostel that won the top honor did not even win one monthly award in 2012. Hostels work very hard to improve their product and the review system recognition is a great way to improve. Many hostels also use the awards and review system for staff bonuses/incentives. The worst part is no hostels were informed of this prior to the award ceremony. I know of a few hostels that attended the awards expecting some sort of recognition even to win their category and were shocked and demoralized to not even be recognized in the top ten for their category, after they won the majority of the monthly awards for 2012. I know that these hostels read the forums so I will let them tell their stories if they choose to. Our hostel won the Hoscar for top hostel in North America and Canada in 2009. The recognition was great then, but now it feels just a bit hollow.

So maybe they should also publish the % of reviews written for the number of bookings? That would be very interesting and helpful information.

I have been in communication with Hostelworld as well. We were # 4 best "extra large" hostel in 2012...and this year #7 - even though we were rated higher than 4 of the better rated properties. We had 466 reviews in 2012 averaged around 91.7% for the year. I got this response from Hostelworld explaining the new algorithm, but it also says that it needs further tweaking.

Quote from e-mail:

With regard to the Hostel awards we made a few changes this year to try make the system fairer for all. I believe it is fairer but think there still might be things we can improve next year.

This year the algorithm gives a different weighting to each of: Average rating, bookings per bed ( capacity), reviews per bed ( capacity). Each hostel is then given an overall score. The hostel with the highest score wins in each category. Previously, we used to pick a minimum number of reviews for each category and we then picked the highest rated hostel once they had met this minimum. Feedback on this was it excluded hostels that may have exceeded the min number of reviews but had a slightly lower rating.

For example:

Min number of reviews is 400
Hostel A has a rating of 84% and has 450 reviews
Hostel B has an 83% rating but has 800 reviews.

According to the old system Hostel A would be the winner but hostels complained that the hostel maintaining the 83% from 800 reviews should be the winner.

So this year we wanted to combine the ratings and numbers of reviews that each hostel receives against their capacity. Bookings per bed also looks at how popular each hostel is with customers. I think this gives small and large hostels a fair chance but I do believe that next year we should ensure that there isn’t so much variance in the overall rating. Does that make sense?

I agree with others about the way they go about informing properties of their methodology. I believe that set criteria should be given to all hostels and that everyone is working towards the same set of rules. Finding out criteria and secretive algorithms after the fact simply make you wonder if you're getting the real story.

I hope that Hostelworld will be more transparent with the Hoscars. We all want to receive awards, but it can feel like a smack in the face if you've worked hard, your staff worked hard and then surprisingly the rules were changed - and you didn't know.