What players on the Kronos private World of Warcraft server think of their experiences so far : A survey of a 100 quasi-randomly selected Horde players

Abstract

[Objectives]
To find out what the current Kronos I players think of their experiences on the server so far by collecting testimonies
[Methods]
A convenience 100 Horde player sample was identified with quasi-random /who searches on Kronos I, on the 5th of August 2016. They were then sent in-game questionnaires about their experience on Kronos I with the in-game post system from a level 1 character. On the 20th of August, 83 reminders were sent to players that hadn't yet completed the survey. Finally, on the 5th of September all answers were manually collected in a Word document. Names and personal informations were removed to maintain anonymity.
[Results]
18 players participated in the survey. 13 (72%) were from Europe and Russia, 3 (17%) were from the USA and the others didn't specify. 9 players had positive experiences (50%) and the others were all mixed (50%), none were negative-only. Emerging topics from the narrative answers were latency, the blizzlike vanilla feeling, bug abuse, world chat, the server population, PvP, the "refugee" status, missing the server information about planned downtime, the community and being new on the server.
[Conclusions]
Due to multiple factors (low answer-rate, Horde-only sample, one-time sample, etc.) the results cannot be considered to be representative of the experiences of players on Kronos. Nevertheless they can help give a better understanding of  Horde Europeans player's views of the server. This research emphasizes the need to carefuly create and test a questionnaire for response-rate before sending it to a large population and how reminders can fail when not made correctly.

Keywords : testimonials, peer information, survey, World of Warcraft, video games, Kronos, private servers

Introduction

A testimonial is a person's statement extolling the virtue of a product [9]. For testimonials to be effective, the audience must believe that they are given by real people and not fabricated by the business itself [9]. World of Warcraft private server managers and developers offer products in the form of services to players (access to the server, updates, feedback, server maintainance, etc.). In this environment, testimonials are an important part of understanding how a population perceives this service and can further influence the behaviour of potential future customers [9] or guide changes.

Also, some World of Warcraft private servers testimonials can stimulate a strong interest by other players, as shown by Dodgykebaab's Youtube server reviews with over 100,000 views [5]. Moreover, players very often ask for peer-advices to inform their server decision, as can be found on the WoW Private Server subreddit [6-8]. The extent of testimonials on World of Warcraft private server players actual behaviours is not well documented, but word of mouth is often said to be the main way players discover private servers.

While testimonials can be found on the Kronos server forums [14-16] and on subreddit threads [6-8], players writing them might not be representative of the players that currently play on the server, indeed a previous experiment of mine showed 39% of a 97 players sample hadn't or didn't visit the forums at all [13]. The amount of players actually contributing on the forums might be even lower. Also, reviews and testimonials tend to be spread out over many threads and forums, making reading them a tiring task. As in-game World of Warcraft messages allow players to rapidly communicate together and share personal information and testimonials, they represent a useful tool for a survey about personal experiences.

No permissions were asked as previous research was well accepted according to personal feedback I had received from players, readers and the Kronos team. Ethical approval wasn't deemed required.

This survey was based and informed on the MIT survey guidelines [4], the Center for Evidence Based Management (CEBMA) survey critical appraisal tool [10] and the Standards and Guidelines for Statistical Surveys [17].

Methods

Assuming an expected 40% response rate, just like in one of my previous experiment with in-game post messages on Kronos I in November 2015 [1] and looking for a 95% confidence level with a 10% confidence interval (CI) on an estimated 3000 players population on Kronos I (see [2]), I would need 93 players answering to correctly reflect my target population in terms of sample size, as calculated with the Survey System calculator [3]. This would mean sending the questionnaire to approximately 253 players (93/40*100), which I deem too time-consuming to do, leading to a non-representative 100 players sample. Statisticals tests have thus not been used and percentages were favoured.

On the 5th of August 2016, at 11:00 am CEST, a 100 Horde players names were collected manually (although using addons is greatly recommended for future research) by using the /who World of Warcraft command, which is currently not working as intended on Kronos and showing quasi-random results on subsequent calls. A 2-3 minute delay between command use was used as shorter intervals end up showing most of the same players again. Player names were then analyzed with Sporkforge's Word Counter & Text Analyzer [11] to remove duplicates and ensure all names were unique. An in-game message with two questions was then sent to each of them. Questions were developed by the author and adressed the continent of origin and the overall personal experience of the responder on the Kronos server. Continent of origin was deemed worthy of inclusion as Kronos is an international server [12] and players differ between countries of origin; it would also allow comparisons between continent's representation on the selected sample.

Greetings,

I am making a survey to better understand the experiences of the Kronos I players and I would appreciate your testimonial. All answers will be kept anonymous.

Where do you come from ? (EU/US/NA/Asia/Africa)
In a couple detailed sentences, how would you summarize your experiences on Kronos I ? (positive/negative/mixed/other)

Thank you for your time.

Hello,

I know you are doing your best in what matters to you and you might not have much time left for a survey and that’s fine. Assuming you do get a couple spare minutes, I would love to find out about your experience. Here is the anonymous survey again :

Where do you come from ? (EU/US/NA/Asia/Africa/RU)
In a couple sentences, how would you summarize your experiences on Kronos I ? (positive/negative/mixed/other)

Thank you again and have a great week!

All messages were manually sent once with the in-game post system, from a level 1 undead character named Tekaia on Kronos I. With a 30 copper (in-game money) cost per message the initial 100 messages were worth a total of 30 silver (30 s) to send. And additional 83 reminders were sent to players that hadn't completed the questionnaire on the 20th August, leading to a total cost of (83*0,3 + 30 = 54,9) 54,9 silver. While an addon could have been used to automate this part, it could have been detected by the bot-detection system and banned, which is why I didn't use one. No particular incentives were given to participating players. Upon sending the messages, 6 players were found either not to exist or to have been deleted; they were replaced with other players found with another /who command use.

A screenshot was taken of each answer received and the content was then copied to a Word document. All answers, whether only negative, mixed or only positive were included verbatim. Identifiying information (player names, specific country of origin or personal details) was replaced with "__". Racist comments or insults were replaced with special characters (eg. %&!^*). All participants received a thank you message upon collection of their answers.

The survey wasn't tested before use, for convenience's sake, which means it might have felt too long, too complicated, boring or might have been incorrectly interpretated. The survey was ended on the 5th september 2016, as in-game messages sent with the post system get deleted after 30 days. Data was categorized according to the player's continent of origin.

Results will be disseminated on the Kronos forums (http://forum.twinstar.cz/), in a new thread.

Results & Analysis

[5th Sept : Results are all there I'm only missing the time to parse through them]

I did not expect that players would end up answering EU/RU, due to Russia being somewhat in-between, which led me to add a RU category after a couple answers.

On the 20th August 2016, at half the duration of the survey, only 17 players (17% responses) had participated in it, with 8 answers mentioning only having positive experiences and 9 others being mixed. 7 of the positive answers were shorter than 4 words. 11 answers came from the EU, 3 from the US, 2 from RU and 1 from an undefined country. The topics emerging were latency, the blizzlike vanilla feeling, bug abuse, world chat, population, PvP, the "refugee" status, missing the server information about planned downtime, the community and being new on the server.

On the 29th August 2016, a single additional player had answered the questionnaire; the reminder used in this survey can thus be considered mostly ineffective in the short-term. Future surveys should rephrase it or replace it and search the literature for more effective strategies.

Overall, 18 players have participated in this survey. 13 (72%) were from Europe and Russia, 3 (17%) were from the USA and the others didn't specify. This survey thus better represents the views from European players playing in the late-morning. 9 players had positive experiences (50%) and the others were all mixed (50%), none were negative-only.

Due to a very low response-rate (18%) the risk of non-response bias (when the answers of respondants can significantly differ from those who did not answer) and selection bias (when the sample considered might is not representative of the overall population) can be considered high, further reducing the generalizability of my findings. Perhaps only the most positive players went ahead and discussed how they felt, but I would have expected to have at least a couple negative feedbacks from outliers. This did not happen. It might be that on average players have a positive view of the server or that those unhappy leave too fast to end up participating in the surveys.

Some players answered the questionnaire with a single "Positive." sentence, which could be due to the questionnaire being unclearly written and explained. Or they might not have felt like it deserved their time and attention. Also, rewards could have been used to increase the response-rate, in the form of in-game silver or gold, or getting a joke upon completion of the questionnaire. Although this could have lead to players answering to get the rewards. Another solution would have been to whisper players for feedback, which would have also been much faster and cost-effective.

1	EU Positive.
2	EU Positiv
3	EU Kronos I is great!
4	EU Positive
5	US So far I’ve experience Kornos to be a bit laggy as it skips a few frames here and there. My lowest ping was 213 ms and have not seen it go any lower. I was unable to find an answer on the internet whether or not users in the US also experience these types of ping. It sort of makes PvP unplayable. Other than that I enjoy the throwback to blizz-like vanilla.
6	? Positive.
7	EU / RU Mixed. Server has a lot of imperfections and defects, so the experience is often represents the knowledge about these errors, not player skills.
8	EU Very good server :D
9	EU / RU Positive :)
10	EU Enjoy playing here a lot. Prefer K1 over K2. Leveled 2 new chars recently and found a healthy leveling population. Same for endgame dungeons and raids. PvP scene is a little messed but might get better with some changes.
11	US Since I am originally from Nostalrius its definitely been a mixed experience. Being considered a refugee and not being respected as a veteran. Mostly a good time though.
12	? Mixed.
13	EU My experience with Kronos is very positive. Altho I miss a “Informationtext” when the server is about to go offline. That is pretty much the only negative about the server.
14	US I would describe my experience as neutral leaning towards positive overall. Mostly because I’m still newish to the server so I really haven’t found any particular niche yet. On first impression though, I’d have to say I’m pretty happy with how the server is run and it’s community thus far.
15	EU I enjoy the Kronos server very much. The experience is better than retail due to that the playerbase is more dedicated and helpful. Keep up the good work!
16	EU   I’ve been playing for a few months. Long time ago now since the interest just kinda went off a bit.
17	EU Nice community. A bit too much trolling on world chat and americans being americans but other than that I really like this server.
18	EU Dear _, I started playing on Kronos I not a week ago and after the retail experience back in the days. I must say that I feel the same vibe and thrill as I was playing WoW for the first time. Thank you for the awesome job. Regards, _

 Table 1 : Raw results from the survey

Limitations

The sample size was not representative and neither was the sample population, which introduced sampling bias. Indeed, Kronos is an international server [12] where european, american, asian and oceanian players meet and their playtimes aren't well coordinated, thus 11 AM CEST might not have been the ideal time to gather player names. Names were also collected on a friday, which might have had some influence due to cultural habbits and free time on different countries. The answer rate has probably been affected by the amount of players that couldn't understand the question (asian and russian players mostly) and by lack of interest. The questionnaire wasn't tested before use and could have mislead some players or been misunderstood. Some questionnaires were sent back to me at the end of the survey and the text contents appeared as a wall of text, which might have been what the players had seen and could have reduced their motivation to answer. The same players might have been surveyed on multiple characters. What surveyed players say doesn't always match what they do and players are free to lie on their answer or exagerate claims. The experiences of Horde players might not match those of Alliance players on the same server. Some errors might have been introduced by copying the responders's answers from the game to the Word document.

All results from this survey should thus be considered with caution as they are probably not sufficiently representative of the Kronos I experiences. Nevertheless an exhaustive survey wasn't the goal and the current results should help bring an overall appreciation of the multiple experiences of  Horde European players playing during the late-morning on Kronos I and their topics of interest.

This survey also isn't representative of all the players that have stopped playing on Kronos I. Most of those players probably left fast and didn't share the reasons for their leave. I would love to hear from them in the comments.

Conclusions and discussion

This research emphasizes the need to carefuly create and test a questionnaire for response-rate before sending it to a large population and how reminders can fail when not made correctly. Although the results aren't what was expected, negative research is also useful to build upon and as a way to learn about what does not work, thus preventing future loss of ressources.

Acknowledgments

I (Tekai) used to play on the Kronos I server as a level 60 warlock for months in a progression raiding guild, I wrote extensively about it, contributed to bugreports, quality improvements, discussions and have published the (now completed) Kronos WoW Comics (humoristic drawings inspired from the server) on the official Project Kronos Facebook page. I don't have any financial interest in this survey and its results and I am not financed in any way by the Kronos team or anyone else.

This is mostly an exercise of curiosity and for fun (yes my fun is wierd), please don't take it as a scientific study, but go ahead and do refer to it if you find it interesting.

References (not in APA style)

1. http://progression-annexe.blogspot.ch/2015/11/would-kronos-raiders-do-more-pvp.html
2. https://docs.google.com/spreadsheets/d/1qvqvaSJxgINkTQUXvRkpel1ij_-HGKT5o2A4bi28gZQ/edit#gid=0
3. http://www.surveysystem.com/sscalc.htm
4. http://web.mit.edu/ir/surveys/pdf/survey-guidelines.pdf
5. https://www.youtube.com/user/dodgykebaab/videos
6. https://www.reddit.com/r/wowservers/comments/4w69u2/private_server_noob_here/
7. https://www.reddit.com/r/wowservers/comments/4w4pr2/looking_for_pvp_server/
8. https://www.reddit.com/r/wowservers/comments/4jpkmw/an_honest_review_of_vengeancevoa_wow/
9. https://en.wikipedia.org/wiki/Testimonial
10. https://www.cebma.org/wp-content/uploads/Critical-Appraisal-Questions-for-a-Survey.pdf
11. http://sporkforge.com/
12. http://vanilla-wow.wikia.com/wiki/Kronos_WoW
13. http://progression-annexe.blogspot.ch/2016/02/short-experiment-do-players-come-on.html
14. https://forum.twinstar.cz/showthread.php/92687-I-guess-I-ll-be-the-one-to-say-it
15. https://forum.twinstar.cz/showthread.php/85407-What-makes-you-want-to-play-Vanilla-World-of-Warcraft-again
16. https://forum.twinstar.cz/showthread.php/98502-Why-Kronos
17. https://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf