Statistical analysis of missions

  • Greetings!


    A few years ago I decided to put some science in the game, and do some statistics to improve my strategy, and get some understanding of how the mission system exactly works. I have collected the data of more than 10 000 missions so far, and I made several observations. I open this thread to share the knowledge, and probably answer questions.

    I am playing on four servers (all Hungarian), and collected the data with knights of different levels, covering all the map locations and both karma, ordinary and event opponents.

    In my posts I am going to not just answer some topics, but show statistical evidence, mathematical deduction and diagrams to support my statements.

    In some cases, I found that official description and Battleknight Lexicon is false or inaccurate.


    While I can give a comprehensive model of the mission system, I have also found the formula to calculate the daily silver cost of order castle developments, and still working on the exploration of dueling and group mission system.


    I will write about topics like:
    - How the level of the knights counts in calculating the silver loot?
    - How the size of mission counts (spoiler: it's obvious, but I will prove it for fun)?
    - What is the meaning of profit classes (Normal, Large, Phenomenal)?
    - What is the frequency of profit classes? How are they depend on Luck?
    - How profit changes in towns, mission locations?
    - How profit depends on the various opponents?
    - How profit depends on winning, losing, or draw?

    - etc.


    If you don't find the proving methods interesting, you can skip it: at the end of each post, I will write a short summary.

  • Really interesting, how do you collect this information? Can you share underlying data ? Is it available in a database or a different object storage service?

  • 10k data are really few to base it to different locarion, levels and type of missions. Moreover you should also give the analisys formula.



    You added too many variablea to be robust.

    .ie If you use 4 knight to different levels, it's 2500 each one

    if you think about the localities avalaible they vary from 3 to 18 (fehan excluded), it means 139 each locality, that you have to divide to the different level of mission 20-40-60, so 46-47, and again to different gains, so normal, great and fenomenal.


    so, not much results tk be statistically worthy


    wysiwyg image

  • All right, let's start!

    CHAPTER 1: Profit range

    There are a lot of factors, which can change in missions. Until we don't know, how exactly any of these factors count, we shall choose as many circumstances constant, as we can.

    The Easter Event is a good opportunity, because in Grand, the Egghatch Grotto location appears, with only one opponent, the Guardian.

    This way, we can attack in one town, one location, one opponent over and over again. I choose one knight, with 89 level, which won't change. I discount profits in classes "Large" and "Phenomenal", and losses or draws. I do only 10 minutes missions. This knight also have a +3% silver bonus.

    So the following factors are constant:

    - level
    - profit class
    - town
    - location
    - fight result
    - size
    - bonus

    The result is 179 missions, with a minimum profit of 2069 and maximum of 3081.

    Dividing the max with min gives the ratio, which is 1.4891251812469792.

    It is clear, that the profit changes randomly between a lower and higher limit, and we can suspect, that higher limit = lower limit * 1.5
    The observation can be repeated with a level 19 and a level 69 knight:

    LEVEL 19: COUNT: 98, MIN: 436, MAX: 647, RATIO: 1.4839449541284404
    LEVEL 69: COUNT: 97, MIN: 1572, MAX: 2303, RATIO: 1.4650127226463103

    Finally, these diagrams show the profits in ascending order:


    Level 19:

    b24ZpDH.png


    Level 69

    2dXJ82n.png


    Level 89

    5DEmDpE.png


    As the lines look somewhat straight, we can guess, that the distribution of values in range might be uniform (every value has the same probability), however, it will be easier to prove with bigger amount of data and different type of diagrams.

    Don't forget: we excluded many factors to make these observations!

    SUMMARY:
    - The range of basic profit has an upper and lower bound.
    - The upper bound equals lower bound * 1.5
    - The profit values might have equal probability inside the range.

  • CHAPTER 2: Levels

    In this post, I show the relationship between the level of the knight and the profit.

    In the previous post I used three sets of data, differing only in the levels of the knights: 19, 69 and 89. One by one, they were convenient to raise suspicion, that the profit ranges between +- 20% limits. However, if we understand the level factor, we can examine the three sets of data as one, and see a more precise diagram.

    First, I illustrate the average of profit of each three levels. Note, that the averages fit on a zero origin straight line:

    5TSw4R3m.png?1   92iBh4cm.png


    The slope of the line is 28.15. This will be important later.

    The zero origin straight line suggests direct proportionality. In other words, level factor counts as a simple multiplier when calculating the profit. If we take profits of different level knight, we can divide the profit with the levels to get them on common grounds. Let's see, if it works:

    This diagram shows the profits divided by level of three knights in ascending order.

    ASltoWJl.png


    This way the three lines are very similar. From now, we can take the profits of all different levels as one, after we divided with the level.

    Lets see the experiment of the first post again, but this time, instead of three sets of 179, 98 and 97 data, with one set of 519 (including missions with any levels in the database).

    COUNT: 519, MIN: 22.497420020639833, MAX: 33.74025974025974, RATIO: 1.4997390682711784

    Even in five times greater sample, the ratio is close to 1.5, so the previous statement was right. To be exact, we have discovered two multipliers now: one is the level of the knight, and the second is a random factor, that changes between 1.0 and 1.5 and provides the variability of the profit.


    If I illustrate the values in ascending order, the diagram is still almost straight, suggesting, that the distribution of the random variable is uniform, every value between 1.0 and 1.5 has equal probability, and the distribution is continuous, which means that the value can be any real number in range (opposing discrete distributions, in which case the diagram would look like a stair)

    BJS7Q2Dl.png


    SUMMARY:

    - Level of the knight counts as a simple multiplier

    - The statements of the first post are valid, even with bigger data set.

  • you are not using in the right way the data. You see something and you modify everything to fit the data. Still you ammass the data instead to detail them. You have too few data and work on.



    Moreover you assume on the event of Easter, that can be valuable, but you are at the best simple making an analysis of the event itself, not of the usual days


    you make assumptions and adjust the ideas based on that.


    there are some thimgs that are true, es, the level factor, but it's somethimg that you know from the beginning and ccan't be said to be deduced.


    You are cutting out too many errors in the calculations.


    wysiwyg image

  • CHAPTER 3: Size of mission

    The size of the missions can be small (10 minutes), medium (20 minutes) and large (30 minutes).

    Most of the time I do small missions to maximize the number of missions. This way, data gathers faster. However, this way my database consists of almost small missions: 10360 small missions, 65 medium missions, and 350 large missions.

    The data we have seen so far, belonged to one opponent. Unfortunately, all of them are small missions.

    The factor of missions size looks the most obvious among factors: 1x, 2x and 3x multiplier.

    Actually, it's really that simple.
    However, if I want to prove it with numbers, I must cheat a little, and make a big leap: let's assume, that EVERY factor is known and dropped out.
    Now, only two things changes: the random factor, and the size of mission.
    We have seen, that the random factor changed between 1.0 and 1.5, so the average is 1.25. If the above statement is true, then we expect medium missions to change between 2.0 and 3.0 with average 2.5, and large missions to change between 3.0 and 4.5 with average 3.75.
    Also, there is a huge difference in the number of data rows, so lets random choice 65 small, 65 medium and 65 large mission, to see a nice diagram.

    JABhaL4.png


    Let's not waste more words on the obvious.

    SUMMARY:

    The small, medium and large mission means 1x, 2x and 3x multiplier.

  • Deynon

    Note, that statistics starts with setting up work hypothesis, and my analysis builds up by dropping factors, to examine only one at a time. I have covered like 500 data out of 10000 so far. Please be more patient, because this is a series of posts, and I'm moving step by step. Also, feel free to point out actual errors in the deduction, instead of repeating that the data is too few. I will cover every location and every opponent, as I promised. There is good reason behind the order I make statements.

  • CHAPTER 4: Profit range (round 2)

    Deynon pointed out right, that the previous data was all based on the Easter event. It was justified however by the fact, that location and opponent type are two unknown factors (yet), and unknown factors are dropped out of the examination by being made constant. This way, the field of examination can be narrowed down to one factor at a time. I have showed statistical evidence of the role of three factors so far. Even is some of them was previously known or suspected, I have shown hard evidence know. Further calculations will be always made by taking into consideration the previous ones. Making assumptions and hypotheses are perfectly fine in statistics, even if they come from small sets of data. Hypotheses can be tried again on bigger and different data sets, to see if they are right or wrong, and if right, are they generally right or just in special cases. I don't want to spoiler too much, but I say so much, that the level and mission size factor are generally right to every mission. The random factor is tricky, but there are some other factors I must uncover, before I return to that.

    As we have seen, level and mission size are multipliers, so we can divide by them to obtain unified data, independent of these two known variables. The following data shall be interpreted this way. We still look at "Normal" profit class, and winned missions only.

    My hypothesis at the Guardian was that the ratio of the maximum and minimum profit is 1.5 at most. Yes, one might object, this is just one opponent, and not even a regular one, but one from an event. Let's see the case with some different opponents:


    NAME TOWN LOCATION NUMBER MIN PROFIT
    MAX PROFIT MAX/MIN
    Magician Endalain Mystical Tower
    225 15.0 22.5 1.5
    Assassin Asgal Coast 87 8.02879895276535 11.8141158503327 1.47146739130435
    Black dragon Talfour Dragon Lair
    90 22.5220139986453 33.6538461538462 1.4942645074224


    Let's be honest: I have proved the 1.5 limit to 3 more opponent (non-event ones), but not all of them (not yet).
    Instead of listing all of them right know, I want to show something more interesting in the next chapter. Let's just state know, if we can find just one of any town, any location, any opponent, any mission, that goes significantly outside the 1.5 range, that would contradict the hypothesis.

    SUMMARY:
    I showed, that three non-event opponents also give profit in +-20% range.

  • CHAPTER 5: Towns and locations

    In this chapter I am going to examine the towns and locations effect on the profit.

    We know, that one type of opponent can be found on several locations on the map. This is not completely random: each mission location is inhabited by 3 different opponent for each karma, that are fixed. In most of the locations one of the opponents is common for the two kind of karma, but for a few locations it is possible to have two or zero in common. I will show a complete table of this later, but today, I will use the fact, that one opponent can be found on more than one place. This way, if town and location has any effect on the profit, it can be shown on the different occurrences of the same type of opponent.

    Bandits are exclusive positive karma opponents, and they dwell on the locations below:

    TOWN LOCATION
    Talmet Cave
    Ramstill Cave
    Tarant Bandit lair
    Waile Harbour
    Hatwig Bridge


    As we have seen in the previous post, one opponent on one location is expected to give profit in +-20% range.
    If this is true, than one specific Bandits may give profits in it's own range, but if town or location has an effect on the profit, than other Bandits are expected to have profits extending beyond this first range.


    Let's see a graph with all the 83 Bandits profits (divided by level and size, Large and Phenomenal profit class, draw and defeat excluded). The different color means different location.

    CbFdJIN.png


    It is a bit messy, so lets shift the different locations vertically:


    3qpgqdB.png


    We can see, that 7.5-11.25 is the range, where all the Bandits profit fits, regardless of the location.

    Let's see two other opponents: Scout (exclusive negative karma) and Troll (exclusive positive karma)

    ub0ij5L.png


    The Scout has 114 missions on 6 different places, fitting in the range 6.0-9.0. The Troll has 67 missions on 5 different places, fitting in the range 9-13.5.

    SUMMARY:
    This is further proof of the 1.5 range.
    There is proof, that town and location have no effect on the profit. (I must warn however, that in the next post, I will show exceptions)
    The graphs also show, that three different opponent has three different profit range.

  • Chapter 6: Towns and locations (round 2)


    As I stated, town and location don't effect, how much profit a specific opponent will give. This concept works in almost all cases. However, as we collect data from all the places, two exceptions appear.

    First, I illustrate the missions of Phoenixes on different locations, the same way as in the previous post:

    nNNruBp.png


    As we can see, there is one Phoenix among the others, that gives significantly more silver, than all the others. The others fit inside a 15.0-22.5 range, while the exception fits inside 22.5-33.75. This exceptions is the Phoenix of the Dragon lair in Talfour.

    The second example is the Mermaid:

    bgbrZWb.png


    The average Mermaid fits between 7.5 and 11.25. The exceptional Mermaid is one from Fehan Island. Fehan Island is an event location, and every opponent gives high profit there. All opponents give the same profit on the island, to be exact.


    3h47EXA.png


    SUMMARY:

    - There is further proof, that town and location have no effect on the profit that a specific opponent gives.

    - There is two exceptions however, the Phoenix of Dragon lair in Talfour, and the Mermaid on Fehan Island's Lagoon.
    - The opponents of Fehan Island have the same profit range.

  • Chapter 7: Fight results


    The result of a mission can be win, lose or draw. The profit of the mission depends on the result. Today I'm going to show how.

    When I do missions, I win almost every time. To gather lost missions, I took my weapon and armor off. Gathering draw missions is almost impossible. I only managed 2 times out of 11000 missions. This is not much, but even from this little information we can make a careful estimation.

    First, let's see only the win and lose missions:

    K1W5u33.png


    This diagram clearly shows, that missions that are won give exactly 3x more silver, than lost ones. Multiplying lost missions with three, or dividing won missions with three unifies the data of different fight results. In the future, I am going to divide won missions by 3 to drop out the result of the fight from the calculation, and I state, that winning means a 3x multiplier.

    If we look back at the diagram, we can notice, there is a perfect place for draw missions halfway between lost and won missions, and we can speculate the multiplier to be 2x.

    LM3O64X.png


    Even the two draw missions are enough to contradict this speculation. We cannot determine the full range of draw missions (because the two samples are too close to each other), therefore we cannot say for sure, if draw mission profits equal won mission profits. However we can say for sure, that the multiplier must be greater, than 2x.


    SUMMARY:

    - Fight result is a multiplier: 1x for losing, 3x for winning.
    - Draw mission multiplier is probably 3x, the same as winning.

  • Chapter 8: Profit leagues


    The previous chapters hinted, that different opponents give different profit. Today I show an all inclusive diagram.

    There are 70 opponents in the game in total. The Phoenix in the Dragon lair, and the Mermaid in Fehan Island are registered as separate opponents from normal Phoenix and normal Mermaid. The number is 70 this way. The dragons of the Dragon Event, the monsters of the Halloween Event, the Guardian of the cave of Easter Event, and the six opponent of Fehan Island are included.

    As I showed before, the same opponent of different locations gives the same profit. As a result, I can mass together the samples of different locations. The profit range is always +-20% from the average. We filter the data to examine profits of Normal class. The profits are divided by level, size factor and result factor. As there are 70 opponents, each opponent is represented on it's own row along the y-axis. Opponents are sorted by their determined league.


    I8cjN5t.png


    Every opponent can be described with one number. That number is what I call profit league. This number means a theoretic absolute minimum profit. As we can note from the diagram above, some distinct opponents belong to the same profit league, so the numbers are not completely unique. In other words, if a 1 level knight attacks an opponent, chooses small size mission, gets a Normal class profit, loses the battle, has no bonus and the random factor happens to be the minimum 1.0, then the theoretic absolute minimum profit is acquired (disregarding the rounding).


    There are 16 leagues, and their values are:

    1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.
    Value 1.0 1.25 1.75
    2.0 2.25 2.5 2.625 2.75 3.0 3.25 3.5 3.75 4.25 5.0 7.5 10.0
    Number of
    opponents
    1 1 1 7 2 14 1 2 3 2 1 8 2 6 13 6


    This is how the diagram looks with the leagues indicated with black lines:

    suedo6s.png


    Now, we can take the samples of all the opponents, divide them with one more factor, their respective profit league factor, and then the only thing should be left is the random factor, that is known to be 1.0-1.5. This will be demonstrated with 6212 sample!

    tSzGKBb.png


    Once again, it is between 1.0 and 1.5, and it is straight, so it's uniform distribution. Level, mission size, town, location, fight result and opponents make no difference.


    SUMMARY:

    - There are 70 opponents in the game, event opponents included.

    - The profit of each opponent is described by the theoretic absolute minimum profit, what I name "league" or "profit league".

    - There are 16 leagues. Some opponents belong to the same league.

    - The lowest league is 1.0, the highest is 10.0.

    - The random factor is between 1.0 and 1.5 and is uniformly distributed. The factors surveyed so far make no change to this.

  • So, can you please summarize the point? Which missions should we do for the best development or which missions give us the most silver?

    What is the best amount for attributes? For example: Is it enough if my stats are around myLevel*4 or should it be myLevel*5? Of course not all the attributes, luck is more important than health... But how much is the difference? Please try to make it clear

  • There are a few more posts left, until I can summarize all. However, because I have lots of work these days, I take some break before the next chapter. It will be a longer one, explaining each mission opponent regarding their leagues, and the karma and xp they give, their location and the respective possibility they can be found. After that, in another chapter, I will show the list of mission locations ordered by profit. Finally, I will explain the profit classes. This will result in a complete model of the mission system, and I will be able to show, that attributes have no effect on the profit from missions.

    I want to explain the effect of attributes in dueling too, but I didn't even started to examine that, I am only gathering the data so far. I am going to open a new thread to discuss that topic.

  • Thanks to the data provided by Mistrz Marek , I can redraw the diagram from Chapter 7, further proving, that draw missions indeed give the same profit as won missions.

    (y axis has no meaning here. The three sets of data are pulled apart along the y-axis for illustrative reason)

    4ceGaIY.png