Motivation

In this project, we apply machine learning techniques to predict the prices of goods in a video game’s virtual economy. This task is similar to the classic task of predicting stock prices on the stock market, yet also has the added novelty of being tied into the video game’s well defined dynamics, making this a very interesting task. While stocks in the real marketplace exhibit chaotic and uncertain relationships with one another, video game market items have pre-programmed and well-documented interactions and relationships. Choosing a virtual game-based economy gives us an opportunity to pioneer the entire machine learning process in a classically familiar yet unique space.

In our investigation, we focus on Runescape’s virtual economy due to the game’s centralization of the economy in a well documented virtual marketplace where data is aggregated and historical records can be accessed through provided APIs.

Data

The Runescape API provides price and category data, but does not provide any other attributes such as other pricing or trading metrics. Using their provided API, we first scraped this and arrived at a daily updated times series list of prices for all 4106 tradeable virtual items over the course of 205 days, each with a category attribute attached.

Final Feature Set

After an iterative feature set creation process, we settled on a delta approach feature set. This delta approach feature set contained summary variables based on adjusted daily changes as a set of changes over 30 days as follows:

% change from today to yesterday

% change from yesterday to 2 days ago

…

% change from 28 days ago to 29 days ago

Using this set, we predict the direction (e.g. up/down/same) of the change, and also the exact % change (e.g. --2%,0,+1%) for the next day. This way, we get to test regression and classification, and see how a trade off in prediction exactness (for just predicting a direction without giving a magnitude) can affect accuracy. We used many learners including Decision Trees, Nearest neighbor, and Multilayer Perceptrons.

Enhancing Model Generated Predictions

To test how models can predict farther into the future, we also created a long term predictor which iteratively fed results repeatedly through the 1 day regressor to predict prices up to 25 days into the future.

Short term predictions, crafting only

Learner	Accuracy	F-Measure
ZeroR	56.685%	0.410
J48	74.8443%	0.743
Multilayer Perceptron	69.6051%	0.081
k-Nearest Neighbor	66.7737%	0.064

The learners perform reasonably well on immediate validation data, without applying our iterative prediction extender.

Long term predictions crafting and all

Learner	Accuracy on All Data	Accuracy on Crafting Data Only
Perceptron	41.3%	32.7%
REPTree	52.3%	52.3%
Bagging REPTree	44.2%	48.1%
Random Tree (depth of 15)	56.7%	61.5%
ZeroR	60.6%	58.1%

Most learners actually did worse over a long term period than ZeroR. In fact, these results fare worse than our 1 day predictions. We see that some of our learners while effective, are only effective in the short term before applying our iterative learner extension to predict 30 days into the future.

Analysis