Synaptic

Besides novelty filtering that was covered in Part 1, there is another interesting function that a Hebbian Layer can perform and it is called Principal Component Analysis (PCA). This time we are going to take a closer look at PCA and see how it can be used in Synapse in combination with regular neural networks. PCA is a linear transformation that can be used to reduce, compress or simplify a data set. It does this by transforming the data to a coordinate system so that the greatest variance of the data by a projection of the data ends up on the first component (coordinate), the next one in line on the magnitude of variance ends up on the second component and so on. This way one can choose not to use all the components and still capture the most important part of the data.

To understand what this means, we can take a look at a 2D example. Suppose we have some X-Y data that looks something like this:

To see how the data is spread, we encapsulate the data set inside an ellipse and take a look at the major and minor axes that form the vectors P1 and P2

These are the principal component axes - the base vectors that are ordered by the variance of the data. PCA finds these vectors for you and gives you a [X,Y] -> [P1, P2] transformation. While this example was for 2D, PCA works for N-dimensional data, and it is with high dimensionality problems it is generally used.

Let’s take a look at how it can be used in practice, its limitations and how it is done with Hebbian learning:

(more…)

We have released a few updates to Synapse containing the following bug fixes:

• The Function Layer, Weight Layer and Hebbian Layer  components can now lock/release the number of features when changed manually in settings browser. See more detailed explanation below. 
• Plots now save the number of input features to their XML definition.
• A bug in the Generalized Hebbian Algorithm that caused the principal components to be incorrectly scaled has been fixed.

To get the latest updates, make sure that you have automatic updates turned on and that you have a working Internet connection and Synapse will handle the rest. This update contains 5 components and is 452 kb in size.

(more…)

I have been asked to post a few words about novelty filtering stock market data. This can also of course be applied to any other system, including, but not limited to forex data and other financial time series.

What anti-Hebbian novelty filters can do for you:

• Detect previously unseen trading patterns
• Give a quantitative measure of how much this new pattern differs from old trading patterns

What anti-Hebbian novelty filters can’t do for you:

• Predict what consequence (if any) the new trading pattern will have on for instance the stock price
• Give you the direction of or type of change

So why would you want to use novelty filters if they can’t tell you the consequence of the change?

It’s simple: You use neural networks and other technology to try to model the market. You don’t throw a dice or flip a coin.

When the market exhibits behaviour it hasn’t shown earlier and you choose to trade, then you are just picking random actions. All predictive models are based on the assumption that there is an analyzable pattern. You find this pattern by looking at historical data.

If the market is behaving in a way no represented by the historical data, then your model is invalid.

Let’s look at a practical example, and I’ll then show you how to do it in Synapse.

What we have here is the output of an anti-Hebbian filter (top graph) and the closing price of the Microsoft shares. As you can see, around sample 209-210, the novelty filter detected something was up - three days before the stock went down significantly (on April 27). This would have been a very good signal to get out of the market.

Let’s take a closer look on how this is done in Synapse and what it means:

(more…)