Fuzzy Math, Part 1, The Theory
If you are interested in a very flexible rule-based system and want it to be easily integrated with for instance adaptive systems, then fuzzy logic provides a good solution.
So what is fuzzy logic?
Take a look at the first sentence in this text. Expressions like "very flexible", "easily integrated" and "good solution" were used. This type of vague expressions are characteristic of the way we humans communicate through language and as such is an integral part of our thinking process. This contrasts sharply with the traditional Boolean logic of computer programming. Fuzzy logic bridges the gap between them, providing a framework that allows you to numerically encode linguistic expressions and through that gives you a flexible rule-based system.
This is the first part of a two-part tutorial and will cover the basic fuzzy logic theory. The second part will be more practical and deal with fuzzy logic in Synapse and the creation of hybrid fuzzy-adaptive systems.
Updates (1.04) Released
We have released a few updates to Synapse containing a number of improvements, bug fixes and new features:
- The Synapse core loader has been optimized for faster startup of the program.
- A number of GUI improvements have been made and some GUI related bugs have been fixed.
- The Update Engine has been rebuilt to minimize timeouts and network latency, resulting in a faster startup.
- The Data Source component is now optimizable so that you can use an optimizer to select the optimal input features.
- A new and better fitness-evaluation function has been implemented for the optimizers
- Data flow bugs in the Crop Filter and the Mixer Filter have been fixed
- A memory leak in the Gamma Memory has been fixed.
- A new filter, the Wavelet Filter has been added. It performs a discrete wavelet decomposition of a signal and is a very powerful tool for analyzing time series data and provides very helpful preprocessing for dynamic adaptive systems.
To get the latest updates, make sure that you have automatic updates turned on and that you have a working Internet connection and Synapse will handle the rest. This update contains 18 components and is 3.2 MB in size.
/Thomas @ Peltarion
Classifier Showdown
In this article we will take a closer look at three different classifiers and discuss three different types of classifiers: naive bayesian classifiers, support vector machines and modular multilayer perceptron neural networks.
To help us investigate the relative benefits of the system we're going to use a simple application that uses adaptive systems deployed from Synapse.
The talented Dr.Hebb, Part 2, PCA
Besides novelty filtering that was covered in Part 1, there is another interesting function that a Hebbian Layer can perform and it is called Principal Component Analysis (PCA). This time we are going to take a closer look at PCA and see how it can be used in Synapse in combination with regular neural networks. PCA is a linear transformation that can be used to reduce, compress or simplify a data set. It does this by transforming the data to a coordinate system so that the greatest variance of the data by a projection of the data ends up on the first component (coordinate), the next one in line on the magnitude of variance ends up on the second component and so on. This way one can choose not to use all the components and still capture the most important part of the data.
To understand what this means, we can take a look at a 2D example. Suppose we have some X-Y data that looks something like this:
To see how the data is spread, we encapsulate the data set inside an ellipse and take a look at the major and minor axes that form the vectors P1 and P2
These are the principal component axes - the base vectors that are ordered by the variance of the data. PCA finds these vectors for you and gives you a [X,Y] -> [P1, P2] transformation. While this example was for 2D, PCA works for N-dimensional data, and it is with high dimensionality problems it is generally used.
Let's take a look at how it can be used in practice, its limitations and how it is done with Hebbian learning:
Updates
We have released a few updates to Synapse containing the following bug fixes:
- The Function Layer, Weight Layer and Hebbian Layer components can now lock/release the number of features when changed manually in settings browser. See more detailed explanation below.
- Plots now save the number of input features to their XML definition.
- A bug in the Generalized Hebbian Algorithm that caused the principal components to be incorrectly scaled has been fixed.
To get the latest updates, make sure that you have automatic updates turned on and that you have a working Internet connection and Synapse will handle the rest. This update contains 5 components and is 452 kb in size.
Hebbian novelty filters for financial analysis
I have been asked to post a few words about novelty filtering stock market data. This can also of course be applied to any other system, including, but not limited to forex data and other financial time series.
What anti-Hebbian novelty filters can do for you:
- Detect previously unseen trading patterns
- Give a quantitative measure of how much this new pattern differs from old trading patterns
What anti-Hebbian novelty filters can't do for you:
- Predict what consequence (if any) the new trading pattern will have on for instance the stock price
- Give you the direction of or type of change
So why would you want to use novelty filters if they can't tell you the consequence of the change?
It's simple: You use neural networks and other technology to try to model the market. You don't throw a dice or flip a coin.
When the market exhibits behaviour it hasn't shown earlier and you choose to trade, then you are just picking random actions. All predictive models are based on the assumption that there is an analyzable pattern. You find this pattern by looking at historical data.
If the market is behaving in a way no represented by the historical data, then your model is invalid.
Let's look at a practical example, and I'll then show you how to do it in Synapse.
What we have here is the output of an anti-Hebbian filter (top graph) and the closing price of the Microsoft shares. As you can see, around sample 209-210, the novelty filter detected something was up - three days before the stock went down significantly (on April 27). This would have been a very good signal to get out of the market.
Let's take a closer look on how this is done in Synapse and what it means:
Update 3.4 Released
We have rolled out a new update, (1.0)/3.4 which contains a number of bug fixes and some new features. The update will be automatically downloaded and installed when you start Synapse (provided that you are connected to the internet)
New features:
- The new Updater2 update engine with an improved versioning system, download system and support for a more advanced release structure.
- Improved CSV input format that provides more features for handling string data (enumeration and removal in addition to the existing expansion):
- Generalized Hebbian Algorithm (GHA) update rule. A new Hebbian type update rule that performs PCA. The difference between it and Oja's rule is that GHA results in sorted principal components.
- Splash screen, minor improvement with verbose loading of plugins
- Start screen, some visual improvements
- Two more plot line styles and marker support added to the Scatter Plot:
- Data embedded in solution is now compressed
Bug fixes:
- Component GUI flicker fixed
- Select Filter inadvertent sort fixed
- Hebbian component GUI refresh in training fixed
- Script Filter, disconnected GUI elements removed
- Unstage (scaling) in the Sink component fixed
- Column resize in GridView enabled
The Talented Dr. Hebb, Part 1, Novelty Filtering
One of the most under-appreciated types of adaptive system are the ones that use Hebbian learning. It is because of their simplicity that they get ignored, but as we shall show, they do have some practical applications for which they are really good. In Synapse Hebbian learning is embodied by the Hebbian Layer component and the Hebbian and Oja's update rules.
Synapse Tutorial IV Released
The fourth tutorial, "The Cows of Navarone", in the Synapse tutorial series has been published. It covers the use of visualizers for creating local filters, how to create user snippets, how to trace errors back to source data using the Error Analyzer, how to real-time test systems using the Probe and how to make a qualitative analysis of a system using sensitivity analysis.
The tutorial assumes that the three previous tutorials have been completed and the material understood.
Excerpt from the tutorial intro:
"Greece and the islands of the Aegean Sea have given birth to many myths and legends of war and agriculture. And those once-proud stones, of ruined and shattered temples bear witness to the civilization that flourished and then died here and to the demigods and heroes who inspired those legends on this land and these islands. But, though the stage is the same, ours is a legend of our own times, and its heroes are not demigods, but the people form the National Statistical Service of Greece.
In the last decade of the 20th century, so the story goes, people from NSSG collected agricultural data in the rural parts of the Greek mainland and islands. Their goal is said to be collecting background data for a new taxation model. Truth be told, we may never know as the full description has been lost in the vast regions of the public domain. The written records of the data however remain, and we shall use them to model the presence on these lands of the bovine creatures named "The Cows of Navarone".
The data contains a number of variables measured at different villages in Greece. Each sample represents one village. One of these variables is the number of cows in the village, and this is the one we will try to model, given the other variables."