LECTURE 4 TERM 2:
MSIN0097
Predictive Analytics
A P MOORE
SYSTEMS DESIGN
Original problem
DEALING WITH DIFFICULT PROBLEMS
Improving bad solutions
StartwithabadSolution(weaklearner)andimproveit
Buildupabettersolutionbythinkingabouthowpartialsolutionscan support/correct each others mistakes
DEALING WITH DIFFICULT PROBLEMS
Improving bad solutions
StartwithabadSolution(weaklearner)andimproveit
Buildupabettersolutionbythinkingabouthowpartialsolutionscan support/correct each others mistakes
Make the problem simpler Divideandconcur
Problemdecomposition
DEALING WITH DIFFICULT PROBLEMS
Improving bad solutions
StartwithabadSolution(weaklearner)andimproveit
Buildupabettersolutionbythinkingabouthowpartialsolutionscan support/correct each others mistakes
Make the problem simpler Divideandconcur
Problemdecomposition
Building much better solutions Deepmodels
ENSEMBLES
IMPROVING BAD SOLUTIONS
Start with a bad Solution (weak learner) and improve it
Build up a better solution by thinking about how partial solutions can support/correct each others mistakes
ENSEMBLES
IMPROVING BAD SOLUTIONS
Voting
Majorityvoting
Bagging and Pasting
Out-of-bagevaluation
Boosting
AdaptiveBoosting(Adaboost) GradientBoosting
XGBoost
Stacking
MAJORITY VOTING
B AGGING
GRADIENT BOOSTING FITTING RESIDUAL ERRORS
DECOMPOSITION
STARTING WITH EASIER PROBLEMS
Start with a hard Problem
Break the problem into a lot of easier sub-tasks
Make each subtask support the analysis in subsequent tasks easier
A B C- D ALGORITHMIC APPROACHES
A. ClAssification
B. Regression
Super vised
C. Clustering
D. Decomposition
Unsuper vised
A B C- D ALGORITHMIC APPROACHES
A. ClAssification
B. Regression
We know what the right answer is
Super vised
C. Clustering
D. Decomposition
Unsuper vised
A B C- D ALGORITHMIC APPROACHES
A. ClAssification
B. Regression
Super vised
C. Clustering
D. Decomposition
Unsuper vised
We dont know what the right answer is but we can recognize a good answer if we find it
A B C- D ALGORITHMIC APPROACHES
A. ClAssification
B. Regression
Super vised
C. Clustering
D. Decomposition
Unsuper vised
We dont know what the right answer is but we can recognize a good answer if we find it
MOTIVATING DECOMPOSITION
COMPRESSION
D. DECOMPOSITION 2. PROJECTION METHODS
Dimensionality reduction
D. DECOMPOSITION 2. KERNEL METHODS
D. DECOMPOSITION 3. MANIFOLD LEARNING
CURSE OF DIMENSIONALITY
SUBSPACES
MOTIVATING DECOMPOSITION
LOW DIMENSIONAL SUBSPACES
DECOMPOSITION
THREE APPROACHES
Dimensionality Reduction / Projection Kernel Methods
Manifold Learning
B. REGRESSION REAL VALUED VARIABLE
MOTIVATING PROJECTION INSTABILITY
FINDING THE RIGHT DIMENSION
SUBSPACES
PROJECTION IN MULTIPLE DIMENSIONS
REDUCTION TO A SINGLE DIMENSION
COMPRESSION
MNIST 95% VARIANCE PRESERVED
PROBLEMS WITH PROJECTION
PROBLEMS WITH PROJECTION
KERNEL METHODS
Kernel spaces
KERNEL PCA
MANIFOLD METHODS
Manifold learning
MANIFOLD LEARNING
MANIFOLD LEARNING
OTHER TECHNIQUES
LOCAL LINEAR EMBEDDING
DECOMPOSITION METHODS
Random Projections
Multidimensional Scaling (MDS) Isomap
Linear Discriminant Analysis (LDA)
ADVANTAGES
The main motivations for dimensionality reduction are:
To speed up a subsequent training algorithm (in some cases it may even remove noise and redundant features, making the training algorithm perform better).
To visualize the data and gain insights on the most important features. Simply to save space (compression).
DIS AD VAN TAGES
The main drawbacks are:
Some information is lost, possibly degrading the performance of subsequent training algorithms.
It can be computationally intensive.
It adds some complexity to your Machine Learning pipelines. Transformed features are often hard to interpret.
WHEN IT DOESNT WORK
IMPLICIT ASSUMPTION IT MAKES THE PROBLEM EASIER
EMBEDDING PROJECTOR
GOOGLE BRAIN TEAM 2016
Reviews
There are no reviews yet.