Scribd Logo
Încărcați

Bine ați venit la Scribd!

  • Neural Network Ensemble 정리 자료

    Încărcat de

    이성령
    186 vizualizări15 pagini
    1. Neural networks have high variance which can be reduced using ensemble learning by combining predictions from multiple neural network models. 2. There are three main techniques for creating ensembles: varying the training data, varying the models, and varying the combinations of predictions. 3. Varying the training data involves training models on different subsets of data using techniques like cross-validation, bagging, and random training subsets. Varying the models involves changing properties of the models like hyperparameters, initialization, and architecture. Varying combinations involves averaging predictions, weighted averaging, stacking, and boosting.

    Descriere originală:

    Ensemble Neural Netowork.

    Drepturi de autor

    © © All Rights Reserved

    Formate disponibile

    PPTX, PDF, TXT sau citiți online pe Scribd

    Vi se pare util acest document?

    Este necorespunzător acest conținut?

    Raportați acest document
    1. Neural networks have high variance which can be reduced using ensemble learning by combining predictions from multiple neural network models. 2. There are three main techniques for creating ensembles: varying the training data, varying the models, and varying the combinations of predictions. 3. Varying the training data involves training models on different subsets of data using techniques like cross-validation, bagging, and random training subsets. Varying the models involves changing properties of the models like hyperparameters, initialization, and architecture. Varying combinations involves averaging predictions, weighted averaging, stacking, and boosting.

    Drepturi de autor:

    © All Rights Reserved
    Descărcați ca PPTX, PDF, TXT sau citiți online pe Scribd
    Indicator pentru conținut neadecvat
    186 vizualizări15 pagini

    Neural Network Ensemble 정리 자료

    Încărcat de

    이성령
    1. Neural networks have high variance which can be reduced using ensemble learning by combining predictions from multiple neural network models. 2. There are three main techniques for creating ensembles: varying the training data, varying the models, and varying the combinations of predictions. 3. Varying the training data involves training models on different subsets of data using techniques like cross-validation, bagging, and random training subsets. Varying the models involves changing properties of the models like hyperparameters, initialization, and architecture. Varying combinations involves averaging predictions, weighted averaging, stacking, and boosting.

    Drepturi de autor:

    © All Rights Reserved
    Descărcați ca PPTX, PDF, TXT sau citiți online pe Scribd
    Indicator pentru conținut neadecvat
    din 15

    Neural

    Network
    Ensemble
    •Neural Network drawback
    • Neural Network has a high variance
    • Nonlinear (a high variance)

    Reduce Model •Combine the predictions from multiple nn


    Variance with 1. Reduce the variance of predictions
    2. Reduce Generalization Error
    Ensemble
    Learning •Techniques
    • Training data
    • The model
    • How predictions are combined
    Why we use Ensemble Neural Networks

    Variance Bias Variance Bias


    How to Ensemble Neural Network Models

    • Training Data
    • Vary the choice of data used to train each model in the ensemble.

    • Ensemble Models
    • Vary the choice of the models used in the ensemble.

    • Combinations
    • Vary the choice of the way
    that outcomes from ensemble members are combined.
    Varying Training Data

    K-Fold

    Resampling the training dataset

    Bootstrap

    Select a random subspace


    of the input space
    Varying Models
    Vary the Configuration of each ensemble model
    • Have a lower correlation in their predictions

    • Different capacity • Use Saved Model


    • Number of layers • Checkpoint
    • Number of Nodes • Snapshot
    • Different Conditions • horizontal ensembles
    • Learning rate
    • Vertical Ensemble
    • Regularization
    • Intermediate hidden neural
    • Minibatch
    Networks
    • Random initialization
    Varying Combinations

    1. Average of the predictions


    from the ensemble members

    2. Weighting the predictions


    from each model(model blending)

    3. Model stacking
    Varying Training Data (Chapter 22)
    – k-fold Cross-Validation Ensemble.
    – Bootstrap Aggregation (bagging) Ensemble.
    – Random Training Subset Ensemble.

    Varying Combinations Varying Models


    – Model Averaging Ensemble (Chapter 20). – Multiple Training Run Ensemble (Chapter 20
    – Weighted Average Ensemble (Chapter 21). and Chapter 21).
    – Stacked Generalization (stacking) Ensemble – Hyperparameter Tuning Ensemble.
    (Chapter 25). – Snapshot Ensemble (Chapter 24).
    – Boosting Ensemble. – Horizontal Epochs Ensemble (Chapter 23).
    – Model Weight Averaging Ensemble – Vertical Representational Ensemble.
    (Chapter 26).
    Chapter 20
    Combine Models From Multiple Runs with Model Averaging Ensemble

    Average Class Probability

    Average Class Prediction

    Save and Load each Top Model

    Sensitivity of Variance
    Chapter 21
    Contribute Proportional to Trust with Weighted Average Ensemble

    1/3 -> 1/5 GridSearch Of Weight Coefficients

    RandomSearch Of Weight Coefficients


    1/3 -> 2/5

    Try a Local Search


    1/3 -> 2/5 (differential_evolution)
    Chapter 22
    Fit Models on Different Samples with Resampling Ensembles

    Random Split

    10-fold Cross Validation

    Bootstrap Aggregation

    Combine models using a bagging ensemble


    Chapter 23
    Models from Contiguous Epochs with Horizontal Voting Ensembles

    Epoch(N)
    Model

    Epoch(N+1) Horizontal voting ensemble


    Model
    (Use the contiguous models)

    Epoch(N+2)
    Model
    Chapter 24
    Learn to Combine Predictions with Stacked Generalization Ensemble

    Snapshot Ensembles

    • Vary Cycle Length


    • Vary Maximum Learning Rate
    • Update Learning Rate Per Batch
    • Repeated Evaluation
    • Cyclic Learning Rate
    Chapter 25
    Learn to Combine Predictions with Stacked Generalization Ensemble

    Stacked Model

    • Alternate Meta-Learner
    • Single Level 0 Model
    • Vary Level 0 Models
    • Cross-Validation Stacking Ensemble
    • Use Raw Input in Meta-Learner
    Chapter 26
    Combine Model Parameters with Average Model Weights Ensemble

    Average of the weights from multiple models

    S-ar putea să vă placă și