scikit-learn: machine learning in Python

Easy-to-use and general-purpose machine learning in Python

Scikit-learn integrates machine learning algorithms in the tightly-knit scientific Python world, building upon numpy, scipy, and matplotlib. As a machine-learning module, it provides versatile tools for data mining and analysis in any field of science and engineering. It strives to be simple and efficient, accessible to everybody, and reusable in various contexts.

Supervised learning Support vector machines, linear models, naive Bayes, Gaussian processes...

Unsupervised learning Clustering, Gaussian mixture models, manifold learning, matrix factorization, covariance...

And much more Model selection, datasets, feature extraction... See below.

License: Open source, commercially usable: BSD license (3 clause)

Documentation for scikit-learn version 0.13.1. For other versions and printable format, see Documentation resources.

User Guide

• 1. Installing scikit-learn
  • 1.1. Installing an official release
    • 1.1.1. Getting the dependencies
      • 1.1.1.1. Easy install
      • 1.1.1.2. From source package
    • 1.1.2. Windows installer
    • 1.1.3. Building on windows
  • 1.2. Third party distributions of scikit-learn
    • 1.2.1. Debian and derivatives (Ubuntu)
    • 1.2.2. Python(x, y)
    • 1.2.3. Enthought Python distribution
    • 1.2.4. Macports
    • 1.2.5. Archlinux
    • 1.2.6. NetBSD
  • 1.3. Bleeding Edge
  • 1.4. Testing
• 2. Tutorials: From the bottom up with scikit-learn
  • 1. An introduction to machine learning with scikit-learn
    • 1.1. Machine learning: the problem setting
    • 1.2. Loading an example dataset
    • 1.3. Learning and predicting
    • 1.4. Model persistence
  • 2.2. A tutorial on statistical-learning for scientific data processing
    • 2.2.1. Statistical learning: the setting and the estimator object in the scikit-learn
      • 2.2.1.1. Datasets
      • 2.2.1.2. Estimators objects
    • 2.2.2. Supervised learning: predicting an output variable from high-dimensional observations
      • 2.2.2.1. Nearest neighbor and the curse of dimensionality
        • 2.2.2.1.1. k-Nearest neighbors classifier
        • 2.2.2.1.2. The curse of dimensionality
      • 2.2.2.2. Linear model: from regression to sparsity
        • 2.2.2.2.1. Linear regression
        • 2.2.2.2.2. Shrinkage
        • 2.2.2.2.3. Sparsity
        • 2.2.2.2.4. Classification
      • 2.2.2.3. Support vector machines (SVMs)
        • 2.2.2.3.1. Linear SVMs
        • 2.2.2.3.2. Using kernels
    • 2.2.3. Model selection: choosing estimators and their parameters
      • 2.2.3.1. Score, and cross-validated scores
      • 2.2.3.2. Cross-validation generators
      • 2.2.3.3. Grid-search and cross-validated estimators
        • 2.2.3.3.1. Grid-search
        • 2.2.3.3.2. Cross-validated estimators
    • 2.2.4. Unsupervised learning: seeking representations of the data
      • 2.2.4.1. Clustering: grouping observations together
        • 2.2.4.1.1. K-means clustering
        • 2.2.4.1.2. Hierarchical agglomerative clustering: Ward
          • 2.2.4.1.2.1. Connectivity-constrained clustering
          • 2.2.4.1.2.2. Feature agglomeration
      • 2.2.4.2. Decompositions: from a signal to components and loadings
        • 2.2.4.2.1. Principal component analysis: PCA
        • 2.2.4.2.2. Independent Component Analysis: ICA
    • 2.2.5. Putting it all together
      • 2.2.5.1. Pipelining
      • 2.2.5.2. Face recognition with eigenfaces
      • 2.2.5.3. Open problem: Stock Market Structure
    • 2.2.6. Finding help
      • 2.2.6.1. The project mailing list
      • 2.2.6.2. Q&A communities with Machine Learning practitioners
• 3. Supervised learning
  • 3.1. Generalized Linear Models
    • 3.1.1. Ordinary Least Squares
      • 3.1.1.1. Ordinary Least Squares Complexity
    • 3.1.2. Ridge Regression
      • 3.1.2.1. Ridge Complexity
      • 3.1.2.2. Setting the regularization parameter: generalized Cross-Validation
    • 3.1.3. Lasso
      • 3.1.3.1. Setting regularization parameter
        • 3.1.3.1.1. Using cross-validation
        • 3.1.3.1.2. Information-criteria based model selection
    • 3.1.4. Elastic Net
    • 3.1.5. Multi-task Lasso
    • 3.1.6. Least Angle Regression
    • 3.1.7. LARS Lasso
      • 3.1.7.1. Mathematical formulation
    • 3.1.8. Orthogonal Matching Pursuit (OMP)
    • 3.1.9. Bayesian Regression
      • 3.1.9.1. Bayesian Ridge Regression
      • 3.1.9.2. Automatic Relevance Determination - ARD
    • 3.1.10. Logistic regression
    • 3.1.11. Stochastic Gradient Descent - SGD
    • 3.1.12. Perceptron
    • 3.1.13. Passive Aggressive Algorithms
  • 3.2. Support Vector Machines
    • 3.2.1. Classification
      • 3.2.1.1. Multi-class classification
      • 3.2.1.2. Unbalanced problems
    • 3.2.2. Regression
    • 3.2.3. Density estimation, novelty detection
    • 3.2.4. Complexity
    • 3.2.5. Tips on Practical Use
    • 3.2.6. Kernel functions
      • 3.2.6.1. Custom Kernels
        • 3.2.6.1.1. Using Python functions as kernels
        • 3.2.6.1.2. Using the Gram matrix
        • 3.2.6.1.3. Parameters of the RBF Kernel
    • 3.2.7. Mathematical formulation
      • 3.2.7.1. SVC
      • 3.2.7.2. NuSVC
    • 3.2.8. Implementation details
  • 3.3. Stochastic Gradient Descent
    • 3.3.1. Classification
    • 3.3.2. Regression
    • 3.3.3. Stochastic Gradient Descent for sparse data
    • 3.3.4. Complexity
    • 3.3.5. Tips on Practical Use
    • 3.3.6. Mathematical formulation
      • 3.3.6.1. SGD
    • 3.3.7. Implementation details
  • 3.4. Nearest Neighbors
    • 3.4.1. Unsupervised Nearest Neighbors
    • 3.4.2. Nearest Neighbors Classification
    • 3.4.3. Nearest Neighbors Regression
    • 3.4.4. Nearest Neighbor Algorithms
      • 3.4.4.1. Brute Force
      • 3.4.4.2. K-D Tree
      • 3.4.4.3. Ball Tree
      • 3.4.4.4. Choice of Nearest Neighbors Algorithm
      • 3.4.4.5. Effect of leaf_size
    • 3.4.5. Nearest Centroid Classifier
      • 3.4.5.1. Nearest Shrunken Centroid
  • 3.5. Gaussian Processes
    • 3.5.1. Examples
      • 3.5.1.1. An introductory regression example
      • 3.5.1.2. Fitting Noisy Data
    • 3.5.2. Mathematical formulation
      • 3.5.2.1. The initial assumption
      • 3.5.2.2. The best linear unbiased prediction (BLUP)
      • 3.5.2.3. The empirical best linear unbiased predictor (EBLUP)
    • 3.5.3. Correlation Models
    • 3.5.4. Regression Models
    • 3.5.5. Implementation details
  • 3.6. Partial Least Squares
  • 3.7. Naive Bayes
    • 3.7.1. Gaussian Naive Bayes
    • 3.7.2. Multinomial Naive Bayes
    • 3.7.3. Bernoulli Naive Bayes
  • 3.8. Decision Trees
    • 3.8.1. Classification
    • 3.8.2. Regression
    • 3.8.3. Multi-output problems
    • 3.8.4. Complexity
    • 3.8.5. Tips on practical use
    • 3.8.6. Tree algorithms: ID3, C4.5, C5.0 and CART
    • 3.8.7. Mathematical formulation
      • 3.8.7.1. Classification criteria
      • 3.8.7.2. Regression criteria
  • 3.9. Ensemble methods
    • 3.9.1. Forests of randomized trees
      • 3.9.1.1. Random Forests
      • 3.9.1.2. Extremely Randomized Trees
      • 3.9.1.3. Parameters
      • 3.9.1.4. Parallelization
      • 3.9.1.5. Feature importance evaluation
      • 3.9.1.6. Totally Random Trees Embedding
    • 3.9.2. Gradient Tree Boosting
      • 3.9.2.1. Classification
      • 3.9.2.2. Regression
      • 3.9.2.3. Mathematical formulation
        • 3.9.2.3.1. Loss Functions
      • 3.9.2.4. Regularization
        • 3.9.2.4.1. Shrinkage
        • 3.9.2.4.2. Subsampling
      • 3.9.2.5. Interpretation
        • 3.9.2.5.1. Feature importance
        • 3.9.2.5.2. Partial dependence
  • 3.10. Multiclass and multilabel algorithms
    • 3.10.1. One-Vs-The-Rest
      • 3.10.1.1. Multilabel learning with OvR
    • 3.10.2. One-Vs-One
    • 3.10.3. Error-Correcting Output-Codes
  • 3.11. Feature selection
    • 3.11.1. Univariate feature selection
    • 3.11.2. Recursive feature elimination
    • 3.11.3. L1-based feature selection
      • 3.11.3.1. Selecting non-zero coefficients
      • 3.11.3.2. Randomized sparse models
    • 3.11.4. Tree-based feature selection
  • 3.12. Semi-Supervised
    • 3.12.1. Label Propagation
  • 3.13. Linear and Quadratic Discriminant Analysis
    • 3.13.1. Dimensionality Reduction using LDA
    • 3.13.2. Mathematical Idea
  • 3.14. Isotonic regression
• 4. Unsupervised learning
  • 4.1. Gaussian mixture models
    • 4.1.1. GMM classifier
      • 4.1.1.1. Pros and cons of class GMM: expectation-maximization inference
        • 4.1.1.1.1. Pros
        • 4.1.1.1.2. Cons
      • 4.1.1.2. Selecting the number of components in a classical GMM
      • 4.1.1.3. Estimation algorithm Expectation-maximization
    • 4.1.2. VBGMM classifier: variational Gaussian mixtures
      • 4.1.2.1. Pros and cons of class VBGMM: variational inference
        • 4.1.2.1.1. Pros
        • 4.1.2.1.2. Cons
      • 4.1.2.2. Estimation algorithm: variational inference
    • 4.1.3. DPGMM classifier: Infinite Gaussian mixtures
      • 4.1.3.1. Pros and cons of class DPGMM: Diriclet process mixture model
        • 4.1.3.1.1. Pros
        • 4.1.3.1.2. Cons
      • 4.1.3.2. The Dirichlet Process
  • 4.2. Manifold learning
    • 4.2.1. Introduction
    • 4.2.2. Isomap
      • 4.2.2.1. Complexity
    • 4.2.3. Locally Linear Embedding
      • 4.2.3.1. Complexity
    • 4.2.4. Modified Locally Linear Embedding
      • 4.2.4.1. Complexity
    • 4.2.5. Hessian Eigenmapping
      • 4.2.5.1. Complexity
    • 4.2.6. Spectral Embedding
      • 4.2.6.1. Complexity
    • 4.2.7. Local Tangent Space Alignment
      • 4.2.7.1. Complexity
    • 4.2.8. Multi-dimensional Scaling (MDS)
      • 4.2.8.1. Metric MDS
      • 4.2.8.2. Nonmetric MDS
    • 4.2.9. Tips on practical use
  • 4.3. Clustering
    • 4.3.1. Overview of clustering methods
    • 4.3.2. K-means
      • 4.3.2.1. Mini Batch K-Means
    • 4.3.3. Affinity Propagation
    • 4.3.4. Mean Shift
    • 4.3.5. Spectral clustering
      • 4.3.5.1. Different label assignement strategies
    • 4.3.6. Hierarchical clustering
      • 4.3.6.1. Adding connectivity constraints
    • 4.3.7. DBSCAN
    • 4.3.8. Clustering performance evaluation
      • 4.3.8.1. Inertia
        • 4.3.8.1.1. Presentation and usage
        • 4.3.8.1.2. Advantages
        • 4.3.8.1.3. Drawbacks
      • 4.3.8.2. Adjusted Rand index
        • 4.3.8.2.1. Presentation and usage
        • 4.3.8.2.2. Advantages
        • 4.3.8.2.3. Drawbacks
        • 4.3.8.2.4. Mathematical formulation
      • 4.3.8.3. Mutual Information based scores
        • 4.3.8.3.1. Presentation and usage
        • 4.3.8.3.2. Advantages
        • 4.3.8.3.3. Drawbacks
        • 4.3.8.3.4. Mathematical formulation
      • 4.3.8.4. Homogeneity, completeness and V-measure
        • 4.3.8.4.1. Presentation and usage
        • 4.3.8.4.2. Advantages
        • 4.3.8.4.3. Drawbacks
        • 4.3.8.4.4. Mathematical formulation
      • 4.3.8.5. Silhouette Coefficient
        • 4.3.8.5.1. Presentation and usage
        • 4.3.8.5.2. Advantages
        • 4.3.8.5.3. Drawbacks
  • 4.4. Decomposing signals in components (matrix factorization problems)
    • 4.4.1. Principal component analysis (PCA)
      • 4.4.1.1. Exact PCA and probabilistic interpretation
      • 4.4.1.2. Approximate PCA
      • 4.4.1.3. Kernel PCA
      • 4.4.1.4. Sparse Principal Components Analysis (SparsePCA and MiniBatchSparsePCA)
    • 4.4.2. Dictionary Learning
      • 4.4.2.1. Sparse coding with a precomputed dictionary
      • 4.4.2.2. Generic dictionary learning
      • 4.4.2.3. Mini-batch dictionary learning
    • 4.4.3. Factor Analysis
    • 4.4.4. Independent component analysis (ICA)
    • 4.4.5. Non-negative matrix factorization (NMF or NNMF)
  • 4.5. Covariance estimation
    • 4.5.1. Empirical covariance
    • 4.5.2. Shrunk Covariance
      • 4.5.2.1. Basic shrinkage
      • 4.5.2.2. Ledoit-Wolf shrinkage
      • 4.5.2.3. Oracle Approximating Shrinkage
    • 4.5.3. Sparse inverse covariance
    • 4.5.4. Robust Covariance Estimation
      • 4.5.4.1. Minimum Covariance Determinant
  • 4.6. Novelty and Outlier Detection
    • 4.6.1. Novelty Detection
    • 4.6.2. Outlier Detection
      • 4.6.2.1. Fitting an elliptic envelop
      • 4.6.2.2. One-class SVM versus elliptic envelop
  • 4.7. Hidden Markov Models
    • 4.7.1. Using HMM
      • 4.7.1.1. Building HMM and generating samples
      • 4.7.1.2. Training HMM parameters and infering the hidden states
      • 4.7.1.3. Implementing HMMs with custom emission probabilities
• 5. Model selection and evaluation
  • 5.1. Cross-Validation: evaluating estimator performance
    • 5.1.1. Computing cross-validated metrics
    • 5.1.2. Cross validation iterators
      • 5.1.2.1. K-fold
      • 5.1.2.2. Stratified K-Fold
      • 5.1.2.3. Leave-One-Out - LOO
      • 5.1.2.4. Leave-P-Out - LPO
      • 5.1.2.5. Leave-One-Label-Out - LOLO
      • 5.1.2.6. Leave-P-Label-Out
      • 5.1.2.7. Random permutations cross-validation a.k.a. Shuffle & Split
      • 5.1.2.8. See also
      • 5.1.2.9. Bootstrapping cross-validation
    • 5.1.3. Cross validation and model selection
  • 5.2. Grid Search: setting estimator parameters
    • 5.2.1. GridSearchCV
    • 5.2.2. Examples
    • 5.2.3. Alternatives to brute force grid search
      • 5.2.3.1. Model specific cross-validation
        • 8.17.25. sklearn.linear_model.RidgeCV
        • 8.17.24. sklearn.linear_model.RidgeClassifierCV
        • 8.17.6. sklearn.linear_model.LarsCV
        • 8.17.10. sklearn.linear_model.LassoLarsCV
        • 8.17.8. sklearn.linear_model.LassoCV
        • 8.17.4. sklearn.linear_model.ElasticNetCV
      • 5.2.3.2. Information Criterion
        • 8.17.11. sklearn.linear_model.LassoLarsIC
      • 5.2.3.3. Out of Bag Estimates
        • 8.7.1. sklearn.ensemble.RandomForestClassifier
        • 8.7.3. sklearn.ensemble.RandomForestRegressor
        • 8.7.4. sklearn.ensemble.ExtraTreesClassifier
        • 8.7.5. sklearn.ensemble.ExtraTreesRegressor
        • 8.7.6. sklearn.ensemble.GradientBoostingClassifier
        • 8.7.7. sklearn.ensemble.GradientBoostingRegressor
  • 5.3. Pipeline: chaining estimators
    • 5.3.1. Usage
    • 5.3.2. Notes
  • 5.4. FeatureUnion: Combining feature extractors
    • 5.4.1. Usage
  • 5.5. Model evaluation
    • 5.5.1. Classification metrics
      • 5.5.1.1. Accuracy score
      • 5.5.1.2. Area under the curve (AUC)
      • 5.5.1.3. Average precision score
      • 5.5.1.4. Confusion matrix
      • 5.5.1.5. Classification report
      • 5.5.1.6. Precision, recall and F-measures
        • 5.5.1.6.1. Binary classification
        • 5.5.1.6.2. Multiclass and multilabels classification
      • 5.5.1.7. Hinge loss
      • 5.5.1.8. Matthews correlation coefficient
      • 5.5.1.9. Receiver operating characteristic (ROC)
      • 5.5.1.10. Zero one loss
    • 5.5.2. Regression metrics
      • 5.5.2.1. Explained variance score
      • 5.5.2.2. Mean absolute error
      • 5.5.2.3. Mean squared error
      • 5.5.2.4. R² score, the coefficient of determination
    • 5.5.3. Clustering metrics
    • 5.5.4. Dummy estimators
• 6. Dataset transformations
  • 6.1. Preprocessing data
    • 6.1.1. Standardization or Mean Removal and Variance Scaling
      • 6.1.1.1. Scaling Features to a Range
    • 6.1.2. Normalization
    • 6.1.3. Binarization
      • 6.1.3.1. Feature binarization
    • 6.1.4. Encoding categorical features
    • 6.1.5. Label preprocessing
      • 6.1.5.1. Label binarization
      • 6.1.5.2. Label encoding
  • 6.2. Feature extraction
    • 6.2.1. Loading features from dicts
    • 6.2.2. Feature hashing
      • 6.2.2.1. Implementation details
    • 6.2.3. Text feature extraction
      • 6.2.3.1. The Bag of Words representation
      • 6.2.3.2. Sparsity
      • 6.2.3.3. Common Vectorizer usage
      • 6.2.3.4. Tf–idf term weighting
      • 6.2.3.5. Applications and examples
      • 6.2.3.6. Limitations of the Bag of Words representation
      • 6.2.3.7. Vectorizing a large text corpus with the hashing trick
      • 6.2.3.8. Customizing the vectorizer classes
    • 6.2.4. Image feature extraction
      • 6.2.4.1. Patch extraction
      • 6.2.4.2. Connectivity graph of an image
  • 6.3. Kernel Approximation
    • 6.3.1. Nystroem Method for Kernel Approximation
    • 6.3.2. Radial Basis Function Kernel
    • 6.3.3. Additive Chi Squared Kernel
    • 6.3.4. Skewed Chi Squared Kernel
    • 6.3.5. Mathematical Details
  • 6.4. Random Projection
    • 6.4.1. The Johnson-Lindenstrauss lemma
    • 6.4.2. Gaussian random projection
    • 6.4.3. Sparse random projection
  • 6.5. Pairwise metrics, Affinities and Kernels
    • 6.5.1. Cosine similarity
    • 6.5.2. Chi Squared Kernel
• 7. Dataset loading utilities
  • 7.1. General dataset API
  • 7.2. Toy datasets
  • 7.3. Sample images
  • 7.4. Sample generators
  • 7.5. Datasets in svmlight / libsvm format
  • 7.6. The Olivetti faces dataset
  • 7.7. The 20 newsgroups text dataset
    • 7.7.1. Usage
  • 7.8. Downloading datasets from the mldata.org repository
  • 7.9. The Labeled Faces in the Wild face recognition dataset
    • 7.9.1. Usage
    • 7.9.2. Examples
  • 7.10. Forest covertypes
• 8. Reference
  • 8.1. sklearn.cluster: Clustering
    • 8.1.1. Classes
      • 8.1.1.1. sklearn.cluster.AffinityPropagation
      • 8.1.1.2. sklearn.cluster.DBSCAN
      • 8.1.1.3. sklearn.cluster.KMeans
      • 8.1.1.4. sklearn.cluster.MiniBatchKMeans
      • 8.1.1.5. sklearn.cluster.MeanShift
      • 8.1.1.6. sklearn.cluster.SpectralClustering
      • 8.1.1.7. sklearn.cluster.Ward
    • 8.1.2. Functions
      • 8.1.2.1. sklearn.cluster.estimate_bandwidth
      • 8.1.2.2. sklearn.cluster.k_means
      • 8.1.2.3. sklearn.cluster.ward_tree
      • 8.1.2.4. sklearn.cluster.affinity_propagation
      • 8.1.2.5. sklearn.cluster.dbscan
      • 8.1.2.6. sklearn.cluster.mean_shift
      • 8.1.2.7. sklearn.cluster.spectral_clustering
  • 8.2. sklearn.covariance: Covariance Estimators
    • 8.2.1. sklearn.covariance.EmpiricalCovariance
    • 8.2.2. sklearn.covariance.EllipticEnvelope
    • 8.2.3. sklearn.covariance.GraphLasso
    • 8.2.4. sklearn.covariance.GraphLassoCV
    • 8.2.5. sklearn.covariance.LedoitWolf
    • 8.2.6. sklearn.covariance.MinCovDet
    • 8.2.7. sklearn.covariance.OAS
    • 8.2.8. sklearn.covariance.ShrunkCovariance
    • 8.2.9. sklearn.covariance.empirical_covariance
    • 8.2.10. sklearn.covariance.ledoit_wolf
    • 8.2.11. sklearn.covariance.shrunk_covariance
    • 8.2.12. sklearn.covariance.oas
    • 8.2.13. sklearn.covariance.graph_lasso
  • 8.3. sklearn.cross_validation: Cross Validation
    • 8.3.1. sklearn.cross_validation.Bootstrap
    • 8.3.2. sklearn.cross_validation.KFold
    • 8.3.3. sklearn.cross_validation.LeaveOneLabelOut
    • 8.3.4. sklearn.cross_validation.LeaveOneOut
    • 8.3.5. sklearn.cross_validation.LeavePLabelOut
    • 8.3.6. sklearn.cross_validation.LeavePOut
    • 8.3.7. sklearn.cross_validation.StratifiedKFold
    • 8.3.8. sklearn.cross_validation.ShuffleSplit
    • 8.3.9. sklearn.cross_validation.StratifiedShuffleSplit
    • 8.3.10. sklearn.cross_validation.train_test_split
    • 8.3.11. sklearn.cross_validation.cross_val_score
    • 8.3.12. sklearn.cross_validation.permutation_test_score
    • 8.3.13. sklearn.cross_validation.check_cv
  • 8.4. sklearn.datasets: Datasets
    • 8.4.1. Loaders
      • 8.4.1.1. sklearn.datasets.fetch_20newsgroups
      • 8.4.1.2. sklearn.datasets.fetch_20newsgroups_vectorized
      • 8.4.1.3. sklearn.datasets.load_boston
      • 8.4.1.4. sklearn.datasets.load_diabetes
      • 8.4.1.5. sklearn.datasets.load_digits
      • 8.4.1.6. sklearn.datasets.load_files
      • 8.4.1.7. sklearn.datasets.load_iris
      • 8.4.1.8. sklearn.datasets.load_lfw_pairs
      • 8.4.1.9. sklearn.datasets.fetch_lfw_pairs
      • 8.4.1.10. sklearn.datasets.load_lfw_people
      • 8.4.1.11. sklearn.datasets.fetch_lfw_people
      • 8.4.1.12. sklearn.datasets.load_linnerud
      • 8.4.1.13. sklearn.datasets.fetch_mldata
      • 8.4.1.14. sklearn.datasets.fetch_olivetti_faces
      • 8.4.1.15. sklearn.datasets.fetch_california_housing
      • 8.4.1.16. sklearn.datasets.load_sample_image
      • 8.4.1.17. sklearn.datasets.load_sample_images
      • 8.4.1.18. sklearn.datasets.load_svmlight_file
      • 8.4.1.19. sklearn.datasets.dump_svmlight_file
    • 8.4.2. Samples generator
      • 8.4.2.1. sklearn.datasets.make_blobs
      • 8.4.2.2. sklearn.datasets.make_classification
      • 8.4.2.3. sklearn.datasets.make_circles
      • 8.4.2.4. sklearn.datasets.make_friedman1
      • 8.4.2.5. sklearn.datasets.make_friedman2
      • 8.4.2.6. sklearn.datasets.make_friedman3
      • 8.4.2.7. sklearn.datasets.make_hastie_10_2
      • 8.4.2.8. sklearn.datasets.make_low_rank_matrix
      • 8.4.2.9. sklearn.datasets.make_moons
      • 8.4.2.10. sklearn.datasets.make_multilabel_classification
      • 8.4.2.11. sklearn.datasets.make_regression
      • 8.4.2.12. sklearn.datasets.make_s_curve
      • 8.4.2.13. sklearn.datasets.make_sparse_coded_signal
      • 8.4.2.14. sklearn.datasets.make_sparse_spd_matrix
      • 8.4.2.15. sklearn.datasets.make_sparse_uncorrelated
      • 8.4.2.16. sklearn.datasets.make_spd_matrix
      • 8.4.2.17. sklearn.datasets.make_swiss_roll
  • 8.5. sklearn.decomposition: Matrix Decomposition
    • 8.5.1. sklearn.decomposition.PCA
    • 8.5.2. sklearn.decomposition.ProbabilisticPCA
    • 8.5.3. sklearn.decomposition.ProjectedGradientNMF
    • 8.5.4. sklearn.decomposition.RandomizedPCA
    • 8.5.5. sklearn.decomposition.KernelPCA
    • 8.5.6. sklearn.decomposition.FactorAnalysis
    • 8.5.7. sklearn.decomposition.FastICA
    • 8.5.8. sklearn.decomposition.NMF
    • 8.5.9. sklearn.decomposition.SparsePCA
    • 8.5.10. sklearn.decomposition.MiniBatchSparsePCA
    • 8.5.11. sklearn.decomposition.SparseCoder
    • 8.5.12. sklearn.decomposition.DictionaryLearning
    • 8.5.13. sklearn.decomposition.MiniBatchDictionaryLearning
    • 8.5.14. sklearn.decomposition.fastica
    • 8.5.15. sklearn.decomposition.dict_learning
    • 8.5.16. sklearn.decomposition.dict_learning_online
    • 8.5.17. sklearn.decomposition.sparse_encode
  • 8.6. sklearn.dummy: Dummy estimators
    • 8.6.1. sklearn.dummy.DummyClassifier
    • 8.6.2. sklearn.dummy.DummyRegressor
  • 8.7. sklearn.ensemble: Ensemble Methods
    • 8.7.1. sklearn.ensemble.RandomForestClassifier
    • 8.7.2. sklearn.ensemble.RandomTreesEmbedding
    • 8.7.3. sklearn.ensemble.RandomForestRegressor
    • 8.7.4. sklearn.ensemble.ExtraTreesClassifier
    • 8.7.5. sklearn.ensemble.ExtraTreesRegressor
    • 8.7.6. sklearn.ensemble.GradientBoostingClassifier
    • 8.7.7. sklearn.ensemble.GradientBoostingRegressor
    • 8.7.8. partial dependence
      • 8.7.8.1. sklearn.ensemble.partial_dependence.partial_dependence
      • 8.7.8.2. sklearn.ensemble.partial_dependence.plot_partial_dependence
  • 8.8. sklearn.feature_extraction: Feature Extraction
    • 8.8.1. sklearn.feature_extraction.DictVectorizer
    • 8.8.2. sklearn.feature_extraction.FeatureHasher
    • 8.8.3. From images
      • 8.8.3.1. sklearn.feature_extraction.image.img_to_graph
      • 8.8.3.2. sklearn.feature_extraction.image.grid_to_graph
      • 8.8.3.3. sklearn.feature_extraction.image.extract_patches_2d
      • 8.8.3.4. sklearn.feature_extraction.image.reconstruct_from_patches_2d
      • 8.8.3.5. sklearn.feature_extraction.image.PatchExtractor
    • 8.8.4. From text
      • 8.8.4.1. sklearn.feature_extraction.text.CountVectorizer
      • 8.8.4.2. sklearn.feature_extraction.text.HashingVectorizer
      • 8.8.4.3. sklearn.feature_extraction.text.TfidfTransformer
      • 8.8.4.4. sklearn.feature_extraction.text.TfidfVectorizer
  • 8.9. sklearn.feature_selection: Feature Selection
    • 8.9.1. sklearn.feature_selection.SelectPercentile
    • 8.9.2. sklearn.feature_selection.SelectKBest
    • 8.9.3. sklearn.feature_selection.SelectFpr
    • 8.9.4. sklearn.feature_selection.SelectFdr
    • 8.9.5. sklearn.feature_selection.SelectFwe
    • 8.9.6. sklearn.feature_selection.RFE
    • 8.9.7. sklearn.feature_selection.RFECV
    • 8.9.8. sklearn.feature_selection.chi2
    • 8.9.9. sklearn.feature_selection.f_classif
    • 8.9.10. sklearn.feature_selection.f_regression
  • 8.10. sklearn.gaussian_process: Gaussian Processes
    • 8.10.1. sklearn.gaussian_process.GaussianProcess
    • 8.10.2. sklearn.gaussian_process.correlation_models.absolute_exponential
    • 8.10.3. sklearn.gaussian_process.correlation_models.squared_exponential
    • 8.10.4. sklearn.gaussian_process.correlation_models.generalized_exponential
    • 8.10.5. sklearn.gaussian_process.correlation_models.pure_nugget
    • 8.10.6. sklearn.gaussian_process.correlation_models.cubic
    • 8.10.7. sklearn.gaussian_process.correlation_models.linear
    • 8.10.8. sklearn.gaussian_process.regression_models.constant
    • 8.10.9. sklearn.gaussian_process.regression_models.linear
    • 8.10.10. sklearn.gaussian_process.regression_models.quadratic
  • 8.11. sklearn.grid_search: Grid Search
    • 8.11.1. sklearn.grid_search.GridSearchCV
    • 8.11.2. sklearn.grid_search.IterGrid
  • 8.12. sklearn.hmm: Hidden Markov Models
    • 8.12.1. sklearn.hmm.GaussianHMM
    • 8.12.2. sklearn.hmm.MultinomialHMM
    • 8.12.3. sklearn.hmm.GMMHMM
  • 8.13. sklearn.isotonic: Isotonic regression
    • 8.13.1. sklearn.isotonic.IsotonicRegression
    • 8.13.2. sklearn.isotonic.isotonic_regression
  • 8.14. sklearn.kernel_approximation Kernel Approximation
    • 8.14.1. sklearn.kernel_approximation.AdditiveChi2Sampler
    • 8.14.2. sklearn.kernel_approximation.Nystroem
    • 8.14.3. sklearn.kernel_approximation.RBFSampler
    • 8.14.4. sklearn.kernel_approximation.SkewedChi2Sampler
  • 8.15. sklearn.semi_supervised Semi-Supervised Learning
    • 8.15.1. sklearn.semi_supervised.LabelPropagation
    • 8.15.2. sklearn.semi_supervised.LabelSpreading
  • 8.16. sklearn.lda: Linear Discriminant Analysis
    • 8.16.1. sklearn.lda.LDA
  • 8.17. sklearn.linear_model: Generalized Linear Models
    • 8.17.1. sklearn.linear_model.ARDRegression
    • 8.17.2. sklearn.linear_model.BayesianRidge
    • 8.17.3. sklearn.linear_model.ElasticNet
    • 8.17.4. sklearn.linear_model.ElasticNetCV
    • 8.17.5. sklearn.linear_model.Lars
    • 8.17.6. sklearn.linear_model.LarsCV
    • 8.17.7. sklearn.linear_model.Lasso
    • 8.17.8. sklearn.linear_model.LassoCV
    • 8.17.9. sklearn.linear_model.LassoLars
    • 8.17.10. sklearn.linear_model.LassoLarsCV
    • 8.17.11. sklearn.linear_model.LassoLarsIC
    • 8.17.12. sklearn.linear_model.LinearRegression
    • 8.17.13. sklearn.linear_model.LogisticRegression
    • 8.17.14. sklearn.linear_model.MultiTaskLasso
    • 8.17.15. sklearn.linear_model.MultiTaskElasticNet
    • 8.17.16. sklearn.linear_model.OrthogonalMatchingPursuit
    • 8.17.17. sklearn.linear_model.PassiveAggressiveClassifier
    • 8.17.18. sklearn.linear_model.PassiveAggressiveRegressor
    • 8.17.19. sklearn.linear_model.Perceptron
    • 8.17.20. sklearn.linear_model.RandomizedLasso
    • 8.17.21. sklearn.linear_model.RandomizedLogisticRegression
    • 8.17.22. sklearn.linear_model.Ridge
    • 8.17.23. sklearn.linear_model.RidgeClassifier
    • 8.17.24. sklearn.linear_model.RidgeClassifierCV
    • 8.17.25. sklearn.linear_model.RidgeCV
    • 8.17.26. sklearn.linear_model.SGDClassifier
    • 8.17.27. sklearn.linear_model.SGDRegressor
    • 8.17.28. sklearn.linear_model.lars_path
    • 8.17.29. sklearn.linear_model.lasso_path
    • 8.17.30. sklearn.linear_model.lasso_stability_path
    • 8.17.31. sklearn.linear_model.orthogonal_mp
    • 8.17.32. sklearn.linear_model.orthogonal_mp_gram
  • 8.18. sklearn.manifold: Manifold Learning
    • 8.18.1. sklearn.manifold.LocallyLinearEmbedding
    • 8.18.2. sklearn.manifold.Isomap
    • 8.18.3. sklearn.manifold.MDS
    • 8.18.4. sklearn.manifold.SpectralEmbedding
    • 8.18.5. sklearn.manifold.locally_linear_embedding
    • 8.18.6. sklearn.manifold.spectral_embedding
  • 8.19. sklearn.metrics: Metrics
    • 8.19.1. Classification metrics
      • 8.19.1.1. sklearn.metrics.accuracy_score
      • 8.19.1.2. sklearn.metrics.auc
      • 8.19.1.3. sklearn.metrics.auc_score
      • 8.19.1.4. sklearn.metrics.average_precision_score
      • 8.19.1.5. sklearn.metrics.classification_report
      • 8.19.1.6. sklearn.metrics.confusion_matrix
      • 8.19.1.7. sklearn.metrics.f1_score
      • 8.19.1.8. sklearn.metrics.fbeta_score
      • 8.19.1.9. sklearn.metrics.hinge_loss
      • 8.19.1.10. sklearn.metrics.matthews_corrcoef
      • 8.19.1.11. sklearn.metrics.precision_recall_curve
      • 8.19.1.12. sklearn.metrics.precision_recall_fscore_support
      • 8.19.1.13. sklearn.metrics.precision_score
      • 8.19.1.14. sklearn.metrics.recall_score
      • 8.19.1.15. sklearn.metrics.roc_curve
      • 8.19.1.16. sklearn.metrics.zero_one_loss
    • 8.19.2. Regression metrics
      • 8.19.2.1. sklearn.metrics.explained_variance_score
      • 8.19.2.2. sklearn.metrics.mean_absolute_error
      • 8.19.2.3. sklearn.metrics.mean_squared_error
      • 8.19.2.4. sklearn.metrics.r2_score
    • 8.19.3. Clustering metrics
      • 8.19.3.1. sklearn.metrics.adjusted_mutual_info_score
      • 8.19.3.2. sklearn.metrics.adjusted_rand_score
      • 8.19.3.3. sklearn.metrics.completeness_score
      • 8.19.3.4. sklearn.metrics.homogeneity_completeness_v_measure
      • 8.19.3.5. sklearn.metrics.homogeneity_score
      • 8.19.3.6. sklearn.metrics.mutual_info_score
      • 8.19.3.7. sklearn.metrics.normalized_mutual_info_score
      • 8.19.3.8. sklearn.metrics.silhouette_score
      • 8.19.3.9. sklearn.metrics.silhouette_samples
      • 8.19.3.10. sklearn.metrics.v_measure_score
    • 8.19.4. Pairwise metrics
      • 8.19.4.1. sklearn.metrics.pairwise.additive_chi2_kernel
      • 8.19.4.2. sklearn.metrics.pairwise.chi2_kernel
      • 8.19.4.3. sklearn.metrics.pairwise.distance_metrics
      • 8.19.4.4. sklearn.metrics.pairwise.euclidean_distances
      • 8.19.4.5. sklearn.metrics.pairwise.kernel_metrics
      • 8.19.4.6. sklearn.metrics.pairwise.linear_kernel
      • 8.19.4.7. sklearn.metrics.pairwise.manhattan_distances
      • 8.19.4.8. sklearn.metrics.pairwise.pairwise_distances
      • 8.19.4.9. sklearn.metrics.pairwise.pairwise_kernels
      • 8.19.4.10. sklearn.metrics.pairwise.polynomial_kernel
      • 8.19.4.11. sklearn.metrics.pairwise.rbf_kernel
  • 8.20. sklearn.mixture: Gaussian Mixture Models
    • 8.20.1. sklearn.mixture.GMM
    • 8.20.2. sklearn.mixture.DPGMM
    • 8.20.3. sklearn.mixture.VBGMM
  • 8.21. sklearn.multiclass: Multiclass and multilabel classification
    • 8.21.1. Multiclass and multilabel classification strategies
    • 8.21.2. sklearn.multiclass.OneVsRestClassifier
    • 8.21.3. sklearn.multiclass.OneVsOneClassifier
    • 8.21.4. sklearn.multiclass.OutputCodeClassifier
    • 8.21.5. sklearn.multiclass.fit_ovr
    • 8.21.6. sklearn.multiclass.predict_ovr
    • 8.21.7. sklearn.multiclass.fit_ovo
    • 8.21.8. sklearn.multiclass.predict_ovo
    • 8.21.9. sklearn.multiclass.fit_ecoc
    • 8.21.10. sklearn.multiclass.predict_ecoc
  • 8.22. sklearn.naive_bayes: Naive Bayes
    • 8.22.1. sklearn.naive_bayes.GaussianNB
    • 8.22.2. sklearn.naive_bayes.MultinomialNB
    • 8.22.3. sklearn.naive_bayes.BernoulliNB
  • 8.23. sklearn.neighbors: Nearest Neighbors
    • 8.23.1. sklearn.neighbors.NearestNeighbors
    • 8.23.2. sklearn.neighbors.KNeighborsClassifier
    • 8.23.3. sklearn.neighbors.RadiusNeighborsClassifier
    • 8.23.4. sklearn.neighbors.KNeighborsRegressor
    • 8.23.5. sklearn.neighbors.RadiusNeighborsRegressor
    • 8.23.6. sklearn.neighbors.BallTree
    • 8.23.7. sklearn.neighbors.NearestCentroid
    • 8.23.8. sklearn.neighbors.kneighbors_graph
    • 8.23.9. sklearn.neighbors.radius_neighbors_graph
  • 8.24. sklearn.pls: Partial Least Squares
    • 8.24.1. sklearn.pls.PLSRegression
    • 8.24.2. sklearn.pls.PLSCanonical
    • 8.24.3. sklearn.pls.CCA
    • 8.24.4. sklearn.pls.PLSSVD
  • 8.25. sklearn.pipeline: Pipeline
    • 8.25.1. sklearn.pipeline.Pipeline
    • 8.25.2. sklearn.pipeline.FeatureUnion
  • 8.26. sklearn.preprocessing: Preprocessing and Normalization
    • 8.26.1. sklearn.preprocessing.Binarizer
    • 8.26.2. sklearn.preprocessing.KernelCenterer
    • 8.26.3. sklearn.preprocessing.LabelBinarizer
    • 8.26.4. sklearn.preprocessing.LabelEncoder
    • 8.26.5. sklearn.preprocessing.MinMaxScaler
    • 8.26.6. sklearn.preprocessing.Normalizer
    • 8.26.7. sklearn.preprocessing.OneHotEncoder
    • 8.26.8. sklearn.preprocessing.StandardScaler
    • 8.26.9. sklearn.preprocessing.add_dummy_feature
    • 8.26.10. sklearn.preprocessing.balance_weights
    • 8.26.11. sklearn.preprocessing.binarize
    • 8.26.12. sklearn.preprocessing.normalize
    • 8.26.13. sklearn.preprocessing.scale
  • 8.27. sklearn.qda: Quadratic Discriminant Analysis
    • 8.27.1. sklearn.qda.QDA
  • 8.28. sklearn.random_projection: Random projection
    • 8.28.1. sklearn.random_projection.GaussianRandomProjection
    • 8.28.2. sklearn.random_projection.SparseRandomProjection
    • 8.28.3. sklearn.random_projection.johnson_lindenstrauss_min_dim
  • 8.29. sklearn.svm: Support Vector Machines
    • 8.29.1. Estimators
      • 8.29.1.1. sklearn.svm.SVC
      • 8.29.1.2. sklearn.svm.LinearSVC
      • 8.29.1.3. sklearn.svm.NuSVC
      • 8.29.1.4. sklearn.svm.SVR
      • 8.29.1.5. sklearn.svm.NuSVR
      • 8.29.1.6. sklearn.svm.OneClassSVM
      • 8.29.1.7. sklearn.svm.l1_min_c
    • 8.29.2. Low-level methods
      • 8.29.2.1. sklearn.svm.libsvm.fit
      • 8.29.2.2. sklearn.svm.libsvm.decision_function
      • 8.29.2.3. sklearn.svm.libsvm.predict
      • 8.29.2.4. sklearn.svm.libsvm.predict_proba
      • 8.29.2.5. sklearn.svm.libsvm.cross_validation
  • 8.30. sklearn.tree: Decision Trees
    • 8.30.1. sklearn.tree.DecisionTreeClassifier
    • 8.30.2. sklearn.tree.DecisionTreeRegressor
    • 8.30.3. sklearn.tree.ExtraTreeClassifier
    • 8.30.4. sklearn.tree.ExtraTreeRegressor
    • 8.30.5. sklearn.tree.export_graphviz
  • 8.31. sklearn.utils: Utilities
    • 8.31.1. sklearn.utils.check_random_state
    • 8.31.2. sklearn.utils.resample
    • 8.31.3. sklearn.utils.shuffle

Example Gallery

• Examples
  • General examples
  • Examples based on real world datasets
  • Clustering
  • Covariance estimation
  • Dataset examples
  • Decomposition
  • Ensemble methods
  • Tutorial exercises
  • Gaussian Process for Machine Learning
  • Generalized Linear Models
  • Manifold learning
  • Gaussian Mixture Models
  • Nearest Neighbors
  • Semi Supervised Classification
  • Support Vector Machines
  • Decision Trees

Development

• 1. Contributing
  • 1.1. Submitting a bug report
  • 1.2. Retrieving the latest code
  • 1.3. Contributing code
    • 1.3.1. How to contribute
    • 1.3.2. Easy Issues
    • 1.3.3. Documentation
    • 1.3.4. Testing and improving test coverage
    • 1.3.5. Developers web site
    • 1.3.6. Issue Tracker Tags
  • 1.4. Other ways to contribute
  • 1.5. Coding guidelines
    • 1.5.1. Input validation
    • 1.5.2. Random Numbers
  • 1.6. APIs of scikit-learn objects
    • 1.6.1. Different objects
    • 1.6.2. Estimators
      • 1.6.2.1. Instantiation
      • 1.6.2.2. Fitting
      • 1.6.2.3. Estimated Attributes
      • 1.6.2.4. Optional Arguments
    • 1.6.3. Unresolved API issues
    • 1.6.4. Working notes
    • 1.6.5. Specific models
• 2. How to optimize for speed
  • 2.1. Python, Cython or C/C++?
  • 2.2. Profiling Python code
  • 2.3. Memory usage profiling
  • 2.4. Performance tips for the Cython developer
  • 2.5. Profiling compiled extensions
    • 2.5.1. Using yep and google-perftools
    • 2.5.2. Using gprof
    • 2.5.3. Using valgrind / callgrind / kcachegrind
  • 2.6. Multi-core parallelism using joblib.Parallel
  • 2.7. A sample algorithmic trick: warm restarts for cross validation
• 3. Utilities for Developers
  • 3.1. Validation Tools
  • 3.2. Efficient Linear Algebra & Array Operations
  • 3.3. Efficient Random Sampling
  • 3.4. Efficient Routines for Sparse Matrices
  • 3.5. Graph Routines
  • 3.6. Backports
    • 3.6.1. ARPACK
    • 3.6.2. Benchmarking
  • 3.7. Testing Functions
  • 3.8. Helper Functions
  • 3.9. Hash Functions
  • 3.10. Warnings and Exceptions
• 4. Developers’ Tips for Debugging
  • 4.1. Memory errors: debugging Cython with valgrind
• 5. Maintainer / core-developer information
  • 5.1. Making a release
• 6. About us
  • 6.1. History
  • 6.2. People
  • 6.3. Citing scikit-learn
  • 6.4. Funding