Subject description - A0B01PSI

Summary of Study | Summary of Branches | All Subject Groups | All Subjects | List of Roles | Explanatory Notes               Instructions
A0B01PSI Probability, Statistics, and Theory of Information Extent of teaching:4+2
Guarantors:Navara M. Roles:P,V Language of
teaching:
CS
Teachers:Navara M. Completion:Z,ZK
Responsible Department:13101 Credits:6 Semester:Z

Anotation:

Basics of probability theory, mathematical statistics, information theory, and coding. Includes descriptions of probability, random variables and their distributions, characteristics and operations with random variables. Basics of mathematical statistics: Point and interval estimates, methods of parameters estimation and hypotheses testing, least squares method. Basic notions and results of the theory of Markov chains. Shannon entropy, mutual and conditional information.

Study targets:

Basics of probability theory and their application in statistical estimates and tests. The use of Markov chains in modeling. Basic notions of information theory.

Course outlines:

1. Basic notions of probability theory. Kolmogorov model of probability. Independence, conditional probability, Bayes formula.
2. Random variables and their description. Random vector. Probability distribution function.
3. Quantile function. Mixture of random variables.
4. Characteristics of random variables and their properties. Operations with random variables.
Basic types of distributions.
5. Characteristics of random vectors. Covariance, correlation. Chebyshev inequality. Law of large numbers. Central limit theorem.
6. Basic notions of statistics. Sample mean, sample variance.
Interval estimates of mean and variance.
7. Method of moments, method of maximum likelihood. EM algorithm.
8. Hypotheses testing. Goodness-of-fit tests, tests of correlation, non-parametic tests.
9. Discrete random processes. Stationary processes. Markov chains.
10. Classification of states of Markov chains.
11. Asymptotic properties of Markov chains. Overview of applications.
12. Shannon entropy. Entropy rate of a stationary information source.
13. Fundamentals of coding. Kraft inequality. Huffman coding.
14. Mutual information, capacity of an information channel.

Exercises outline:

1. Elementary probability.
2. Kolmogorov model of probability. Independence, conditional probability, Bayes formula.
3. Mixture of random variables. Mean. Unary operations with random variables.
4. Dispersion (variance). Random vector, joint distribution. Binary operations with random variables.
5. Sample mean, sample variance. Chebyshev inequality. Central limit theorem.
6. Interval estimates of mean and variance.
7. Method of moments, method of maximum likelihood.
8. Hypotheses testing. Goodness-of-fit tests, tests of correlation, non-parametic tests.
9. Discrete random processes. Stationary processes. Markov chains.
10. Classification of states of Markov chains.
11. Asymptotic properties of Markov chains.
12. Shannon entropy. Entropy rate of a stationary information source.
13. Fundamentals of coding. Kraft inequality. Huffman coding.
14. Mutual information, capacity of an information channel.

Literature:

[1] Papoulis, A.: Probability and Statistics, Prentice-Hall, 1990.
[2] Stewart W.J.: Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling. Princeton University Press 2009.
[3] David J.C. MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.

Requirements:

Linear Algebra, Calculus, Discrete Mathematics

Note:

A necessary condition for the assignment is active participation at seminars, successful test, and one homework. More info: http://cmp.felk.cvut.cz/~navara/psi/

Webpage:

http://cmp.felk.cvut.cz/~navara/psi/

Keywords:

probability theory, statistical estimate, hypotheses testing, Markov chain

Subject is included into these academic programs:

Program Branch Role Recommended semester
BPKYR1 Robotics P 3
BPKYR_BO Common courses P 3
BPKYR3 Systems and Control P 3
BPKYR2 Sensors and Instrumentation P 3
BPOI2 Computer and Information Science P 3
BPOI_BO Common courses P 3
BPOI1 Computer Systems P 3
BPOI3 Software Systems P 3
BPKME_BO Common courses V 3
BPKME5 Komunikace a elektronika V 3
BPKME4 Network and Information Technology V 3
BPKME3 Applied Electronics V 3
BPKME2 Multimedia Technology V 3
BPKME1 Communication Technology V 3
BPEEM1 Applied Electrical Engineering V 3
BPEEM_BO Common courses V 3
BPEEM2 Electrical Engineering and Management V 3
BMI(ECTS) Manager Informatics V 3
BWM(ECTS) Web and Multimedia V 3
BIS(ECTS) Intelligent Systems V 3
BSI(ECTS) Software Engineering V 3


Page updated 13.5.2019 17:52:55, semester: Z,L/2020-1, L/2019-20, Z,L/2018-9, Z/2019-20, Send comments about the content to the Administrators of the Academic Programs Proposal and Realization: I. Halaška (K336), J. Novák (K336)