ECE 3522: Stochastic Process in Signals and Systems

Course Syllabus
Contact Information:

Lecture	MWF: 9:00 AM - 9:50 AM   (ENGR 616)
Lecturer	Joseph Picone, Professor     Office: EA 703A     Office Hours: (MWF) 8 AM - 9 AM, 10 AM - 11 AM     Phone: 215-204-4841     Email: picone@temple.edu     Skype: joseph.picone
Social Media	https://www.facebook.com/groups/temple.engineering.ece3522/     temple.engineering.ece3522@groups.facebook.com
Email	http://groups.google.com/group/temple-engineering-ece3522     temple-engineering-ece3522@googlegroups.com
Website	http://www.isip.piconepress.com/courses/temple/ece_3522
Textbook	Oliver Ibe   Fundamentals of Applied Probability and Random Processes, 2nd Edition     Academic Press     July 7, 2014, 456 pages     ISBN: 978-0128008522     URL: Fundamentals of Applied Probability and Random Processes (2nd Edition)
Reference Textbooks	R.E. Walpole   Probability and Statistics for Engineers and Scientists, 9th Edition     Pearson     January 6, 2011, 816 pages     ISBN: 978-0321629111     URL: Probability and Statistics for Engineers and Scientists (9th Edition)     R.A. Bailey   Design of Comparative Experiments     Cambridge Series in Statistical and Probabilistic Mathematics (Book 25)     April 17, 2008, 346 pages     URL: Design of Comparative Experiments
Other Reference Materials	MATLAB: The Statistical Toolbox     The R Project for Statistical Computing     Online Statistics Education: An Interactive Multimedia Course of Study   Introduction to Probability, Statistics and Random Processes
Prerequisites	C- or better in ECE 3512

Lecture Grading Policies:

Item	Weight
Exam No. 1	10%
Exam No. 2	10%
Exam No. 3	10%
Final Exam	10%
Quizzes	20%
Computer Assignments	30%
Homework Assignments	10%
TOTAL:	100%

Statistical analysis has become integral to engineering. For example, most optimization techniques today are based on statistical measures that use generative or discriminative models rather than more traditional measures of error such as mean square error. In this course, you will be introduced to a wide variety of topics in statistics that can be applied to many different types of engineering problems (e.g., experimental design for quality control, time series prediction for signal modeling or financial analysis). To maximize your success in this course, you must follow a process that includes reading the textbook, working homework problems and verifying your solutions using computer simulations. Lectures are designed to introduce the essential theoretical concepts as well as a discussion of some applications. The computer assignments are designed to reinforce these concepts with practical data. We will make extensive use of MATLAB in this course.

We will have three in-class exams in this course and a comprehensive final. Each in-class exam will be closed books and notes. You will be allowed one page (double-sided) of notes for the in-class exams. For the final exam you will be allowed four pages of notes, presumably the same notes you used for the in-class exams. The exams will resemble the homework problems, so it is important that you thoroughly study the homework problems.

Unannounced quizzes will be given periodically throughout the course to encourage you to attend lecture classes and keep up with the daily work. If you miss a quiz without a prior excuse from the instructor, you receive a zero for that quiz with no exception. Make-up quizzes will not be given. The same policy applies to in-class exams and the final exam as well.

Detailed homework solutions will be prepared in an 8.5"x11" notebook. You are required to use a three-ring binder with tab dividers labeled by the homework assignment number and type (e.g., "HW #7" or "CA #3"). These will be turned in during each exam and graded. Students are expected to prepare detailed solutions that include a problem statement and a well-documented step-by-step solution. Students can collaborate on homework solutions, but the solutions you provide must be unique. Grading will take into account the accuracy of your solution as well as the quality of your explanation. Simply providing answers with no explanations gets no credit.

There will be weekly computer assignments that involve MATLAB. These will be based on real world data. A template for the solutions to be turned in is provided here: here. You must conform to the template provided. The easiest way to do this is to start an assignment by editing the template provided and to use the format painter tool in Word. Consult with your Technical Communications instructor if you have questions about how to do various things in Word or have questions about what is expected for content.

Lecture Schedule:

The lecture component of ECE 3522 meets three times a week and will cover the following topics:

Class	Date	Sections	Topic(s)
1	01/12	Sects. 1.1 - 1.6	Basic Probability Concepts:       Sample Space and Events       Elementary Set Theory       Probability Definitions and Properties
2	01/14	Sects. 1.7 - 1.9	Basic Probability Concepts:       Conditional Probabilities       Total Probability       Bayes Theorem       The Binary Symmetric Channel       Tree Diagrams       Independent Events
3	01/16	Sects. 1.10 - 1.13	Basic Probability Concepts:       Combined Experiments       Combinatorial Analysis       Reliability Theory       Quiz
--	01/19	Martin Luther King Jr. Day
4	01/21	Sects. 2.1 - 2.5	Random Variables:       Definition of a Random Variable       Distribution Functions       Discrete Random Variables
5	01/23	Sects. 2.6 - 2.7	Random Variables:       Continuous Random Variables       Examples       Quiz
6	01/26	Sects. 3.1 - 3.4	Moments and Expectations:       Averages and Expectations       First-Order Moments       General Moments       Variance and Standard Deviation
7	01/28	Sects. 3.5 - 3.8	Conditional Expectations:       Conditional Events       The Markov Inequality       The Chebyshev Inequality       Examples
8	01/30	Sects. 4.1, 4.10, 4.11	Special Probability Functions:       The Uniform Distribution       The Normal (Gaussian) Distribution       Quiz
9	02/02	Sects. 4.2 - 4.9, 4.12 - 4.14	Special Probability Functions:       Bernoulli Trials       Binomial Distribution       Exponential Distribution
10	02/04	Sects. 5.1 - 5.5	Multiple Random Variables:       Joint CDFs       Discrete Bivariate Random Variables       Computations and Examples
11	02/06	Sects. 5.6 - 5.10	Multiple Random Variables:       Conditional Distributions and Probabilities       Covariance       Multivariate Random Variables       Quiz
12	02/09	6.1 - 6.3	Functions of Random Variables:       Functions of Random Variables       Expectations of Random Variables       Expectation of a Conditional Expectation
13	02/11	Exam No. 1	Chapters 1 - 4
14	02/13	Sects. 6.4 - 6.7	Functions of Random Variables:       Sums of Independent Random Variables       Minimum of Two Independent Random Variables       Maximum of Two Independent Random Variables
15	02/16	Sects. 6.8 - 6.11	Functions of Random Variables:       Two Functions of Two Random Variables       A Sum of Two Correlated Random Variables       Weak Law of Large Numbers       Strong Law of Large Numbers       The Central Limit Theorem       Order Statistics
16	02/18	Sects. 7.1 - 7.3	Transform Methods:       Characteristic Functions       Moment Generating Property       Sums of Independent Random Variables       The Laplace Transform
17	02/20	Sects. 7.4, 7.5	Transform Methods:       The z-Transform       Moment Generating Property       Sums of Independent Random Variables       Examples       Quiz
18	02/23	Sects. 8.1 - 8.5	Descriptive Statistics:       Measures of Central Tendency       Measures of Dispersion       Visualizations
19	02/25	Sects. 8.6, 8.7	Descriptive Statistics:       Skewness       Peakedness       Examples       More Visualizations
20	02/27	Sects. 9.1, 9.2	Inferential Statistics:       Sampling Theory       The Sample Mean       Variance of the Sample Mean       The Sample Variance       Sampling Distributions       Quiz
--	03/02	Spring Break
--	03/04	Spring Break
--	03/06	Spring Break
21	03/09	Sects. 9.3	Inferential Statistics:       Estimation Theory       Unbiased Estimaters       Efficient Estimators       Consistent Estimators       Confidence Intervals       Maximum Likelihood Estimation
22	03/11	Sects. 9.4	Inferential Statistics:       Maximum Likelihood Estimation       Minimum Mean Squared Error Estimation       Hypothesis Testing       Procedure       Type I and II Errors
23	03/13	Sect. 9.5	Inferential Statistics:       One-Tailed and Two-Tailed Tests       Example       Quiz
24	03/16	Sect. 9.5	Inferential Statistics:       Regression Analysis       Multivariate Linear Regression       Review
25	03/18	Sects. 10.1 - 10.4	Random Processes:       Characterization       Mean and Autocorrelation       Autocovariance       Crosscorrelation and Crosscovariance
26	03/20	Exam No. 2	Chapters 5 - 8
27	03/23	Sects. 10.6	Random Processes:       Stationarity       Autocorrelation Properties       Covariance Properties
28	03/25	Sect. 10.7	Random Processes:       Ergodicity       Power Spectral Density       White Noise
29	03/27	Sect. 10.8	Random Processes:       Discrete-Time Random Processes       Autocorrelation       Covariance       Power Spectral Density
30	03/30	Secs. 11.1 - 11.4	Linear Systems:       Deterministic Inputs       Continuous-Time Inputs       Discrete-Time Inputs       Power Spectral Density (Revisited)
31	04/01	Sects. 11.5, 11.6	Linear Systems:       Moving Average Processes       Autoregressive Moving Average Processes       Filters       Frequency Domain Analysis
32	04/03	Sects. 12.1 - 12.5	Special Random Processes:       Bernoulli Processes       Random Walk       Gaussian Process       Poisson Processes
33	04/06	Notes	Principal Components Analysis:       Covariance       Eigenvalues and Eigenvectors       Whitening Transformations
34	04/08	Notes	Maximum Likelihood Classification:       The Binary Symmetric Channel       The Two-Class Problem       Maximum Likelihood       Threshold Decoding
35	04/10	Notes	Maximum Likelihood Classification:       2D Classification Examples       Visualization of Variance       Overlappling Gaussians       Other Special Distributions
36	04/13	12.6, 12.7	Hidden Markov Models:   x     Markov Processes       First-Order Markov Processes       Observable Models       Examples of Hidden Models
37	04/15	12.8 - 12.10	Hidden Markov Models:       Definitions       Basic Calculations       Three Related Challenges       Parameter Estimation       Continuous Distributions       Gaussian Mixture Models
38	04/17	Exam No. 3	Chapters 9 - 12.5
39	04/20	Special Topics	Clustering:       Hierarchical Approaches       Agglomerative Clustering       K-Means Clustering       Examples
40	04/22	Special Topics	Clustering:       Quality Control Example       Parameter Estimation       Maximum Likelihood Classification
41	04/24	Special Topics	Clustering:       Gaussian Mixture Models       The Expectation Maximization Theorem       Examples
42	04/27	Special Topics	Fun With Statistics:       Game Shows       Coin Tosses       Signal Processing
43	05/04	Final Exam (08:00 - 10:00 AM)	Special Topics (Comprehensive)

Please note that the dates above are fixed since they have been arranged to optimize a number of constraints. If you have conflicts with other classes, such as too many exams within the same week, we need to resolve that the first week of classes.

Homework:

The textbook homework schedule is as follows:

HW	Due Date	Item(s)
1	01/16	1.2, 1.14, 1.16, 1.20, 1.21, 1.27, 1.34, 1.38, 1.46, 1.49
2	01/23	2.2, 2.4, 2.9, 2.15, 2.19, 2.21, 2.25, 2.34
3	01/30	3.1, 3.5, 3.8, 3.10, 3.18, 3.20, 3.24, 3.25, 3.26, 3.27
4	02/06	4.3, 4.6, 4.8, 4.19, 4.44 4.48, 4.58, 4.59, 4.61, 4.63
5	02/13	5.1, 5.4, 5.9, 5.14, 5.16, 5.17, 5.18, 5.20, 5.22, 5.23
6	02/20	6.1, 6.2, 6.8, 6.9, 6.11, 6.17, 6.22, 6.26, 6.32, 6.35
7	02/27	7.1, 7.7, 7.9, 7.12, 7.17, 7.22
8	03/13	8.3, 8.6, 8.13
9	03/20	9.1, 9.7, 9.10, 9.13, 9.14, 9.18
10	03/27	10.2, 10.6, 10.12, 10.16, 10.18, 10.25, 10.29, 10.35, 10.45, 10.46
11	04/01	11.6, 11.8, 11.14, 11.17, 11.20, 11.23, 11.26, 11.30
12	04/08	12.7, 12.10, 12.12, 12.46, 12.51

The computer assignment schedule is as follows:

CA	Due Date	Item(s)
1	01/20	Simple Statistics
2	01/26	Regression and Histograms
3	02/02	Variance
4	02/09	Model Fitting
5	02/16	Covariance and Correlation
6	02/23	Means and Variances Revisited
7	03/09	Visualization
8	03/16	Central Limit Theorem
9	03/23	Signal to Noise Ratios and Filtering
10	03/30	Statistical Significance
11	04/06	Autocorrelation and Power Spectral Density
12	04/13	Principal Components Analysis
13	04/20	Maximum Likelihood Classification
14	04/27	TBD

University Policy Statements:

• Any student who has a need for accommodation based on the impact of a disability should contact me privately to discuss the specific situation as soon as possible. Contact Disability Resources and Services at 215-204-1280 in 100 Ritter Annex to coordinate reasonable accommodations for students with documented disabilities.


• Freedom to teach and freedom to learn are 2 inseparable facets of academic freedom. The University has a policy on Student and Faculty and Academic Rights and Responsibilities (Policy #03.70.02) which can be accessed through the following link: http://policies.temple.edu/getdoc.asp?policy_no=03.70.02.


• Please review the university's academic dishonesty policy Students who are caught plagiarizing or cheating will receive a grade of 0 on the corresponding assignment for a first offense. A second offense will result in failure of the course.