IKT721 - Statistical Signal Processing

General Information

This PhD course explores estimation and detection/hypothesis testing theory. 

Schedule

The approximate schedule is available here.

Time

Lectures will generally take place in the following time slots (see details in the schedule above):

• Tue. 09:15 to 10:00
• Fri. 10:15 to 12:00, with a break from 11:00 to 11:15

Additionally, the schedule reserves the following time slot

• Tue. 10:15 to 11:00 

on certain days. No lectures will generally take place in these time slots. They will be used to recover lectures that are moved to accommodate the instructors' or attendees' agendas.

If you wish to move a lecture to a reserved time slot due to an academic reason (PhD forum, conference, clash with another course), please inform the instructors. 

10 minutes before the lecture, there will be a review on the previous material. Attendance to this part is optional. 

Location

Lectures will take place through zoom. The links are available on Canvas.

Material

The material is available here.

Reading

Main Resources

It is very useful to read the sections indicated in the above schedule before each lecture.

We will be mainly following two books:

• Kay, S. M. "Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory." PTR Prentice-Hall, Englewood Cliffs, 1993.

• Kay, S. M. "Fundamentals of Statistical Signal Processing, Volume II: Detection Theory." PTR Prentice-Hall, Englewood Cliffs, 1998.

If you have difficulties to obtain these books, please contact the instructors of the course. 

Complementary Resources

Lovers of mathematical elegance and beauty will find the book by Billingsley as the "Bible" of probability and measure theory. While not necessary to follow this course, background in measure theory is often required when solving research problems, reading papers, and, in general, to understand the foundations of probability theory and statistical inference.

• Patrick Billingsley. Probability and measure. John Wiley & Sons, 2008.

The following are two advanced texts which cover and expand some of the contents of the course from a statistics perspective.

• Lehmann and Casella. Theory of point estimation. Springer Science & Business Media, 2006.
• Lehmann and Romano. Testing statistical hypotheses. Springer Science & Business Media, 2006.

An easier text is the book by Scharf. Chapter 11 on modal analysis is especially useful for those students that need to do frequency estimation or angle-of-arrival estimation.

• Louis L. Scharf and Cédric Demeure. Statistical signal processing: detection, estimation, and time series analysis. Prentice Hall, 1991.

Since we do not have time to cover many of the specifics of time-series analysis, a good complement is the book by Stoica. Here, students can learn about stochastic processes, autocorrelations, power spectral densities, and so on.

• Petre Stoica and Randolph L. Moses. "Spectral analysis of signals." (2005).

Effort

This PhD course has 5 credit points, which amount to 125-150 hours of effort to pass the course.

This will include lectures as well as time for self-study, solving homework, and working on the project.

The average homework load will be one problem per week. In past years, problem sets with N problems were handled every N weeks. This proved ineffective for N>1 as many students waited until the last minute for solving the problems and, for this reason, they encountered difficulties to follow the course. Thus, we will stick to N=1 as much as possible.

Please inform the instructors if the amount of time you need to spend on this course significantly deviates from the aforementioned interval.

Evaluation

There are three possibilities, sorted next in increasing difficulty order:

• Attend at least 80% of the lectures. Submit the homework. Submit and pass the project description. Submit the final project. Final grade = Pass iff (homework_grade + project_grade)/2 ≥ B.
• Attend at least 80% of the lectures. Submit the homework. Take the take-home exam. Final grade = Pass iff (homework_grade + exam_grade)/2 ≥ B.
• Take a final exam. 

Homework Guidelines

• All answers must be properly justified.
• Please submit a clean and clear document. The grader will not spend time trying to guess what the student tried to say in the submission, meaning that what cannot be easily understood will not be graded. If your document is handwritten, make sure that your writing is easy to understand.
• It is recommended that you do not submit the last day. If some circumstance prevents you from submitting the last day, then you will not be able to submit.
• Humans make mistakes and therefore any researcher/scientist/engineer must develop techniques to prevent and detect mistakes. Performing each derivation or computation at least twice, for instance, or carrying out sanity checks helps detect mistakes. Being rigorous with notation and performing multiple small steps rather than few big steps helps prevent mistakes. To encourage students to implement the aforementioned techniques, the default criterion will be to assign a zero grade to answers containing basic math mistakes; e.g. issues with signs, constants, direction of inequalities, etc. Note that these errors are not admissible either on research papers or engineering projects. 

Honor Code

Submission of any assignment implies acceptation of this honor code. If you are not willing to accept it, please inform the instructors. 

It is not correct to share the solutions of problems with other students from other years. Neither give nor receive.

Homework assignments are individual unless stated otherwise. Students must not share solutions, neither give nor receive, before the submission deadline. Please think that you will be doing a favor neither to your colleagues nor to yourself.

Please note that it is easy for an instructor to detect if you have violated these rules. In that case, you will fail the course and will be reported to the University. 

Standard Notation

Documents: Boldface lowercase letters denote column vectors. Boldface uppercase letters denote matrices.

Handwritten math: Underlined minuscule letters denote column vectors. Matrices are denoted by capital letters.

Terminology

The word "data" comes from latin and originally it was the plural of the word "datum". It is still common to hear people saying "many data" or "we process a datum". However, in modern usage, the word "data" is uncountable and, therefore, singular. Thus, one can say phrases such as "too much data" or "very little data". The singular counterpart would be "data instance", "data vector", "point", "sample", etc. In this course, we stick to the contemporary usage. More information here.