Foundations of Supervised Machine Learning with Graphs

Graph-structured data is ubiquitous across application domains ranging from chemo- and bioinformatics to image and social network analysis. To develop successful machine learning algorithms, we need techniques that map the rich information inherent in the graph structure to a vectorial representation in a meaningful way—so-called graph embeddings. Designing such embeddings comes with unique challenges. The embedding has to account for the complex structure of (real-world) networks and additional high-dimensional continuous vectors attached to nodes and edges in a (permutation) invariant way while being scalable to massive graphs or sets of graphs. Moreover, when used in supervised machine learning, the model trained with such embeddings must generalize well to new or previously unseen (graph) instances. Hence, more abstractly, designing graph embeddings results in a trade-off between (1) expressivity, (2) scalability, and (3) generalization. In this seminar, we want to discuss the current progress on the theoretical foundations of machine learning on graphs, penetrating the above-listed three challenges.

Requirements for Passing

To pass the seminar, you need to fulfill the following:
  1. Give a 30-minute-long talk about your assigned paper.
  2. Write a 12- to 15-page (excluding title page) detailed report about your assigned paper.
  3. Peer-review your fellow students' reports.
  4. Attend all meetings and actively participate; see below for dates.

Talks

At the end of the semester, each student will give a 30-minute-long talk about their assigned paper. You should provide an overview of your choosen/assigned paper and highlight the most important concepts and ideas. Ideally, your presentation should give the audience (i.e., your fellow students) a good understanding of your assigned paper.

Reports

The report gives a detailed overview of the choosen/assigned paper. The required report length is 12 to 15 pages, using the provided LaTeX template. This means that after you get your paper assigned, you write your report and submit it for "peer review" by your fellow students. You will receive constructive feedback to improve the paper; afterward, you will receive additional feedback from the seminar organizers. You can then submit an updated, final version, which will be graded. Note that this means that you will also have to write some short reviews on the reports by your fellow students.

Organization

  1. More details are given during the mandatory kick-off meeting.
  2. Papers will be assigned after the kick-off meeting.
  3. The long talks will be presented in day-long block seminar.
  4. All meetings (kick-off, peer-review, and final talks) will take place in Room 228, Theaterstraße 35 - 39.

Dates

Date
02.10.2023, 10:00   Kick-off meeting (in person).
03.11.2023, 24:00 Submission of report drafts.
01.12.2023, 24:00 Submission of reports for peer review.
14.12.2023, 24:00 Submission of peer reviews.
18.12.2023, 12:00 Discussion of peer reviews (in person).
09.01.2024, 24:00 Submission of reports.
16.01.2024, 24:00 Feedback by the organizers.
31.01.2024, 24:00 Submission of of presentation slides.
09.02.2024, 24:00 Submission of final reports.
19.02.2024, 10:00 Talks (in person).

Papers

The papers can be chosen from the following list.
  1. Tree Mover’s Distance: Bridging Graph Metrics and Stability of Graph Neural Networks
  2. Exponentially Improving the Complexity of Simulating the Weisfeiler-Lehman Test with Graph Neural Networks
  3. Graph Inductive Biases in Transformers without Message Passing
  4. PLANE: Representation Learning over Planar Graphs
  5. A Theory of Link Prediction via Relational Weisfeiler-Leman
  6. Rethinking the Expressive Power of GNNs via Graph Biconnectivity
  7. The Power of Recursion in Graph Neural Networks for Counting Substructures
  8. Pure Transformers are Powerful Graph Learners
  9. Optimality of Message-Passing Architectures for Sparse Graphs