Yannic Kilcher Videos (Audio Only)

Neural Networks are Decision Trees (w/ Alexander Mattick)

Oct 23, 2022

31:50

Creator website

Episode guests

Alexander Mattick

Highlights

AI Chapters

Episode notes

How to Rewrite a Neural Network

01:35

Is the Depth of a Decision Tree a Neural Network?

02:00

GPUs Don't Do Well With Lots of Decision Making

01:26

A Neural Network Is a Fixed Skeleleton With Lots of Blank Spaces

03:47

Introduction

2min

Is There a Linearity in a Neural Network?

6min

Neural Networks and Decision Trees Using a Theorem

3min

Is the Depth of a Decision Tree a Neural Network?

3min

A Neural Network Is a Fixed Skeleleton With Lots of Blank Spaces

4min

The Proof That Neural Networks Are Decision Trees

2min

Neural Networks Mapping Well to Decision Trees

2min

The Decision Tree and the Neural Networks

4min

GPUs Aren't Suited to GPUs

5min

10.

The Self-Supervised Approach to Learning

2min

#neuralnetworks #machinelearning #ai

Alexander Mattick joins me to discuss the paper "Neural Networks are Decision Trees", which has generated a lot of hype on social media. We ask the question: Has this paper solved one of the large mysteries of deep learning and opened the black-box neural networks up to interpretability?

OUTLINE:

0:00 - Introduction

2:20 - Aren't Neural Networks non-linear?

5:20 - What does it all mean?

8:00 - How large do these trees get?

11:50 - Decision Trees vs Neural Networks

17:15 - Is this paper new?

22:20 - Experimental results

27:30 - Can Trees and Networks work together?

Paper: https://arxiv.org/abs/2210.05189

Abstract:

In this manuscript, we show that any feedforward neural network having piece-wise linear activation functions can be represented as a decision tree. The representation is equivalence and not an approximation, thus keeping the accuracy of the neural network exactly as is. We believe that this work paves the way to tackle the black-box nature of neural networks. We share equivalent trees of some neural networks and show that besides providing interpretability, tree representation can also achieve some computational advantages. The analysis holds both for fully connected and convolutional networks, which may or may not also include skip connections and/or normalizations.

Author: Caglar Aytekin

Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2

Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m

Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.