The Challenge Is How to Write a Data Flow Graph

PyTorch programmers are writing code in PyTorch to a programming model that looks like standard programming, they're describing what they're doing in terms of graphs. The number of operations they do is actually quite small matrix multiply, convolution, some version of a relu, glu, you know, softmax,. There's a bunch of tensor modifications, but the number of operators in that space is low. So the challenge is like, how do we come up with a program that can go fast and not have to do things like reading, write all the data, the memory all the time?