How to Make a Targeted Edit to a Model

We've been looking at making it possible for a practitioner to quickly edit a model. We collect a small dataset, much smaller than the kind of the dataset the original model was trained on. That gives examples of how we want to edit the model. One approach that we've taken is to have the editor actually directly change the weights of the model with a rank one update. A rank one update to each of the weights makes it so that we don't actually have to output something as large as the entire weight vector of the neural network. It can actually output something much smaller than it can expand it.