The Difference Between Supervised Time Series Stuff and Reinforcement Learning

Try a walking desk to stay healthy while you study or work!

Notes and resources at  ocdevel.com/mlg/23 

• Vanilla Neural Networks (Feedforward Networks):

  • Used for general classification or regression tasks.
  • Examples include predicting housing costs or classifying images as cat, dog, or tree.

• Convolutional Neural Networks (CNNs):

  • Primarily used for image-related tasks.

• Recurrent Neural Networks (RNNs):

  • Used for sequence-based tasks such as weather predictions, stock market predictions, and natural language processing.
  • Differ from feedforward networks as they loop back onto previous steps to handle sequences over time.

• Supervised vs Reinforcement Learning:

  • Supervised learning involves training models using labeled data to learn patterns and create labels autonomously.
  • Reinforcement learning focuses on learning actions to maximize a reward function over time, suitable for tasks like gaming AI but less so for tasks like NLP.

• Encoder-Decoder Models:

  • These models process entire input sequences before producing output, crucial for tasks like machine translation, where full context is needed before output generation.
  • Transforms sequences to a vector space (encoding) and reconstructs it to another sequence (decoding).

• Gradient Problems & Solutions:

  • Vanishing and Exploding Gradient Problems occur during training due to backpropagation over time steps, causing information loss or overflow, notably in longer sequences.
  • Long Short-Term Memory (LSTM) Cells solve these by allowing RNNs to retain important information over longer time sequences, effectively mitigating gradient issues.

• An LSTM cell replaces traditional neurons in an RNN with complex machinery that regulates information flow.
• Components within an LSTM cell:
  • Forget Gate: Decides which information to discard from the cell state.
  • Input Gate: Determines which information to update.
  • Output Gate: Controls the output from the cell.