MLA 005 Shapes and Sizes: Tensors and NDArrays
Machine Learning Guide
How to Transpose a NumPi Dot Array
The embedding dimension is the number of elements in a vector. The word and coatings, the number of columns t s to incode a word using the word tovec model that i discussed in the n l p episodes. And we pipe that into our recurrent nule network. Oftentimes what you'll find is these last two dimensions, the sentence length,. What's often called the time steps. But swap axes allows you to swap any two axes at once.
Explains the fundamental differences between tensor dimensions, size, and shape, clarifying frequent misconceptions—such as the distinction between the number of features (“columns”) and true data dimensions—while also demystifying reshaping operations like expand_dims, squeeze, and transpose in NumPy. Through practical examples from images and natural language processing, listeners learn how to manipulate tensors to match model requirements, including scenarios like adding dummy dimensions for grayscale images or reordering axes for sequence data.
Links- Notes and resources at ocdevel.com/mlg/mla-5
- Try a walking desk stay healthy & sharp while you learn & code
-
Tensor: A general term for an array of any number of dimensions.
- 0D Tensor (Scalar): A single number (e.g., 5).
- 1D Tensor (Vector): A simple list of numbers.
- 2D Tensor (Matrix): A grid of numbers (rows and columns).
- 3D+ Tensors: Higher-dimensional arrays, such as images or batches of images.
-
NDArray (NumPy): Stands for "N-dimensional array," the foundational array type in NumPy, synonymous with "tensor."
- Number of nested levels in the array (e.g., a matrix has two dimensions: rows and columns).
- Access in NumPy: Via .ndim property (e.g., array.ndim).
- Total number of elements in the tensor.
- Examples:
- Scalar: size = 1
- Vector: size equals number of elements (e.g., 5 for [1, 2, 3, 4, 5])
- Matrix: size = rows × columns (e.g., 10×10 = 100)
- Access in NumPy: Via .size property.
- Tuple listing the number of elements per dimension.
- Example: An image with 256×256 pixels and 3 color channels has shape = (256, 256, 3).
- CSV/Spreadsheet Example: Dataset with 1 million housing examples and 50 features:
- Shape: (1_000_000, 50)
- Size: 50,000,000
- Image Example (RGB): 256×256 pixel image:
- Shape: (256, 256, 3)
- Dimensions: 3 (width, height, channels)
- Batching for Models:
- For a convolutional neural network, shape might become (batch_size, width, height, channels), e.g., (32, 256, 256, 3).
- The term "dimensions" in data science often refers to features (columns), but technically in tensors it means the number of structural axes.
- The "curse of dimensionality" often uses "dimensions" to refer to features, not tensor axes.
-
Adding Dimensions:
- Useful when a model expects higher-dimensional input than currently available (e.g., converting grayscale image from shape (256, 256) to (256, 256, 1)).
- Use np.expand_dims or array.reshape.
-
Removing Singleton Dimensions:
- Occurs when, for example, model output is (N, 1) and single dimension should be removed to yield (N,).
- Use np.squeeze or array.reshape.
-
Wildcard with -1:
- In reshaping, -1 is a placeholder for NumPy to infer the correct size, useful when batch size or another dimension is variable.
-
Flattening:
- Use np.ravel to turn a multi-dimensional tensor into a contiguous 1D array.
- Transposing Axes:
- Needed when model input or output expects axes in a different order (e.g., sequence length and embedding dimensions in NLP).
- Use np.transpose for general axis permutations.
- Use np.swapaxes to swap two specific axes but prefer transpose for clarity and flexibility.
- In NLP sequence models:
- 3D tensor with (batch_size, sequence_length, embedding_dim) might need to be reordered to (batch_size, embedding_dim, sequence_length) for certain models.
- Achieved using: array.transpose(0, 2, 1)
- reshape: General function for changing the shape of a tensor, including adding or removing dimensions.
- expand_dims: Adds a new axis with size 1.
- squeeze: Removes axes with size 1.
- ravel: Flattens to 1D.
- transpose: Changes the order of axes.
- swapaxes: Swaps specified axes (less general than transpose).
- A deep understanding of tensor structure - dimensions, size, and shape - is vital for preparing data for machine learning models.
- Reshaping, expanding, squeezing, and transposing tensors are everyday tasks in model development, especially for adapting standard datasets and models to each other.
The AI-powered Podcast Player
