Tensor API guidelines

`Tensor` API guidelines

Tensor APIs are divided among computed properties, instance methods, and computed properties.

Here are the current rules:

Computed properties provide the most fluent, property-like usage syntax.
- These are used for basic Tensor properties, like Tensor.rank, Tensor.shape, and Tensor.scalarCount.
Instance methods are fluent but not property-like, because they must be invoked with parentheses: x.instanceMethod(). The extra parentheses convey that the operation is non-trivial and not a basic property access. Instance methods are used for:
- (Seemingly) all math operations that aren't top-level functions, e.g. Tensor.transposed(), Tensor.squared().
- Tensor-to-scalar reduction operations, e.g. Tensor.sum(), Tensor.mean(), Tensor.any(). These reduction operations also have variants taking reduction dimensions, e.g. Tensor.sum(alongAxes:) and Tensor.sum(squeezingAxes:).
Top-level functions are not fluent.
- These are used for most element-wise operations, e.g. abs(_:), exp(_:), mean(_:), pow(_:_:). These are distinct from tensor-to-scalar reduction ops of the same name: mean(_:) is element-wise mean, while Tensor.mean() is scalar-reduction mean.
- These are also used for more complex, domain-specific operations that can't be described using a single short verb, and for which a fluent API is not desirable. For example, linear algebra operations like cholesky(_:) and qrDecomposition(_:fullMatrices:) fit in this category.

🚧 Rules are open for discussion and subject to change!

Math operations are defined as top-level functions or instance methods, and rarely as computed properties.

The Tensor APIs are consistent with this prior art: no math operations are defined as computed properties.

Python libraries also face the decision of whether to define an API as a top-level function or an instance method.

Classic TensorFlow defines almost everything as a top-level function, e.g. tf.add.

PyTorch uses both torch top-level functions and torch.tensor methods. torch.tensor methods are defined in terms of corresponding torch top-level functions.