Generalize matrix function signatures

[t4c1]

Description

Matrix functions can be generalized to work with Eigen expressions not just matrices. By propagating matrix expressions around we can avoid creating temporary matrices, improving performance. This could also make adding support for sparse matrices easier.

There was some discoussion about this on discourse. Also an abandoned PR #1309. Contrary to this PR I intend to split the this into smaller chunks:

• element-wise functions (ones that use apply_scalar_unary). Prim implementation of these can in many cases use Eigen implementations that can be faster than std.
• operator-like functions (add, subtract, multiply, divide, elt_multiply, elt_divide)
• size and view functions (col, row, sub_col, sub_row, head, tail, diagonal, transpose, cols, rows, num_elements, dims)
• reminder of */fun
• */prob
• allow functions to return expressions

Example

Instead of :

Eigen::MatrixXd f(const Eigen::MatrixXd& x)

We can do:

template<typename T>
auto f(const T& x)

Expected Output

More general and possibly faster code.

Current Version:

v3.0.0

[bob-carpenter]

+1 for this. We also need to do this to allow support for sparse matrices. @SteveBronder has been working on that along with perfect forwarding so we can reduce copying and exploit move semantics wherever possible.

[andrjohns]

We'd need to start coding a bit more defensively after implementing this, otherwise there's the risk of accidentally evaluating the expression multiple times. Take log_sum_exp for example:

template <typename T>
auto log_sum_exp(T& x) {
  if (x.size() == 0) {
    return -std::numeric_limits<double>::infinity();
  }

  const double max = x.maxCoeff();
  if (!std::isfinite(max)) {
    return max;
  }
  return max + std::log((x.array() - max).exp().sum());
}

If log_sum_exp is passed an expression, something like log_sum_exp(m1.array().exp().matrix() * m2), then that expression will get evaluated three times: x.size(), x.maxCoeff(), and the final return statement. So we'd need to start evaluating the expression at the beginning of the function, and probably also updating the documentation so that other/new developers know why

[t4c1]

Right. I plan to do that in each function as I generalize its signature. At that time I can also make sure an expression is not used twice. Although I don't think just taking size of an expression evaluates it.

I also agree with the need to document this. Any idea where the doc should be put?

[bob-carpenter]

Absolutely. Is the pattern to do something like: Eigen::Matrix<T::Scalar, -1, 1> x_eval = x.eval() and then use that in the rest of the function? Evaluating `.size()` shouldn't need any evaluation. The expression template will already be evaluated and should at least know its size. `.maxCoeff()` is another story. So we'll have to dig in and see what's costly to evaluate and what's not. It would be a service to devs to update the developer-facing doc. I don't think this calls for any API-level doc changes other than of the acceptable input types for x here in log_sum_exp.

…

On Nov 29, 2019, at 6:05 PM, Andrew Johnson ***@***.***> wrote: We'd need to start coding a bit more defensively after implementing this, otherwise there's the risk of accidentally evaluating the expression multiple times. Take log_sum_exp for example: template <typename T> auto log_sum_exp(T& x) { if (x.size() == 0) { return -std::numeric_limits<double>::infinity(); } const double max = x.maxCoeff(); if (!std::isfinite(max)) { return max; } return max + std::log((x.array() - max).exp().sum()); } If log_sum_exp is passed an expression, something like log_sum_exp(m1.array().exp().matrix() * m2), then that expression will get evaluated three times: x.size(), x.maxCoeff(), and the final return statement. So we'd need to start evaluating the expression at the beginning of the function, and probably also updating the documentation so that other/new developers know why — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[bob-carpenter]

:-) I didn't see this before responding. We're on the same page w.r.t. size(). I think by "evaluate" we mean evaluate operator()(i, j) for every (i, j) pair. It always gets constructed.

…

On Nov 30, 2019, at 4:00 AM, Tadej Ciglarič ***@***.***> wrote: Right. I plan to do that in each function as I generalize its signature. At that time I can also make sure an expression is not used twice. Although I don't think just taking size of an expression evaluates it. I also agree with the need to document this. Any idea where the doc should be put? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[SteveBronder]

I have a little info on adding new docs to the stan site here

https://github.com/stan-dev/math/blob/develop/doxygen/howto_add_page.md

I wonder if for some functions we use so much info about the matrices it would be better to leave their template arguments as Matrix<T, R, C> just to force evaluation.

[t4c1]

If a function needs to evaluate an argument and the template params for that arguments are known (not templated) we can just leave that argument as is - const Matrix<T, R, C>&. However that does not work if we need template deduction at the same time. In that case I intend to use what Bob suggested.

[andrjohns]

If a function needs to evaluate an argument and the template params for that arguments are known (not templated) we can just leave that argument as is - const Matrix<T, R, C>&

You could also use the Ref<> functionality to maintain generality of input types, which (AFAIK) will take expressions as inputs and evaluate them into temporaries as needed. Using an example from Eigen's doc:

// read-only const argument:
void foo2(const Ref<const VectorXf>& x);

MatrixXf A;

foo2(A.row());              // Compilation error because A.row() is a 1xN object while foo2 is expecting a Nx1 object
foo2(A.row().transpose());  // The row is copied into a contiguous temporary
foo2(2*a);                  // The expression is evaluated into a temporary
foo2(A.col().segment(2,4)); // No temporary

Although some of the copies might be too expensive to be worth it

[bob-carpenter]

First, I thought Ref didn't work with slicing, segmenting, etc., but it seems to work OK. Second, I thought it was type not size that determined passing a VectorXd, but the following compiles and runs.

What are the rules for executing into temporaries? I wouldn't have thought any of those template expressions would cause an intermediate.

void foo2(const Eigen::Ref<const Eigen::VectorXd>& x) {
  std::cout << x << std::endl;
}

TEST(foo, bar) {
  Eigen::MatrixXd A(2, 2);
  A << 1, 2, 3, 4;
  foo2(A.row(0).transpose());

  Eigen::VectorXd a(2);
  a << 10, 100;
  foo2(2 * a);
  foo2(A.col(0).segment(0, 1));

  Eigen::MatrixXd B(2, 1);
  foo2(B);
}

[t4c1]

This page says:

By default, a Ref can reference any dense vector expression of float having a contiguous memory layout. Likewise, a Ref can reference any column-major dense matrix expression of float whose column's elements are contiguously stored with the possibility to have a constant space in-between each column, i.e. the inner stride must be equal to 1, but the outer stride (or leading dimension) can be greater than the number of rows.

In the const case, if the input expression does not match the above requirement, then it is evaluated into a temporary before being passed to the function.

[bob-carpenter]

Thanks---that really clarifies what's going on. I wish we'd figured this out a few years ago!

…

On Dec 4, 2019, at 2:24 AM, Tadej Ciglarič ***@***.***> wrote: This page says: By default, a Ref can reference any dense vector expression of float having a contiguous memory layout. Likewise, a Ref can reference any column-major dense matrix expression of float whose column's elements are contiguously stored with the possibility to have a constant space in-between each column, i.e. the inner stride must be equal to 1, but the outer stride (or leading dimension) can be greater than the number of rows. In the const case, if the input expression does not match the above requirement, then it is evaluated into a temporary before being passed to the function. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[t4c1]

I am running out of ideas how to group remaining functions into PRs. So I will just go in alphabetical order.