Instances for Future

[ChristopherDavenport]

Future is largely known to not be referentially transparent. As it starts execution immediately and caches the result.

While it may have been largely necessary for these instances to be in cats before cats-effect, with that solution coming in maturity it may be time for Future instances to be pushed to alleycats.

Imports

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import cats.implicits._ 

Example 1 - Prints Once on the first line

val future1 : Future[Unit] = Future(println("I printed"))
val futureResult1 : Future[Unit] = (future1, future1).mapN(_ => ())

Example 2 - Prints twice on the second line

def future2: Future[Unit] = Future(println("I printed"))
val futureResult2 : Future[Unit] = (future2, future2).mapN(_ => ())

As you can see we have lost equational reasoning.

What are your thoughts on the future of Future?

[djspiewak]

Instances like Monad[Future] and Applicative[Future] are really really dangerous. They've caused problems in scalaz for years. They should definitely be in alleycats, if we have them at all.

[johnynek]

My view is that any instance that is not lawful should not be in cats proper.

In this case, you have treated Future as though it were a Sync instance, which it is not. You put in side effects, print, and had some idea in mind. I think you can come up with other similar examples with other types with call by name arguments. If users see a call by name parameter as an invitation to put a side effect we have a big problem since we often use them just for short circuiting. I argue that is the fault of the user.

I advocate that we stick to the laws and argue for instances based on that. I don’t want cats to pick on some data types, whose instances are lawful, because they have other properties we don’t like.

E.g. I don’t like that scala uses universal equality and hashing for Map by default. However, I still think Map should have a Functor instance, which is lawful.

[tpolecat]

I agree in principle that it's "user error", but the error is committed when using the data types as intended. Future instances are fine if you never side-effect; Try instances are fine if you never throw; cigarettes are fine if you never smoke them. I think we can identify a qualitative difference here that justifies warning labels.

As a practical matter I think this hits beginners hardest, because experienced cats users are unlikely to use these data types at all. Conversations about Future on Gitter invariably lead to "that's not going to work, sorry, have a look at IO" which makes me worries about those who don't ask. My guess is these instances are used almost always by beginners, and almost always incorrectly.

So anyway I think a speed bump is warranted and I also vote for moving these to alleycats. While I'm thinking about it we should move the alleycats README onto the website so the warnings are louder.

[johnynek]

They were in alleycats, we had this discussion, we moved them back. Do we keep having this discussion every year or so and flip flopping? I guess anyone can do a PR and now cite the prior view they agree with and get two cats maintainers to ship it and we can switch.

The catch is we can only flip flop on binary incompatible releases.

PS: I really disagree with the Try example even more. cats-effect IO catches in similar places, but we are arguing to use that. throw is not a function, so trying to argue what is right from first principles does not make a lot of sense to me.

[johnynek]

One more thing: @tpolecat , I feel like your point is true about anything less than always doing pure functional programming, which I like, but cats has not been so dogmatic in the past. For instance, even in cats-effect we recently added non-referentially-transparent methods to allocate deffered instances:

https://github.com/typelevel/cats-effect/blob/master/core/shared/src/main/scala/cats/effect/concurrent/Deferred.scala#L100

The motivation is performance and not forcing flatMaps where a user is already inside a delay/suspend operation.

I think it is hard to not be an extremist: we have to have some answer other than: "all the way, all the time", but dealing with non-pure-FP is something scala programmers I think have to live with.

Lastly, I am not arguing that you should write unsafe code, I am arguing that sometimes when writing safe code the compiler can't help you much, and you have to provide the guarantees yourself. When the compiler's guarantee come with large runtime costs (due to boxing, for instance), this problem is more acute.

[tpolecat]

I think it is hard to not be an extremist

My thinking is that allowing the instances is the extremist position. I want to make things safer for people who aren't 100% there yet.

[djspiewak]

I think the problem I have with Monad[Future], even beyond situations where people side-effect, is the type is simply not referentially transparent. The canonical example:

for {
  a <- Future(computeA)
  b <- Future(computeB)
} yield a + b

// vs

val aF = Future(computeA)
val bF = Future(computeB)

for {
  a <- aF
  b <- bF
} yield a + b

The differences here are observable even without side-effects simply because the computation time will be radically different. Normally I'm of the school of thought that doesn't consider CPU/memory/stack to be effects worth controlling, but Future is by definition a datatype which controls CPU and stack. That's its whole purpose, especially if you don't use it to capture classical side-effects.

So if we start from the premise that side-effect-free Future isn't just an alias for Identity but is instead still serving a purpose, and that purpose is managing CPU scheduling of computation, then its non-referentially-transparent nature relative to this purpose is just as problematic as it is relative to side-effect capture. In other words, Future simply isn't a Monad because expressions which use it are not functions. This is different from Try in that we can't say anything analogous to "don't use throw and everything is fine." Here, the best we could do is say "don't use Future and everything is fine", but if that's the best we can do, then why are we giving people a Monad[Future]?

It should be removed. If we really really want to have it, then it should be in alleycats.

[oscar-stripe]

@djspiewak I disagree with your example. If you have no side effects, those two are the same. If you are expecting runtime to be the same for all rewrites of the code, that is another property not referential transparency. Note, checking the clock is an effect, so if you are timing things, you are back to an effect space. I don't recall any time we have previously argued that substitutions must have the same runtime, if so, use of call-by-name is problematic in almost every case since it will often change the runtime of the code.

[djspiewak]

@oscar-stripe So your argument then is that Future = Identity? If you ignore the effect of controlling CPU scheduling, there is no measurable difference between the types.

[tpolecat]

I think Future = Try = Id as far as equational properties are concerned. For pure computations they are interchangeable.

[johnynek]

@tpolecat yeah, I agree with that take.

Note: I work often in data-engineering areas, where we use Future to get concurrency to do computations in parallel: aggregate a tree of operations, not stateful side effects in many cases. (we actually use Twitter Future, which doesn't have the same Future.apply, you have to explicitly use a FuturePool, which is a bit clearer).

[djspiewak]

@johnynek @tpolecat If they're all equivalent, then why have them all? Why are people wanting to use Future at all? Just use Id. Id has a Monad. If you don't need Future (because it's just Id) then clearly we don't need Monad[Future].

The point is that they're not equivalent. And critically, they're not equivalent in exactly the way that Future is also not referentially transparent. The reason people want to use Future in the first place is, as you said, to get concurrency to do computations in parallel. However, as my example showed, this is precisely the way in which Future fails as an abstraction! The lack of referential transparency shows up precisely in the concurrency and scheduling (assuming you're not being evil and capturing side-effects).

So either you think that Future = Id, in which case there's no need for Monad[Future] (just use Monad[Id]), or you recognize the distinction between the two, which is CPU/stack usage, and Monad[Future] would be unsound because Future is not referentially transparent under that rubric. Either way, it's not a monad.

A related argument is asking whether or not Parallel is actually necessary when we have Applicative. The inconsistency between ap and flatMap is irrelevant if we're ignoring side-effects and CPU/stack. Future's referential opacity is the same thing.

I think Future = Try = Id as far as equational properties are concerned.

To say this, you kind of need to define what you mean by "equal" in the context of your equational reasoning. Clearly, they are different expressions that will evaluate quite differently, so they can't be equal in every conceivable sense (unlike, say, 1 + 2 = 2 + 1, which most definitely is equal in every conceivable sense). My argument is that, for Future, any notion of equality or substitutability must take into account CPU/stack and concurrency impact, because without taking those things into account, Future as a thing is pointless.